Standardization (Z-score scaling) rescales a feature to have mean 0 and standard deviation 1 — it's the default, safest scaling choice for most ML algorithms.
Formula
\(x\) is the original value, \(\mu\) is the feature's mean, and \(\sigma\) is its standard deviation. The result, \(z\), tells you how many standard deviations \(x\) is from the mean — 0 means exactly average, +1 means one standard deviation above average.
Numerical Example
For a feature with \(\mu = 60\), \(\sigma = 10\), and \(x = 70\):
from sklearn.preprocessing import StandardScaler
import numpy as np
X_train = np.array([[50], [55], [60], [65], [70]])
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train) # learns mean & std FROM training data
print(scaler.mean_, scaler.scale_) # [60.] [7.07...]
print(X_train_scaled.ravel()) # roughly [-1.41 -0.71 0. 0.71 1.41]
X_test = np.array([[75]])
X_test_scaled = scaler.transform(X_test) # applies the SAME learned mean/std — no re-fittingWhy Standardization Is Usually the Default
- Doesn't compress the feature into a bounded range, so it's less sensitive to a few extreme outliers than Min-Max scaling
- The result has a directly interpretable meaning (standard deviations from average)
- Required for algorithms that assume roughly zero-centered input, like PCA and many regularized linear models
Common Mistakes
- Calling
fit_transform()on the test set — this recomputes a new mean/std from test data, defeating the purpose of consistent scaling and leaking test information. Usetransform()only for test data. - Standardizing one-hot encoded (0/1) columns — usually unnecessary and can make them harder to interpret; standardization is meant for continuous numeric features.
Interview Relevance
Q: "Why do you fit the scaler on the training set only?" Because the mean and standard deviation used for scaling should reflect only information the model is allowed to learn from — fitting on the full dataset (including test data) leaks test-set statistics into training, inflating evaluation results. See Data Leakage.
Practice Question
Given a feature with \(\mu = 100\), \(\sigma = 20\), compute the standardized value for \(x = 140\).
