Label encoding assigns each category an integer (0, 1, 2, ...). It's simple and memory-efficient — but using it on the wrong kind of feature quietly teaches a model a false ordering.
Python Implementation
from sklearn.preprocessing import LabelEncoder
sizes = ["Small", "Medium", "Large", "Medium", "Small"]
encoder = LabelEncoder()
encoded = encoder.fit_transform(sizes)
print(encoded) # [2 1 0 1 2] — alphabetical order by default!
print(encoder.classes_) # ['Large' 'Medium' 'Small'] — the mapping orderExpected output: LabelEncoder assigns integers in alphabetical order of the category names by default — not in any meaningful size order. For a genuinely ordered feature like this, use Ordinal Encoding with an explicit category order instead.
Where Label Encoding Is Actually Correct
- Encoding the target variable for classification —
y = ["cat", "dog", "cat"]becomes[0, 1, 0]; there's no ordering implication a classifier would misuse, since class labels are treated as discrete categories, not a number line. - Tree-based models (Decision Tree, Random Forest, XGBoost) can sometimes tolerate label-encoded nominal features reasonably well, since trees split on thresholds per feature rather than assuming linear numeric relationships — though one-hot or target encoding is still often safer.
Where It Silently Breaks a Model
Label-encoding a nominal feature like city = ["Delhi", "Mumbai", "Chennai"] into [0, 1, 2] for a linear model implies Chennai (2) is "twice" Mumbai (1) in some numeric sense — which is meaningless, since city names have no inherent order. Linear/logistic regression, KNN and SVM can all be misled by this false numeric relationship.
Common Mistakes
- Using
LabelEncoderon an input feature with no natural order, instead of the target — this is the single most common misuse of this encoder. - Assuming the encoded integers preserve a meaningful order without explicitly checking
encoder.classes_.
Interview Relevance
Q: "What's the risk of label-encoding a nominal categorical feature?" It introduces a false numeric ordering/distance between categories that has no real meaning, which linear and distance-based models can pick up on as if it were a genuine pattern.
Practice Question
You label-encode a "payment_method" column (Cash, Card, UPI, Wallet) and train a logistic regression model. Explain why this could hurt model quality, and what encoding you'd use instead.
