One-Hot Encoding

One-hot encoding turns a categorical column into multiple binary (0/1) columns — one per category — so no false numeric ordering is implied between categories.

How It Works

Python Implementation

import pandas as pd
from sklearn.preprocessing import OneHotEncoder

df = pd.DataFrame({"city": ["Delhi", "Mumbai", "Chennai", "Delhi"]})

# Pandas — quick and readable for exploration
encoded = pd.get_dummies(df, columns=["city"], drop_first=True)
print(encoded)

# scikit-learn — fits into a Pipeline, and handles unseen categories at test time
encoder = OneHotEncoder(drop="first", handle_unknown="ignore", sparse_output=False)
encoded_array = encoder.fit_transform(df[["city"]])
print(encoder.get_feature_names_out())

Expected output: a DataFrame/array with one fewer column than unique categories (when drop_first=True/drop="first") — Delhi becomes the case where every dummy column is 0, avoiding redundant information.

The Dummy Variable Trap

If you keep all \(k\) category columns instead of \(k-1\), the columns become perfectly collinear (knowing any \(k-1\) of them tells you the last one exactly) — this destabilizes linear models' coefficients. drop_first=True (or drop="first") avoids this by dropping one redundant column; tree-based models don't have this issue and it's optional for them.

Why handle_unknown="ignore" Matters in Production

If a category appears at prediction time that the encoder never saw during training (a new city added after deployment), scikit-learn's OneHotEncoder will raise an error by default — handle_unknown="ignore" instead encodes it as all-zeros, letting the pipeline keep running instead of crashing in production.

Practical Use Cases

• Low-cardinality nominal features for linear models, SVM, and KNN
• Any feature where category order genuinely has no meaning

Advantages

• Introduces no false ordering between categories
• Works cleanly with linear models, distance-based models, and neural networks

Limitations

• Column count grows with the number of categories — impractical for high-cardinality features (see Categorical Data)
• Produces a sparse, mostly-zero matrix, which can be memory-intensive at scale

Common Mistakes

• Fitting OneHotEncoder separately on train and test sets — if the test set has categories the train set didn't, you get mismatched columns. Always fit on train only, then transform both.
• Forgetting drop_first/drop="first" when using a linear model, risking the dummy variable trap.

Interview Relevance

Q: "What happens if a new category appears in production that wasn't in your training data?" Without handle_unknown="ignore", scikit-learn's encoder raises an error; with it, the new category is encoded as all-zeros, letting the pipeline degrade gracefully instead of crashing.

Practice Question

A "payment_method" column has 4 categories. How many columns will pd.get_dummies(..., drop_first=True) produce, and which category does an all-zero row represent?

Related ML Notes

• Label Encoding
• Categorical Data
• Preprocessing Pipeline