Categorical Data

Categorical data represents groups or labels rather than numbers — city, product category, education level. Most ML algorithms are purely numeric under the hood, so categorical features must be encoded before a model can use them.

Nominal vs Ordinal — The Distinction That Decides Which Encoding to Use

Cardinality Matters

df["city"].nunique()        # low cardinality (e.g. 15 cities) -> one-hot encoding is fine
df["product_id"].nunique()  # high cardinality (e.g. 50,000 products) -> one-hot would explode column count

A "low-cardinality" categorical feature (few unique values) one-hot encodes cleanly. A "high-cardinality" feature (thousands of unique values — user IDs, product SKUs) one-hot encodes into an impractically wide, mostly-zero matrix; frequency encoding, target encoding, or embeddings are usually better fits there.

Practical Use Cases

• Deciding upfront whether a column is nominal or ordinal — gets the encoding choice right the first time
• Checking cardinality before committing to one-hot encoding on a wide categorical column

Common Mistakes

• Label-encoding a nominal feature (like city) with arbitrary integers, which implies a false order the model may pick up on — see Label Encoding.
• One-hot encoding a high-cardinality column without checking nunique() first, silently creating thousands of sparse columns.

Interview Relevance

Q: "Why shouldn't you always use one-hot encoding for categorical features?" It works well for low-cardinality nominal features, but creates an impractically wide, sparse matrix for high-cardinality features — and it discards ordering information that ordinal encoding would preserve for genuinely ordered categories.

Practice Question

Classify each as nominal or ordinal: (a) blood type, (b) shirt size (S/M/L/XL), (c) country of residence, (d) customer satisfaction rating (1–5).

Related ML Notes

• Label Encoding
• One-Hot Encoding
• Ordinal Encoding