A concise, practical checklist for feature engineering done well — the habits that separate features that genuinely help a model from features that just add noise, risk leakage, or quietly break in production.
The Checklist
| Practice | Why It Matters |
|---|---|
| Split before engineering anything learned from data | Prevents data leakage — fit encoders, scalers, target-encoders on training data only |
| Ask "would this be available at real prediction time?" | Catches target leakage — a feature that only exists after the outcome is known is unusable |
| Start with domain reasoning, not automation | A feature built on a real hypothesis ("recent job loss risk") tends to generalize better than blind automatic feature generation |
| Check each feature's distribution before using it | Skew, outliers and cardinality all determine the right transform/encoding |
| Prefer a pipeline over manual, ad hoc steps | Guarantees the exact same transformations apply consistently at prediction time — see Preprocessing Pipeline |
| Validate that a new feature actually helps | Compare validation performance with and without it — "sounds like it should help" isn't evidence |
| Watch for multicollinearity among engineered features | Ratios, interactions and polynomial terms often correlate heavily with their source features |
| Document what each feature means and how it's computed | Six months later, "days_since_last_X" is meaningless without knowing exactly what X and "since when" refer to |
A Full Before/After Example
# BEFORE -- ad hoc, leakage-prone, undocumented
df["scaled_income"] = (df["income"] - df["income"].mean()) / df["income"].std() # computed on FULL data
df["ratio"] = df["a"] / df["b"] # what do a and b even mean here?
# AFTER -- disciplined, leak-free, documented
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
scaler = StandardScaler().fit(X_train[["income"]]) # fit on TRAIN only
X_train["scaled_income"] = scaler.transform(X_train[["income"]])
X_test["scaled_income"] = scaler.transform(X_test[["income"]]) # same fitted scaler, applied
# debt_to_income_ratio: total outstanding debt / gross annual income
X_train["debt_to_income_ratio"] = X_train["total_debt"] / X_train["annual_income"].clip(lower=1)The "So What" Test
Before adding any new feature, force yourself to answer: "If this feature is genuinely predictive, why — what real-world mechanism connects it to the target?" If you can't answer that in a sentence, either the feature is a coincidental pattern that won't generalize, or you haven't understood the problem well enough yet — both are worth pausing on before adding more complexity.
Practical Use Cases
This checklist applies identically whether you're building a churn model, a fraud detector, or a house price predictor — the specific features differ, but the discipline (split first, check distributions, validate empirically, document) doesn't change.
Common Mistakes
- Engineering dozens of features quickly without validating any of them individually — a large, undocumented, unvalidated feature set is hard to debug when something goes wrong.
- Copying a feature engineering "trick" from another project without checking whether the underlying domain reasoning actually applies to your problem.
- Treating feature engineering as a one-time step instead of an iterative loop informed by model errors and ongoing EDA.
Interview Relevance
Q: "How do you decide whether a new engineered feature is actually worth keeping?" Compare validation performance (using proper cross-validation, not just training accuracy) with and without the feature — a feature that "sounds reasonable" but doesn't measurably improve validation performance usually isn't worth the added complexity and leakage risk.
Practice Question
A teammate proposes adding "customer_lifetime_value" as a feature to predict "will this customer churn next month." Using the "would this be available at prediction time?" test, explain what you'd want to verify before accepting this feature.
