Feature engineering is the process of creating, transforming and selecting the input variables a model actually learns from. It's often the single highest-leverage activity in an ML project — a well-engineered feature can turn a mediocre algorithm into a strong model, while no algorithm can compensate for features that don't carry the right signal.
Why Feature Engineering Usually Matters More Than Algorithm Choice
Swapping logistic regression for a random forest might improve accuracy by a couple of percentage points. Engineering a genuinely new, informative feature — like turning a raw timestamp into "is this a weekend purchase" for a churn model — can improve accuracy far more, because you're giving the model information it literally couldn't extract from the raw column on its own. This is why experienced practitioners spend disproportionately more time here than tuning hyperparameters.
The Three Core Activities
| Activity | What It Means | Example |
|---|---|---|
| Creating features | Deriving new columns from existing raw data | date-time features, interaction features |
| Transforming features | Reshaping an existing feature's scale or distribution | log transforms, polynomial features |
| Selecting features | Deciding which features are actually worth keeping | filter, wrapper, embedded methods |
Feature Engineering by Data Type
- Numerical Features — scaling, binning, transforms
- Categorical Features — encoding strategy by cardinality
- Date-Time Features — extracting cyclical and calendar signal
- Text Features — turning raw text into numeric vectors
A Small Worked Example
import pandas as pd
df = pd.DataFrame({
"signup_date": pd.to_datetime(["2024-01-15", "2024-06-20", "2024-11-02"]),
"last_login": pd.to_datetime(["2024-06-01", "2024-06-25", "2024-11-30"]),
"monthly_spend": [1200, 450, 3000],
})
# Raw dates alone tell a model almost nothing useful. Engineered features do:
df["days_since_signup"] = (df["last_login"] - df["signup_date"]).dt.days
df["signup_month"] = df["signup_date"].dt.month
df["spend_per_day_active"] = df["monthly_spend"] / df["days_since_signup"].clip(lower=1)
print(df[["days_since_signup", "signup_month", "spend_per_day_active"]])None of these three engineered columns existed in the raw data — each one required domain reasoning about what actually predicts the outcome you care about, which is the real skill behind feature engineering, not any specific library call.
Common Mistakes
- Engineering features using information that wouldn't be available at real prediction time — see Data Leakage.
- Creating dozens of features "just in case" without checking whether each one plausibly relates to the target — this inflates dimensionality and overfitting risk for little benefit.
- Engineering features on the full dataset before splitting, instead of fitting any learned transformation (like an encoder or scaler) on the training set only.
Interview Relevance
Q: "Why does feature engineering often matter more than choice of algorithm?" Because a model can only find patterns present in the features it's given — a well-designed feature can expose a relationship directly that the raw data only implied indirectly, while the best algorithm in the world can't invent information that isn't there.
Practice Question
You have raw columns "order_date" and "delivery_date" for an e-commerce dataset predicting customer satisfaction. Propose two engineered features that would likely be more useful than the raw dates themselves.
