Feature extraction derives a new, often more compact set of features from raw or complex data — condensing information rather than just reshaping a single existing column. It's the difference between transforming a feature you already have and creating an entirely new representation of the underlying data.
Feature Extraction vs Feature Transformation — The Distinction
| Feature Extraction | Feature Transformation | |
|---|---|---|
| Input | Often raw/complex data (text, images, many correlated columns) | A single existing numeric feature |
| Output | New, often fewer, more information-dense features | The same feature, reshaped (e.g. log-scaled) |
| Example | PCA components, TF-IDF vectors, statistical aggregates from raw sensor data | Log transform, square root, Box-Cox |
Extraction Technique 1 — Dimensionality Reduction
from sklearn.decomposition import PCA
# 10 correlated numeric features -> 3 extracted components capturing most of the variance
pca = PCA(n_components=3)
X_extracted = pca.fit_transform(X_scaled)
print(pca.explained_variance_ratio_)See PCA for the full method — the 3 resulting components are entirely new features, mathematical combinations of the originals, not any single original column reshaped.
Extraction Technique 2 — Statistical Aggregation
import pandas as pd
# Raw: one row per transaction. Extracted: one row per customer, summarizing many transactions.
transactions = pd.DataFrame({
"customer_id": [1, 1, 1, 2, 2],
"amount": [500, 1200, 300, 5000, 4800],
})
customer_features = transactions.groupby("customer_id")["amount"].agg(
total_spend="sum", avg_transaction="mean", transaction_count="count", max_transaction="max"
)
print(customer_features)This is one of the most common real-world feature extraction patterns: raw event-level data (one row per transaction, per click, per visit) gets aggregated into entity-level features (one row per customer) that a model can actually use for a customer-level prediction task.
Extraction Technique 3 — From Unstructured Data
Turning raw text into TF-IDF vectors, or raw images into pixel statistics or embedding vectors from a pretrained model, are both feature extraction — complex, unstructured input becomes a fixed-size numeric vector a classical ML algorithm can consume.
Practical Use Cases
- Reducing highly correlated numeric features into a smaller set of PCA components
- Aggregating transaction/event-level data up to the entity level a model actually predicts on
- Converting text, images or audio into fixed-size numeric feature vectors
Common Mistakes
- Extracting aggregate features (like "total_spend") using data from after the prediction point in time — a classic, easy-to-miss form of data leakage in aggregation-heavy feature engineering.
- Applying PCA-based extraction before scaling the original features — PCA is scale-sensitive, so unscaled inputs distort which directions appear to have the most variance.
Interview Relevance
Q: "You have raw transaction-level data but need to predict customer churn. What's your first feature engineering step?" Aggregate the transaction-level data up to one row per customer — extracting features like total spend, transaction frequency, and recency — since the model needs to make one prediction per customer, not per transaction.
Practice Question
You have raw website clickstream data (one row per page view) and need to predict whether a user will make a purchase in their session. Propose three extracted, session-level features.
