Filter Methods

Filter methods score and rank features using statistical properties of the data alone — no model is trained at all — making them the fastest way to cut an obviously weak or redundant feature before doing any heavier work.

Technique 1 — Variance Threshold

A feature with almost no variance is nearly constant across every row — and a feature that barely changes can't help a model distinguish between outcomes.

from sklearn.feature_selection import VarianceThreshold
import pandas as pd

df = pd.DataFrame({
    "feature_a": [1, 1, 1, 1, 1, 1, 2],   # almost constant -- low variance
    "feature_b": [10, 45, 22, 88, 15, 60, 33],  # genuinely varies
})

selector = VarianceThreshold(threshold=0.1)
X_filtered = selector.fit_transform(df)
print(selector.get_support())            # [False  True] -- feature_a removed
print(df.columns[selector.get_support()])

Caveat: variance threshold is scale-sensitive — a feature ranging 0-1 will naturally have far lower variance than one ranging 0-100,000, regardless of how informative each is. Always scale features first, or interpret the threshold relative to each feature's own scale.

Technique 2 — Correlation With the Target

Using the Pearson correlation formula already covered in EDA, filter methods can rank features by how strongly each correlates with the target, keeping only the top-k or those above a chosen threshold.

import pandas as pd

df = pd.DataFrame({
    "size_sqft": [1000, 1500, 1200, 1800, 2200],
    "num_windows": [4, 4, 5, 3, 6],           # weak, roughly unrelated to price
    "price_lakh": [45, 65, 55, 78, 95],
})

correlations = df.corr(numeric_only=True)["price_lakh"].abs().sort_values(ascending=False)
print(correlations)
# size_sqft correlates strongly with price_lakh; num_windows barely does

Technique 3 — Removing Highly Correlated Feature Pairs

import numpy as np

corr_matrix = df.corr(numeric_only=True).abs()
upper_triangle = corr_matrix.where(np.triu(np.ones(corr_matrix.shape), k=1).astype(bool))

to_drop = [col for col in upper_triangle.columns if any(upper_triangle[col] > 0.9)]
print(to_drop)   # features that are near-duplicates of another feature already kept

This directly addresses multicollinearity — when two features carry almost the same information (correlation > 0.9 with each other, not the target), keeping both adds redundancy without adding real signal.

Practical Use Cases

• A fast first pass on a wide dataset before running any expensive wrapper or embedded method
• Removing constant or near-constant features left over from one-hot encoding rare categories
• Catching obviously redundant, highly correlated feature pairs early

Advantages

• Very fast — no model training required
• Model-agnostic — the same filtered feature set can be reused across different algorithms

Limitations

• Ignores feature interactions entirely — a feature with low individual correlation to the target can still be valuable in combination with another feature
• Correlation-based filtering only catches linear relationships, same caveat as correlation analysis generally

Common Mistakes

• Applying a variance threshold on unscaled features without accounting for each feature's natural scale.
• Dropping a feature purely for low target correlation without considering it might matter through an interaction with another feature.

Interview Relevance

Q: "Why might a filter method wrongly discard a genuinely useful feature?" Because filter methods score each feature in isolation — a feature that's only predictive in combination with another (an interaction effect) can show weak individual correlation with the target and get filtered out, even though it would have helped the model.

Practice Question

A one-hot encoded feature "is_currency_XYZ" is 1 for only 2 out of 10,000 rows. Would a variance threshold likely flag this feature for removal? Is that necessarily the right call?

Related ML Notes

• Feature Selection
• Correlation Analysis
• Wrapper Methods