SMOTE (Synthetic Minority Oversampling Technique) fixes plain oversampling's biggest weakness — instead of duplicating existing minority points exactly, it generates new, synthetic minority examples by interpolating between real ones.
The Algorithm
| Step | What Happens |
|---|---|
| 1 | Pick a random minority-class point \(x_i\) |
| 2 | Find its \(k\) nearest minority-class neighbors (typically \(k=5\)) |
| 3 | Randomly pick one neighbor, \(x_{zi}\) |
| 4 | Generate a new synthetic point along the line segment between them |
| 5 | Repeat until the desired class balance is reached |
Formula
\(\lambda\) is a random number between 0 and 1, so the new synthetic point lands somewhere along the straight line connecting \(x_i\) and its neighbor — never exactly on top of either one.
Worked Example
Minority point \(x_i=(2,3)\), its nearest minority neighbor \(x_{zi}=(4,5)\), random \(\lambda=0.5\):
The synthetic point \((3,4)\) sits exactly halfway between the two real minority points — a genuinely new example, not a duplicate of either.
Graphical Intuition
The new point is a plausible interpolation between two real minority examples — not an exact copy of either one.
Python Implementation
from imblearn.over_sampling import SMOTE
from collections import Counter
smote = SMOTE(k_neighbors=5, random_state=42)
X_resampled, y_resampled = smote.fit_resample(X_train, y_train)
print(Counter(y_train)) # {0: 950, 1: 50}
print(Counter(y_resampled)) # {0: 950, 1: 950} -- balanced with SYNTHETIC, not duplicated, minority pointsSMOTE vs Plain Oversampling
| Plain Oversampling | SMOTE | |
|---|---|---|
| New minority points | Exact duplicates | Synthetic, interpolated points |
| Overfitting to specific points? | Higher risk | Lower risk |
| Requires numeric features | No — works on anything | Yes — interpolation needs continuous numeric features |
Practical Use Cases
- The most commonly used resampling technique for imbalanced tabular classification with numeric features
- Fraud, medical, and defect-detection datasets where minority examples are genuinely scarce
Limitations
- Only works cleanly on numeric features — categorical features need a variant like SMOTE-NC
- Can generate synthetic points in ambiguous or noisy regions if minority-class examples are themselves scattered and not well-clustered
- Must still only ever be applied to the training set, same as any resampling technique
Common Mistakes
- Applying vanilla SMOTE to data with categorical features without switching to
SMOTENCor a similar variant. - Using a \(k\) for nearest neighbors larger than the number of available minority samples — SMOTE needs at least \(k+1\) minority examples to work.
Interview Relevance
Q: "Why is SMOTE generally preferred over simple random oversampling?" Random oversampling duplicates existing minority points exactly, which can cause a model to overfit to those specific repeated examples; SMOTE instead generates new, synthetic points by interpolating between real minority examples, giving the model more varied, plausible minority-class data to learn from.
Practice Question
Given minority points \(x_i=(1,1)\) and its neighbor \(x_{zi}=(5,9)\), compute the synthetic point SMOTE would generate with \(\lambda=0.25\).
