Cross-validation evaluates a model across several different train/test splits instead of just one — averaging the results gives a far more reliable performance estimate than any single split can, and a sense of how much that performance actually varies.
The Core Idea
Instead of asking "how did the model do on this one 20% test set?", cross-validation asks the same question several times, using a different 20% each time, and averages the results. A model that performs consistently well across every split is genuinely reliable; a model with wildly different scores per split is telling you something important about its stability that a single split would hide entirely.
Formula
Both the mean and the standard deviation across folds matter — the mean tells you expected performance, the standard deviation tells you how much to trust that number.
Python Implementation
from sklearn.model_selection import cross_val_score
from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier(n_estimators=100, random_state=42)
scores = cross_val_score(model, X_train, y_train, cv=5, scoring="accuracy")
print(scores) # one score per fold
print(scores.mean(), scores.std()) # the two numbers that actually matterTwo models with the same mean CV score but very different standard deviations are not equally trustworthy — the one with lower variance across folds is more likely to perform consistently on genuinely new data.
Practical Use Cases
- Comparing candidate models or hyperparameter settings more reliably than a single validation split
- Getting a realistic performance estimate on smaller datasets, where a single test set would be too small to trust alone
Common Mistakes
- Fitting preprocessing steps (scalers, encoders) once on the full dataset instead of refitting them within each fold — see Data Leakage and Preprocessing Pipeline.
- Reporting only the mean CV score without also checking its standard deviation.
Interview Relevance
Q: "Why is a single train/test split sometimes insufficient for evaluating a model?" Its result depends partly on which specific rows happened to land in the test set — cross-validation averages across multiple different splits, giving a more stable estimate and revealing how much performance genuinely varies, which a single split can't show at all.
Practice Question
Model A: CV scores across 5 folds average 0.85 with std 0.01. Model B: average 0.87 with std 0.08. Which would you trust more for production, and why might you not simply pick the higher mean?
