Statistics is the discipline of summarizing data and drawing reliable conclusions from it despite uncertainty. In machine learning it plays two distinct roles: it gives you the vocabulary to describe a dataset before you model it, and the tools to decide whether your model's results are real or just noise.
Why a Note Like This Exists Before You Touch a Single Algorithm
It's tempting to skip straight to model.fit(). But almost every serious modeling mistake — trusting a misleading average, missing a skewed distribution, believing a model improvement that's actually random chance — traces back to a statistics concept that was skipped, not an algorithm that was misunderstood. This section builds that foundation deliberately, the same way Math for ML built the linear algebra and calculus foundation before the algorithms that use it.
Descriptive Statistics vs Inferential Statistics
Statistics splits cleanly into two jobs, and confusing them is one of the most common analytical mistakes in data work:
| Descriptive Statistics | Inferential Statistics | |
|---|---|---|
| Question it answers | "What does this data look like?" | "What can I conclude about the wider population, or about whether a difference is real?" |
| Tools | Mean, Median, Standard Deviation, Percentiles | Hypothesis Testing, p-values, Confidence Intervals |
| Example in ML | Summarizing a feature's distribution before modeling it | Deciding if Model B is genuinely more accurate than Model A, or just got lucky on this test set |
| Scope of the conclusion | Only describes the data you actually have | Generalizes beyond the specific sample you measured |
The Statistics Every ML Practitioner Actually Uses
Four groups of statistical tools — this hub links to a full note on each one.
A Concrete Walkthrough — Why Skipping Statistics Costs You
Imagine you're evaluating a churn-prediction model and its accuracy improves from 84.1% to 84.6% after a feature engineering change. Is that a genuine improvement, or could it be explained by random variation in which customers happened to land in your test set this time? Without hypothesis testing or at least a confidence interval around each accuracy estimate, you're guessing — and teams that skip this step end up shipping "improvements" that are actually noise, then get confused when the "improvement" doesn't hold up in production.
import numpy as np
# Two models' accuracy across 10 different random train/test splits
model_a_scores = np.array([0.841, 0.838, 0.845, 0.839, 0.842, 0.840, 0.837, 0.843, 0.841, 0.839])
model_b_scores = np.array([0.846, 0.843, 0.849, 0.844, 0.847, 0.845, 0.842, 0.848, 0.846, 0.844])
print("Model A mean:", model_a_scores.mean(), "std:", model_a_scores.std())
print("Model B mean:", model_b_scores.mean(), "std:", model_b_scores.std())
# Only by looking at the SPREAD across runs (not just one number) can you judge
# whether Model B's apparent edge is consistent or could be noise.Practical Use Cases Across the ML Lifecycle
- Before modeling: summarizing feature distributions during EDA to catch skew, outliers, and data quality issues
- During feature engineering: using correlation to spot redundant or predictive features
- During evaluation: using confidence intervals and hypothesis tests to decide if a model change is a genuine improvement
- In production monitoring: comparing live feature distributions against training distributions to detect data drift
Common Mistakes
- Reporting a single summary statistic (like accuracy) without ever reporting its spread or uncertainty across multiple runs or folds.
- Treating descriptive statistics (what happened in this specific dataset) as if they were inferential claims (what's generally true) — a 2% higher average in one sample doesn't prove a real, generalizable difference.
- Jumping to modeling without first computing basic descriptive statistics for every feature — this is exactly what Pandas'
.describe()is for.
Interview Relevance
Q: "Why does a data scientist need statistics if scikit-learn handles the modeling?" Because the model only tells you what pattern it found — statistics tells you whether that pattern is meaningful, how confident you should be in it, and whether your evaluation numbers are stable or just a fluke of one particular train/test split.
Q: "What's the difference between descriptive and inferential statistics, with an ML example?" Descriptive: "this training set's average customer age is 34." Inferential: "based on this sample, we're 95% confident the true average age of all customers is between 32 and 36" — inferential statistics reasons about uncertainty and generalization, descriptive statistics just summarizes what's directly in front of you.
Practice Question
You compute the mean accuracy of two models across 5-fold cross-validation: Model A = 0.82, Model B = 0.83. Is this enough information to declare Model B better? What additional statistic would you want to see?
