A probability distribution describes how likely every possible value of a random variable is. A handful of standard distributions show up constantly across ML — recognizing which one fits your data shapes how you model and evaluate it.
The Distributions You'll Actually Meet
| Distribution | Models | ML Example |
|---|---|---|
| Bernoulli | A single yes/no trial | Whether one email is spam |
| Binomial | Number of successes in \(n\) independent yes/no trials | Number of spam emails out of 100 received |
| Normal (Gaussian) | Continuous data clustering around a mean, symmetric | Heights, measurement errors, many natural features |
The Normal Distribution — Formula and Diagram
\(\mu\) is the mean (the peak/center), \(\sigma\) is the standard deviation (how spread out the curve is). Larger \(\sigma\) means a wider, flatter curve; smaller \(\sigma\) means a taller, narrower one.
The classic bell curve — symmetric around the mean μ, with width controlled by σ.
Numerical Example — Bernoulli and Binomial
A single Bernoulli trial with \(p=0.3\) (probability of "success"): \(P(X=1) = 0.3\), \(P(X=0) = 0.7\). For 10 independent trials (Binomial, \(n=10, p=0.3\)), the expected number of successes is \(np = 10 \times 0.3 = 3\).
import numpy as np
# Simulate 10,000 draws of "10 coin flips with p=0.3 success" and check the average
samples = np.random.binomial(n=10, p=0.3, size=10000)
print(samples.mean()) # approximately 3.0, matching n*pWhy This Matters for ML
- Logistic regression's output is modeled as a Bernoulli probability — "probability the label is 1"
- Many statistical tests and confidence intervals assume roughly normally distributed data or errors
- Naive Bayes with continuous features (Gaussian Naive Bayes) explicitly assumes each feature follows a normal distribution per class
Common Mistakes
- Assuming every real-world feature is normally distributed without checking — income, for example, is typically right-skewed, not normal (see Skewness).
- Confusing Bernoulli (one trial) with Binomial (the count across many trials) — Binomial is built from repeated Bernoulli trials, not a different concept entirely.
Interview Relevance
Q: "Why does Gaussian Naive Bayes assume features are normally distributed?" Because it needs a way to compute \(P(\text{feature value} \mid \text{class})\) for continuous features, and the normal distribution's formula gives a simple, well-understood way to estimate that probability from just the class-conditional mean and standard deviation.
Practice Question
For a Binomial distribution with \(n=20\) trials and \(p=0.25\), what's the expected number of successes?
