Linear regression's coefficients and confidence intervals are only trustworthy if certain assumptions about the data roughly hold. Violating them doesn't necessarily break the model outright, but it does undermine exactly how much you should trust its numbers.
The Five Assumptions
| Assumption | Meaning | How to Check |
|---|---|---|
| Linearity | The relationship between features and target is genuinely linear | Scatter plots, residual plots |
| Independence | Residuals aren't correlated with each other | Especially relevant for time-series data (autocorrelation) |
| Homoscedasticity | Residual variance is roughly constant across all predicted values | Residual plot — should look like a random, even scatter |
| Normality of residuals | Residuals are roughly normally distributed | Histogram or Q-Q plot of residuals |
| No multicollinearity | Features aren't highly correlated with each other | Correlation matrix between features |
Reading a Residual Plot — The Single Most Useful Diagnostic
Plot residuals against predicted values: random scatter around zero is healthy; a funnel or curve signals an assumption violation.
Checking Assumptions in Python
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression
import numpy as np
model = LinearRegression().fit(X_train, y_train)
predictions = model.predict(X_train)
residuals = y_train - predictions
# Homoscedasticity / non-linearity check
plt.scatter(predictions, residuals, alpha=0.6)
plt.axhline(0, color="red", linestyle="--")
plt.xlabel("Predicted values"); plt.ylabel("Residuals")
plt.show()
# Normality check
plt.hist(residuals, bins=20)
plt.show()
# Multicollinearity check
import pandas as pd
print(pd.DataFrame(X_train).corr())What to Actually Do When an Assumption Is Violated
| Violation | Common Fix |
|---|---|
| Non-linearity | Add polynomial features, or switch to a non-linear model |
| Heteroscedasticity | Transform the target (e.g. log), or use a model less sensitive to this assumption |
| Non-normal residuals | Often tolerable in practice for prediction; matters more for statistical inference (confidence intervals, p-values) |
| Multicollinearity | Drop or combine correlated features, or use L2 regularization |
Why These Assumptions Matter Less for Pure Prediction, More for Inference
If your only goal is prediction accuracy, moderate assumption violations often matter less than you'd expect — the model can still produce reasonably useful predictions. But if you intend to interpret the coefficients (e.g. "this feature increases price by X, with 95% confidence it's between Y and Z"), assumption violations directly undermine how trustworthy those specific numbers are.
Common Mistakes
- Never checking a residual plot at all — this is the fastest, highest-value diagnostic and is skipped surprisingly often.
- Treating a mild assumption violation as disqualifying, when in practice linear regression is often reasonably robust to small violations for prediction purposes.
- Checking multicollinearity only via a correlation matrix, missing more subtle multi-feature collinearity that a single pairwise check wouldn't reveal.
Interview Relevance
Q: "What does a funnel-shaped residual plot indicate, and why does it matter?" Heteroscedasticity — the model's errors get larger (or smaller) as predicted values change, violating the constant-variance assumption; this undermines the validity of the model's confidence intervals and statistical significance tests, even if raw prediction accuracy is still reasonable.
Practice Question
You plot residuals vs predicted values and see a clear upward curve (not random scatter). Which assumption is violated, and what's a common fix?
