The linear regression questions that come up most often in ML interviews — conceptual, mathematical and practical — with complete answers, not just one-liners.
Q1: What are the key assumptions of linear regression?
Short answer: Linearity, independence of residuals, homoscedasticity, normality of residuals, and no severe multicollinearity.
Detailed: see Linear Regression Assumptions for how to check each one with a residual plot or correlation matrix, and what to do when one is violated.
Interview tip: don't just list the five names — be ready to explain how you'd check each one in practice; that's what separates a memorized answer from a demonstrated understanding.
Q2: What's the difference between R² and Adjusted R²?
Short answer: R² measures the proportion of variance explained; Adjusted R² penalizes R² for adding features that don't genuinely improve the fit.
Detailed: plain R² can only increase (or stay the same) as you add more features, even completely useless ones — which makes it a misleading way to compare models with different numbers of features. Adjusted R² adds a penalty term based on feature count, so it can actually decrease if a new feature doesn't pull its weight. See R² Score.
Common mistake: using plain R² to justify adding more and more features — this always looks good on paper even when the extra features are pure noise.
Q3: Why is MSE used as the cost function instead of MAE?
Short answer: MSE is smooth and differentiable everywhere, making it directly usable by gradient-based optimizers; it also penalizes large errors more heavily.
Detailed: see Cost Function for the full comparison, including MAE's non-differentiable point at zero.
Interview tip: mention that MAE is more robust to outliers as the tradeoff — a strong answer shows you understand this is a deliberate choice with a real cost, not an obvious default.
Q4: How do you detect and handle multicollinearity?
Short answer: Check a correlation matrix between features (or compute VIF, Variance Inflation Factor); fix by dropping/combining correlated features or applying L2 regularization.
Code:
import pandas as pd
print(X.corr()) # look for pairs with |correlation| > 0.8-0.9Common mistake: only checking pairwise correlation — three or more features can be collectively collinear even when no single pair looks highly correlated.
Q5: What happens if you train linear regression on unscaled features?
Short answer: The Normal Equation still finds the mathematically correct solution regardless of scale; gradient descent, however, converges much more slowly on unscaled features due to a distorted, elongated cost surface.
Detailed: see Gradient Descent and Feature Scaling for why. Also, coefficient magnitudes become incomparable across features without scaling, complicating interpretation.
Interview tip: a nuanced answer distinguishes between "does it break the math" (no, for Normal Equation) and "does it hurt in practice" (yes, for optimization speed and interpretability) — most candidates only mention one side.
Q6: Can linear regression be used for classification?
Short answer: Technically you could threshold its continuous output, but it's the wrong tool — use logistic regression instead.
Detailed: linear regression's predictions aren't bounded to [0,1] and don't represent valid probabilities; its squared-error cost function also isn't well-suited to classification's actual objective. Logistic regression's sigmoid function and log-loss cost function exist specifically to fix both problems.
Common mistake: answering "yes, just round the prediction" without acknowledging why this is a poor practice, not just an unusual one.
Q7: What does a coefficient of zero mean for a feature?
Short answer: Holding other features constant, that feature has no linear relationship with the target, according to this model.
Detailed: this doesn't necessarily mean the feature is truly useless — it could have a non-linear relationship the model can't capture, or its effect could be absorbed by a correlated feature already in the model. See Correlation Analysis on why low linear association isn't proof of no relationship.
Q8: How would you explain the difference between correlation and a regression coefficient?
Short answer: Correlation measures a symmetric, bivariate linear association between two variables; a regression coefficient measures a specific feature's effect on the target, adjusted for every other feature in the model.
Detailed: two features can each be individually correlated with the target, yet one's regression coefficient could be near zero once the other is included, if they overlap heavily in what they explain — see Multiple Linear Regression.
Practice Question
Prepare a 2-minute spoken answer to Q1 above (assumptions) as if explaining it in a live interview — focus on how you'd check each assumption, not just naming them.
