Multiple linear regression extends the same idea to two or more input features — instead of fitting a line, it fits a flat plane (or hyperplane, for more than 2 features) through the data.
Formula
Each \(b_i\) is the change in \(\hat{y}\) for a one-unit increase in \(x_i\), holding every other feature constant — this "holding constant" clause is the entire reason multiple regression coefficients differ from what you'd get running separate simple regressions on each feature individually.
Simple vs Multiple — Visually
With 2 features, the model fits a flat plane; with more, a hyperplane — impossible to draw, but mathematically identical.
Worked Example
from sklearn.linear_model import LinearRegression
import numpy as np
# hours studied AND practice tests taken -> exam marks
X = np.array([
[1, 0],
[2, 1],
[3, 1],
[4, 2],
[5, 3],
])
y = np.array([52, 58, 62, 68, 75])
model = LinearRegression().fit(X, y)
print(model.intercept_) # some baseline value
print(model.coef_) # [coef_for_hours, coef_for_tests]
print(model.predict([[6, 3]])) # prediction for 6 hours studied, 3 tests takenUnlike simple linear regression, these coefficients generally can't be computed by a short hand formula — solving for multiple coefficients simultaneously requires matrix algebra (the Normal Equation, \(\vec{b} = (X^TX)^{-1}X^T\vec{y}\)) or gradient descent. See Linear Regression in Python for both implemented directly.
Why "Holding Other Features Constant" Matters
Suppose "hours studied" and "practice tests taken" are themselves correlated (more hours often means more tests attempted). A simple regression of marks on hours alone would credit "hours" with some of the effect that's actually coming from tests. Multiple regression's coefficients separate these effects — \(b_1\) reflects hours' effect specifically after accounting for tests' effect, and vice versa.
Multicollinearity — The Risk That Comes With More Features
When two input features are highly correlated with each other (not the target), the model struggles to reliably separate their individual effects — coefficients can become unstable, even flipping sign with small changes in the data. This is exactly why checking correlation between features matters before fitting a multiple regression model. See Linear Regression Assumptions.
Practical Use Cases
- House price prediction from size, bedrooms, location and age together
- Sales forecasting from ad spend across multiple channels simultaneously
- Any problem where several factors jointly influence the outcome
Advantages
- Captures the combined, mutually-adjusted effect of several predictors at once
- Still fully interpretable — each coefficient has a precise "holding others constant" meaning
Limitations
- Vulnerable to multicollinearity when features are correlated with each other
- Coefficients become harder to interpret in plain language as feature count grows
- Still assumes linear, additive relationships unless features are explicitly engineered
Common Mistakes
- Interpreting a multiple regression coefficient the same way as a simple regression coefficient — the "holding others constant" clause changes its meaning.
- Adding many correlated features without checking for multicollinearity first.
Interview Relevance
Q: "Why can't you just run separate simple regressions for each feature instead of one multiple regression?" Because separate simple regressions ignore how features relate to each other — if two features are correlated, a simple regression conflates their effects, while multiple regression's coefficients are adjusted to isolate each feature's individual contribution.
Practice Question
A multiple regression model predicting salary has coefficients: \(b_{\text{experience}}=5000\), \(b_{\text{education\_years}}=3000\). Explain in plain language what \(b_{\text{experience}}=5000\) means, being careful to include the "holding constant" clause.
