Linear regression predicts a continuous number by fitting a straight line (or flat plane, with more features) through the data — the simplest, most interpretable supervised learning algorithm, and still the right first choice for a huge share of real regression problems.
The Equation
\(\hat{y}\) is the predicted value, \(x\) is the input feature, \(b_1\) is the slope (how much \(\hat{y}\) changes per unit increase in \(x\)), and \(b_0\) is the intercept (the predicted value when \(x=0\)). "Training" a linear regression model means finding the \(b_0\) and \(b_1\) values that fit the data best.
Graphical Intuition — Fitting the Best Line
The "best" line minimizes the total squared length of every residual (the vertical gap between each point and the line) — see Cost Function.
Simple vs Multiple Linear Regression
| Simple Linear Regression | Multiple Linear Regression | |
|---|---|---|
| Features | One | Two or more |
| Equation | \(\hat{y}=b_0+b_1x\) | \(\hat{y}=b_0+b_1x_1+\dots+b_nx_n\) |
| Geometric shape | A line | A flat plane (or hyperplane, in higher dimensions) |
How the Model Is Actually Trained
Two equivalent routes reach the same optimal coefficients: the Normal Equation (an exact, closed-form matrix formula) and gradient descent (an iterative search). See Linear Regression in Python for both implemented directly.
Minimal Working Example
from sklearn.linear_model import LinearRegression
import numpy as np
hours = np.array([[1], [2], [3], [4], [5]])
marks = np.array([52, 58, 62, 68, 75])
model = LinearRegression()
model.fit(hours, marks)
print(model.intercept_, model.coef_) # 46.2 [5.6]
print(model.predict([[6]])) # [79.8]These numbers (\(b_0=46.2\), \(b_1=5.6\)) are hand-verifiable — see the step-by-step calculation in Simple Linear Regression.
Practical Use Cases
- House price prediction from size/location/features
- Sales forecasting from marketing spend and seasonality
- Any problem needing an interpretable "how much does each input matter" answer, not just a prediction
Advantages
- Highly interpretable — each coefficient has a direct, explainable meaning
- Fast to train, even on large datasets, and requires no hyperparameter tuning to get a reasonable baseline
- A strong, well-understood baseline to compare more complex models against
Limitations
- Can only capture linear relationships unless features are explicitly engineered (see Polynomial Features)
- Sensitive to outliers, since it minimizes squared error
- Requires its assumptions to hold reasonably well for its coefficients and confidence intervals to be trustworthy
Common Mistakes
- Fitting a linear model to data with an obviously non-linear relationship without checking a scatter plot first.
- Interpreting a coefficient's size as "importance" without first standardizing features — coefficients on unscaled features aren't directly comparable.
Interview Relevance
Q: "Why is linear regression still used given more powerful algorithms exist?" Interpretability, speed, and being a strong, well-understood baseline — when the relationship is genuinely close to linear, it's often competitive with much more complex models while remaining far easier to explain and debug.
Practice Question
A trained model has \(b_0=10\), \(b_1=3\). What does it predict for \(x=7\), and in plain language, what does \(b_1=3\) mean?
