Despite sharing "regression" in the name and a nearly identical training loop, logistic and linear regression solve fundamentally different problems — this note is the direct, side-by-side comparison worth knowing cold for interviews.
Side-by-Side Comparison
| Linear Regression | Logistic Regression | |
|---|---|---|
| Predicts | A continuous number | A probability (then thresholded into a class) |
| Output range | Any real number, unbounded | (0, 1), bounded by the sigmoid |
| Equation | \(\hat{y} = w^Tx+b\) | \(\hat{y} = \sigma(w^Tx+b)\) |
| Cost function | MSE | Log-loss |
| Cost surface shape | Convex (guaranteed) | Convex (guaranteed, specifically because log-loss is paired with sigmoid) |
| Evaluation metrics | RMSE, R², MAE | Accuracy, Precision/Recall, ROC-AUC |
| Decision boundary | Not applicable (no classes) | Linear in feature space |
| Example use case | Predicting house price | Predicting whether a house sells within 30 days |
What They Actually Share
- Both compute \(z = w^Tx+b\) as the very first step — a linear combination of features
- Both are interpretable — coefficients have a direct, explainable meaning (though logistic regression's are in log-odds terms)
- Both are trained by minimizing a convex cost function via gradient descent or a closed-form-style solver
- Both are common first-choice baselines before reaching for more complex algorithms
The One-Sentence Distinction
Linear regression models the target directly as a linear function of the features; logistic regression models the target's log-odds as a linear function of the features, then converts those log-odds back into a bounded probability with the sigmoid — see Logistic Regression Intuition for the full derivation of why.
Choosing Between Them — A Simple Test
# Look at the target variable itself
if target_is_continuous_number: # price, temperature, sales volume
use_linear_regression()
elif target_is_binary_category: # yes/no, fraud/not-fraud, churn/no-churn
use_logistic_regression()The choice is almost always determined entirely by what kind of variable you're trying to predict — not by dataset size, feature types, or any other consideration.
What Happens If You Use the Wrong One
| Mistake | Consequence |
|---|---|
| Linear regression on a binary target | Predictions aren't bounded to [0,1] and aren't valid probabilities; the model also isn't optimized for a classification objective |
| Logistic regression on a continuous target | Doesn't apply directly — logistic regression's sigmoid output structurally cannot represent an unbounded continuous value |
Common Mistakes
- Assuming "regression" in both names means they're interchangeable or solve similar problems.
- Using linear regression on a binary outcome and rounding predictions to 0 or 1 — technically possible but abandons calibrated probabilities and a proper classification-oriented cost function.
Interview Relevance
Q: "If both models compute z = wᵀx + b as a first step, what's fundamentally different about them?" What happens next: linear regression uses \(z\) directly as its prediction, while logistic regression passes \(z\) through the sigmoid to produce a bounded probability, and is trained with a different cost function (log-loss, not MSE) chosen specifically to keep that combination convex.
Practice Question
For each target, state whether you'd use linear or logistic regression: (a) predicted delivery time in hours, (b) whether a package will arrive late (yes/no), (c) customer satisfaction score on a continuous 0-100 scale.
