Logistic regression is the standard baseline algorithm for binary classification — despite the name, it predicts a probability of belonging to a class, not a continuous number, by passing a linear combination of features through the sigmoid function.
The Equation, In Two Steps
Step one is identical to linear regression — a weighted sum of features plus a bias. Step two passes that sum through the sigmoid function, squashing it into a probability between 0 and 1. This two-step structure is exactly what fixes linear regression's fundamental unsuitability for classification — see Logistic Regression vs Linear Regression.
From Probability to a Class Prediction
0.5 is the default threshold, but it isn't sacred — it can be moved based on the relative cost of false positives vs false negatives, a decision covered in Precision-Recall Curve.
Decision Boundary, Visually
The line where z = wTx + b = 0 (probability exactly 0.5) separates the two predicted classes — everything is linear in feature space, even though the probability curve itself is an S-shape.
Minimal Working Example
from sklearn.linear_model import LogisticRegression
import numpy as np
hours = np.array([[1], [2], [3], [4], [5], [6], [7], [8]])
passed = np.array([0, 0, 0, 0, 1, 1, 1, 1])
model = LogisticRegression()
model.fit(hours, passed)
print(model.predict_proba([[5.5]])) # [[probability of 0, probability of 1]]
print(model.predict([[5.5]])) # the thresholded classPractical Use Cases
- Spam detection, churn prediction, loan approval, disease diagnosis — anywhere the outcome is binary
- As a fast, interpretable baseline before trying more complex classifiers
- Any case where you need a calibrated probability, not just a hard label — critical for risk-based decisions
Advantages
- Outputs genuine probabilities, not just labels — useful for ranking, thresholding, and risk assessment
- Interpretable coefficients, similar to linear regression
- Fast to train and hard to overfit with few features, relative to more complex classifiers
Limitations
- Assumes a linear decision boundary in feature space — struggles with genuinely non-linear class separation unless features are engineered
- Sensitive to unscaled features when trained with gradient-based solvers
- Can perform poorly on strongly imbalanced data without adjustment (class weights, threshold tuning)
Common Mistakes
- Interpreting logistic regression's output as a raw score rather than a genuine probability, and forgetting the 0.5 threshold is adjustable.
- Using logistic regression on data with an obviously non-linear class boundary without adding interaction or polynomial features first.
Interview Relevance
Q: "Is logistic regression a regression or classification algorithm?" Classification — despite the name, it's used to predict discrete class probabilities; the "regression" in the name refers to it modeling the log-odds of the outcome as a linear function of the features, a mathematical detail, not its practical use case.
Practice Question
A trained model gives \(\hat{y} = 0.82\) for a new email. What class does it predict at the default threshold, and what does 0.82 mean in plain language?
