Logistic Regression in Python

A complete logistic regression workflow in Python — dataset through evaluation — plus a look at what's actually different from the linear regression workflow beyond swapping one class name for another.

The Full scikit-learn Workflow

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, confusion_matrix, classification_report

# 1. Dataset
df = pd.DataFrame({
    "hours_studied": [1, 2, 3, 4, 5, 6, 7, 8, 1.5, 2.5, 5.5, 7.5],
    "passed":        [0, 0, 0, 0, 1, 1, 1, 1, 0,   0,   1,   1],
})

# 2. Features and target
X = df[["hours_studied"]]
y = df["passed"]

# 3. Train/test split -- stratify to preserve class balance in both sets
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42, stratify=y
)

# 4. Scale (matters more for gradient-based solvers with multiple features)
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

# 5. Model creation and training
model = LogisticRegression()
model.fit(X_train_scaled, y_train)

# 6. Prediction -- both probability AND hard label
probabilities = model.predict_proba(X_test_scaled)
predictions = model.predict(X_test_scaled)

# 7. Evaluation
print("Accuracy:", accuracy_score(y_test, predictions))
print(confusion_matrix(y_test, predictions))
print(classification_report(y_test, predictions))

Steps 1-3 and 5-6 look nearly identical to the linear regression workflow — the real differences are: stratify=y in the split (matters for classification, not regression), predict_proba() existing at all, and entirely different evaluation metrics in step 7.

Visualizing the Sigmoid Fit

import numpy as np
import matplotlib.pyplot as plt

x_range = np.linspace(0, 10, 200).reshape(-1, 1)
x_range_scaled = scaler.transform(x_range)
probs = model.predict_proba(x_range_scaled)[:, 1]

plt.scatter(X, y, color="black", label="Actual data")
plt.plot(x_range, probs, color="red", label="Predicted probability")
plt.axhline(0.5, color="gray", linestyle="--", label="Decision threshold")
plt.xlabel("Hours studied"); plt.ylabel("P(passed)"); plt.legend()
plt.show()

Adjusting the Decision Threshold

# Instead of the default 0.5 threshold, use a custom one
custom_threshold = 0.3
custom_predictions = (probabilities[:, 1] >= custom_threshold).astype(int)
print(custom_predictions)
# Lowering the threshold predicts class 1 more often -- trades precision for recall,
# useful when missing a positive case (a false negative) is costlier than a false alarm

Handling Multiple Classes

# scikit-learn's LogisticRegression handles more than 2 classes automatically
from sklearn.datasets import load_iris

iris = load_iris()
multi_model = LogisticRegression(max_iter=1000)
multi_model.fit(iris.data, iris.target)   # 3 classes -- no extra code needed
print(multi_model.predict_proba(iris.data[:1]))   # probability for each of the 3 classes

For multi-class problems, scikit-learn uses a multinomial (softmax) generalization of the sigmoid internally by default — the same core idea, extended beyond two classes.

Common Mistakes

• Reporting accuracy alone on an imbalanced dataset without checking precision/recall/F1 — accuracy can look deceptively good while the model fails entirely on the minority class.
• Forgetting stratify=y in the train/test split, especially with a rare positive class.
• Using predict() when you actually need calibrated probabilities from predict_proba() — for example, when ranking leads by likelihood to convert.

Interview Relevance

Q: "How would you handle a business requirement to minimize missed fraud cases, even at the cost of more false alarms?" Lower the decision threshold below 0.5 using predict_proba() — this trades precision for recall, catching more true positives at the cost of more false positives, tunable to match the specific cost tradeoff the business cares about.

Practice Question

Modify the workflow above to lower the classification threshold to 0.35 and compare the resulting confusion matrix to the default-threshold version.

Related ML Notes

• Confusion Matrix
• Precision-Recall Curve
• Logistic Regression Example