Logistic Regression Example

A complete, realistic case study — predicting loan default risk — walking through EDA, training, evaluation, and translating a probability output into an actual business decision.

The Business Question

A lender wants to estimate the probability that a loan applicant will default, to inform approval decisions and interest rate pricing — not just a yes/no answer, but a calibrated risk score.

The Dataset

import pandas as pd

df = pd.DataFrame({
    "credit_score":    [750, 680, 620, 590, 710, 640, 800, 560, 690, 730, 600, 770],
    "debt_to_income":  [0.15, 0.35, 0.42, 0.55, 0.20, 0.38, 0.10, 0.60, 0.30, 0.18, 0.50, 0.12],
    "defaulted":       [0,   0,    1,    1,    0,    1,    0,   1,    0,    0,    1,    0],
})
print(df.describe())

Quick EDA

print(df.groupby("defaulted")[["credit_score", "debt_to_income"]].mean())
# Defaulters clearly average a lower credit score and higher debt-to-income --
# both features look predictive, worth confirming with the model itself

Training the Model

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression

X = df[["credit_score", "debt_to_income"]]
y = df["defaulted"]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42, stratify=y)

scaler = StandardScaler().fit(X_train)
X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test)

model = LogisticRegression().fit(X_train_scaled, y_train)
print(dict(zip(X.columns, model.coef_[0])))

Interpreting the Result — Business Language, Not Just Numbers

Suppose the fitted (standardized) coefficients come out approximately: \(b_{\text{credit\_score}} \approx -1.4\), \(b_{\text{debt\_to\_income}} \approx 1.6\). Translated:

• Higher credit score is associated with lower default risk (negative coefficient) — each standard deviation increase in credit score roughly divides the odds of default by \(e^{1.4} \approx 4.1\), holding debt-to-income constant.
• Higher debt-to-income ratio is associated with higher default risk (positive coefficient) — each standard deviation increase roughly multiplies the odds of default by \(e^{1.6} \approx 4.95\), holding credit score constant.

These are exactly the kind of statements a credit risk team can act on directly — and they're only meaningful because the coefficients were fit on standardized features, making "one unit" comparable across credit_score (hundreds) and debt_to_income (a fraction between 0 and 1).

Turning Probability Into a Decision

new_applicant = [[650, 0.40]]
new_applicant_scaled = scaler.transform(new_applicant)
risk_probability = model.predict_proba(new_applicant_scaled)[0][1]
print(f"Default risk: {risk_probability:.1%}")

# The business decision layer -- NOT part of the model itself
if risk_probability < 0.20:
    decision = "Approve at standard rate"
elif risk_probability < 0.50:
    decision = "Approve at higher interest rate"
else:
    decision = "Decline"
print(decision)

This is a genuinely important pattern: the model outputs a probability; a separate, explicit business rule layer converts that probability into a decision, with thresholds the business can adjust without retraining the model.

Evaluating Honestly

from sklearn.metrics import classification_report, roc_auc_score

y_pred = model.predict(X_test_scaled)
y_proba = model.predict_proba(X_test_scaled)[:, 1]

print(classification_report(y_test, y_pred))
print("ROC-AUC:", roc_auc_score(y_test, y_proba))

Being realistic: with a dataset this small (12 rows), these metrics are illustrative only — a real credit risk model would need far more historical data, careful handling of class imbalance (defaults are typically rare), and regulatory fairness review before informing real lending decisions.

Common Mistakes

• Baking business decision thresholds directly into the model instead of keeping them as an adjustable layer on top of the raw probability.
• Not checking for class imbalance — real default datasets are usually heavily skewed toward non-default, requiring the techniques in Imbalanced Data.

Interview Relevance

Q: "Why output a probability instead of just a yes/no decision for loan approval?" A probability lets the business apply its own risk tolerance and pricing logic (different thresholds for approval, decline, or risk-adjusted interest rates) without retraining the model — collapsing to a single hard-coded decision throws away exactly that flexibility.

Practice Question

Using the decision-threshold code above, explain what changes (in terms of who gets approved) if the business lowers the "decline" threshold from 0.50 to 0.35.

Related ML Notes

• ROC-AUC
• Loan Approval Prediction Project
• Imbalanced Data