Log loss (binary cross-entropy) evaluates how well-calibrated a classifier's predicted probabilities are — not just whether the final label was right, but how confidently and correctly it got there.
Formula
This is the exact same formula covered in depth in Logistic Regression Cost Function — there, it's the objective being minimized during training; here, it's used as an evaluation metric on held-out data, applicable to any model that outputs probabilities, not just logistic regression.
Why Log Loss Sees More Than Accuracy
Two models can both correctly classify a sample (both above the 0.5 threshold) yet have very different log loss: predicting 0.51 vs 0.99 for a true positive are both "correct" by accuracy's standard, but log loss rewards the more confident, better-calibrated 0.99 prediction, and would punish an overconfident-but-wrong 0.99 prediction on an actual negative far more severely than a cautious 0.51.
from sklearn.metrics import log_loss
y_true = [1, 1, 0, 0]
y_pred_confident_correct = [0.95, 0.9, 0.05, 0.1]
y_pred_barely_correct = [0.55, 0.51, 0.49, 0.45]
print(log_loss(y_true, y_pred_confident_correct)) # much lower (better) log loss
print(log_loss(y_true, y_pred_barely_correct)) # higher log loss, despite both getting every label "right"Practical Use Cases
- Evaluating probability calibration quality, not just final classification accuracy — important for risk-scoring and ranking applications
- The standard loss function used in many ML competitions where probability outputs are directly scored
Common Mistakes
- Confusing log loss (lower is better) with accuracy (higher is better) when comparing model reports.
- Feeding hard 0/1 predictions instead of probabilities into
log_loss()— this defeats its entire purpose, and extreme values (exactly 0 or 1) can produce undefined or extremely large penalties.
Interview Relevance
Q: "Two models both achieve 90% accuracy. How could log loss still distinguish which is 'better'?" Log loss rewards well-calibrated confidence — a model that's consistently confident and correct will have lower (better) log loss than one that's barely crossing the classification threshold each time, even with identical accuracy, which matters a great deal for any downstream use of the raw probability, not just the final label.
Practice Question
Using the log-loss formula, explain why a model that outputs exactly 0.0 or 1.0 for a wrong prediction receives an extremely large (or undefined) penalty.
