Interpretability and explainability are often used interchangeably — but they mean genuinely different things, and the distinction matters for choosing the right technique for a given model and problem.
The Key Distinction
| Interpretability | Explainability | |
|---|---|---|
| Definition | The model's internal logic is directly, inherently understandable | External techniques extract an understandable explanation from an otherwise opaque model |
| Where understanding comes from | The model's own structure (coefficients, tree splits) | A separate, applied technique (SHAP, LIME, permutation importance) |
| Example | Linear regression — read the coefficients directly | Random Forest + SHAP — the forest itself is opaque, SHAP extracts meaning from it |
| Trust level | Exact — you're reading the actual mechanism | Approximate — an explanation of behavior, not the literal mechanism |
Inherently Interpretable Models
- Linear/Logistic Regression: each coefficient has a direct, precise meaning
- Decision Trees (shallow ones especially): the exact decision path is fully readable
- Naive Bayes: class probabilities are directly computed from readable per-feature likelihoods
Models That Need Post-Hoc Explainability
- Random Forest, Gradient Boosting: hundreds of trees combined — no single readable path exists
- Neural Networks: millions of weighted connections, no direct human-readable structure
- SVM with a non-linear kernel: the decision boundary exists in an implicit, high-dimensional transformed space
Why the Distinction Actually Matters in Practice
An interpretable model's explanation is exact — reading a linear regression coefficient tells you precisely what the model does. A post-hoc explanation (like SHAP on a Random Forest) is an approximation of the opaque model's behavior — extremely useful, but not a literal window into its internal mechanism the way reading a linear model's coefficients is. In high-stakes contexts, this distinction can matter legally and ethically, not just technically.
The Accuracy-Interpretability Tradeoff, Revisited
# A concrete decision point many real projects face
if regulatory_requirement_for_exact_reasoning:
use_inherently_interpretable_model() # linear/logistic regression, shallow tree
elif accuracy_matters_most_and_explanation_can_be_approximate:
use_flexible_model_plus_post_hoc_explainability() # Random Forest/XGBoost + SHAPPractical Use Cases
- Deciding upfront whether a project's requirements demand exact interpretability or can accept approximate post-hoc explainability
- Communicating precisely to stakeholders what kind of "understanding" a given explanation actually provides
Common Mistakes
- Using "interpretable" and "explainable" interchangeably in a context (like a regulatory filing) where the distinction genuinely matters.
- Assuming a post-hoc explanation is exactly equivalent to true model transparency — it's a faithful approximation, not the literal mechanism.
Interview Relevance
Q: "What's the difference between an interpretable model and an explainable one?" An interpretable model's own structure is directly understandable (like linear regression's coefficients); an explainable model is opaque internally, but external techniques (SHAP, LIME) can extract an approximate, understandable explanation of its behavior after the fact — the understanding comes from a separate tool, not the model's own transparent structure.
Practice Question
Classify each as inherently interpretable or requiring post-hoc explainability: (a) a 3-node decision tree, (b) a 500-tree gradient boosting ensemble, (c) logistic regression with 5 features.
