A complete decision tree workflow in Python — training, visualizing the actual tree structure, reading feature importance, and tuning depth via cross-validation.
The Full Workflow
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.tree import DecisionTreeClassifier, plot_tree
from sklearn.metrics import classification_report
from sklearn.datasets import load_breast_cancer
import matplotlib.pyplot as plt
# 1. Dataset
data = load_breast_cancer()
X, y = data.data, data.target
# 2. Train/test split
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42, stratify=y
)
# 3. Tune max_depth via cross-validation (no scaling needed for trees)
param_grid = {"max_depth": [2, 3, 4, 5, 6, 8, 10, None]}
grid_search = GridSearchCV(
DecisionTreeClassifier(random_state=42), param_grid, cv=5, scoring="accuracy"
)
grid_search.fit(X_train, y_train)
print("Best max_depth:", grid_search.best_params_)
# 4. Train the best model
model = grid_search.best_estimator_
# 5. Evaluate
predictions = model.predict(X_test)
print(classification_report(y_test, predictions))
# 6. Visualize the actual tree structure
plt.figure(figsize=(16, 8))
plot_tree(model, feature_names=data.feature_names, class_names=data.target_names,
filled=True, max_depth=3, fontsize=8) # max_depth here limits the PLOT, not the model
plt.show()Being able to literally look at the tree — every split, every threshold, every leaf's class distribution — is one of decision trees' most valuable practical properties, unavailable with most other model types.
Reading Feature Importance
import pandas as pd
importances = pd.Series(model.feature_importances_, index=data.feature_names).sort_values(ascending=False)
print(importances.head(10))A tree's feature_importances_ reflects how much each feature contributed to reducing impurity across all its splits, weighted by how many samples passed through each split — see Feature Importance for the general concept.
Regression Trees — The Same Pattern
from sklearn.tree import DecisionTreeRegressor
from sklearn.metrics import mean_absolute_error
reg_model = DecisionTreeRegressor(max_depth=4, min_samples_leaf=5, random_state=42)
reg_model.fit(X_train_reg, y_train_reg)
preds = reg_model.predict(X_test_reg)
print(mean_absolute_error(y_test_reg, preds))Extracting Human-Readable Rules
from sklearn.tree import export_text
rules = export_text(model, feature_names=list(data.feature_names), max_depth=3)
print(rules)
# Prints the tree as nested if/else text -- directly readable without any plotting libraryThis text export is genuinely useful for sharing a model's logic with non-technical stakeholders, or for a quick sanity check without needing matplotlib.
Common Mistakes
- Applying feature scaling before training a tree — harmless but unnecessary, since trees split on thresholds, not distances or gradients.
- Not setting
random_state— trees can behave slightly differently across runs when there are ties in split quality, making results hard to reproduce. - Interpreting
feature_importances_as proof of causation, the same caveat covered in Feature Importance.
Interview Relevance
Q: "How would you explain a specific prediction from a decision tree to a non-technical stakeholder?" Trace the exact path from root to the leaf that produced the prediction, reading off each threshold the sample crossed — export_text() or plot_tree() make this literal, unlike most other model types where "why" requires a separate explainability technique.
Practice Question
Modify the workflow above to also tune min_samples_leaf alongside max_depth in the grid search.
