Random Forest in Python

A complete, production-shaped Random Forest workflow — training, tuning the key hyperparameters via cross-validation, and using the OOB score as a fast sanity check along the way.

The Full Workflow

from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import classification_report
from sklearn.datasets import load_breast_cancer

# 1. Dataset
data = load_breast_cancer()
X, y = data.data, data.target

# 2. Train/test split
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

# 3. Quick OOB check before formal tuning -- a fast sanity signal, not a substitute for CV
quick_model = RandomForestClassifier(n_estimators=300, oob_score=True, random_state=42)
quick_model.fit(X_train, y_train)
print("OOB score:", quick_model.oob_score_)

# 4. Tune the key hyperparameters via cross-validation
param_grid = {
    "n_estimators": [100, 300],
    "max_depth": [None, 6, 10],
    "max_features": ["sqrt", "log2"],
    "min_samples_leaf": [1, 3, 5],
}
grid_search = GridSearchCV(
    RandomForestClassifier(random_state=42), param_grid, cv=5, scoring="accuracy", n_jobs=-1
)
grid_search.fit(X_train, y_train)
print("Best params:", grid_search.best_params_)

# 5. Evaluate the tuned model
best_model = grid_search.best_estimator_
predictions = best_model.predict(X_test)
print(classification_report(y_test, predictions))

The Hyperparameters That Actually Matter Most

Parallelizing Training

# n_jobs=-1 uses all available CPU cores -- trees are independent, so this parallelizes cleanly
model = RandomForestClassifier(n_estimators=300, n_jobs=-1, random_state=42)
model.fit(X_train, y_train)

Because every tree in a Random Forest trains completely independently (unlike boosting, where trees depend sequentially on previous ones), training parallelizes almost perfectly across CPU cores — a genuine practical advantage over sequential ensemble methods.

Regression — The Same Pattern

from sklearn.ensemble import RandomForestRegressor

reg_model = RandomForestRegressor(n_estimators=300, max_features="sqrt", n_jobs=-1, random_state=42)
reg_model.fit(X_train_reg, y_train_reg)

Common Mistakes

• Running a massive grid search over every hyperparameter simultaneously — start with n_estimators and max_depth, which usually matter most, before expanding the search.
• Forgetting random_state, making results hard to reproduce across runs given the algorithm's inherent randomness.
• Not using n_jobs=-1 on a multi-core machine, leaving significant training speed on the table for no reason.

Interview Relevance

Q: "Why does Random Forest training parallelize better than gradient boosting?" Random Forest's trees are trained completely independently of each other (each on its own bootstrap sample), so they can all be built simultaneously across CPU cores; gradient boosting trains trees sequentially, where each new tree depends on the errors of all previous ones, making that parallelization impossible.

Practice Question

Modify the grid search above to also include criterion (Gini vs entropy) as a tuned hyperparameter.

Related ML Notes

• Hyperparameter Tuning
• Grid Search
• XGBoost