For classification, Random Forest predicts by majority vote across all its trees — and its predicted probability is simply the fraction of trees that voted for each class, giving a naturally calibrated-feeling confidence score for free.
Formula
\(T_i(x)\) is tree \(i\)'s prediction for input \(x\), and \(m\) is the number of trees. The probability formula counts what fraction of all \(m\) trees voted for class \(c\) — exactly the "3 out of 5 trees voted 1" style calculation from the Random Forest hub note's diagram.
Python Implementation
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, classification_report
from sklearn.datasets import load_breast_cancer
data = load_breast_cancer()
X_train, X_test, y_train, y_test = train_test_split(
data.data, data.target, test_size=0.2, random_state=42, stratify=data.target
)
model = RandomForestClassifier(n_estimators=200, random_state=42)
model.fit(X_train, y_train)
predictions = model.predict(X_test)
probabilities = model.predict_proba(X_test)
print(accuracy_score(y_test, predictions))
print(classification_report(y_test, predictions))
print(probabilities[:5]) # e.g. [[0.02, 0.98], [0.85, 0.15], ...] -- vote fractions per classWhy Random Forest Classification Beats a Single Tree in Practice
A single decision tree's decision boundary is jagged and unstable — small changes in training data can flip which specific splits get chosen. Random Forest's boundary, being an average across many differently-trained trees, tends to be smoother and more stable, closer to the true underlying pattern rather than any one tree's noise-influenced shape.
Practical Use Cases
- Fraud detection, churn prediction, medical diagnosis support — anywhere a robust classification baseline with calibrated-feeling probabilities is valuable
- Multi-class problems — Random Forest handles more than 2 classes natively, same as a single tree
Common Mistakes
- Reporting only the hard class prediction when the vote fraction (probability) carries genuinely useful confidence information — a 51%-vs-49% vote and a 95%-vs-5% vote are very different situations that a hard label alone hides.
- Not addressing class imbalance — Random Forest can still be biased toward the majority class without
class_weight="balanced"or resampling; see Imbalanced Data.
Interview Relevance
Q: "How does Random Forest compute predict_proba, given that individual trees only output hard class labels?" It's the fraction of trees in the forest that voted for each class — e.g. if 180 out of 200 trees predict class 1, the reported probability for class 1 is 0.90 — not a probability any single tree calculated directly, but an emergent property of the ensemble vote.
Practice Question
A Random Forest with 50 trees gives predict_proba output [0.62, 0.38] for a sample. How many individual trees voted for the first class?
