Gini impurity measures how "mixed" a node's classes are — 0 means a node is perfectly pure (all one class), and it rises as the split between classes gets closer to even.
Formula
\(p_i\) is the proportion of class \(i\) in the node, and \(c\) is the number of classes. For binary classification, Gini ranges from 0 (pure) to a maximum of 0.5 (perfectly split 50/50).
Graphical Intuition — Gini Across All Possible Splits
Gini is 0 at the pure extremes (p=0 or p=1) and peaks at 0.5 for a perfectly balanced node — a decision tree always tries to move away from the peak, toward the edges.
Numerical Example
Root node from Decision Tree Classification: 6 Yes, 4 No out of 10 samples.
After splitting on "Sunny?": the Sunny group (1 Yes, 3 No) has \(\text{Gini}=1-(0.25^2+0.75^2)=0.375\); the Not-Sunny group (5 Yes, 1 No) has \(\text{Gini}\approx 0.278\) — both lower than the root's 0.48, confirming the split genuinely reduced impurity.
def gini(labels):
from collections import Counter
counts = Counter(labels)
n = len(labels)
return 1 - sum((count/n)**2 for count in counts.values())
root = [1,1,1,1,1,1,0,0,0,0] # 6 Yes, 4 No
sunny = [1,0,0,0] # 1 Yes, 3 No
not_sunny = [1,1,1,1,1,0] # 5 Yes, 1 No
print(gini(root)) # 0.48
print(gini(sunny)) # 0.375
print(gini(not_sunny)) # 0.2778Gini vs Entropy — Do They Ever Disagree?
Both measure the same underlying concept (node impurity) and usually select the same or very similar splits. Gini is slightly faster to compute (no logarithm), which is why scikit-learn's DecisionTreeClassifier defaults to it. See Entropy for the alternative, information-theoretic formulation.
Practical Use Cases
- The default split criterion in scikit-learn's decision trees and Random Forest
- Quickly comparing how "clean" a proposed split's resulting groups would be
Common Mistakes
- Confusing Gini impurity (used in decision trees) with the unrelated Gini coefficient (used in economics to measure income inequality) — same name, different formula, different purpose.
- Expecting Gini of exactly 0 at every leaf — a tree stopped early (via max depth or minimum leaf size) can have leaves that aren't perfectly pure, by design.
Interview Relevance
Q: "What does a Gini impurity of 0 mean for a node?" The node is perfectly pure — every sample in it belongs to the same class; a tree would have no reason to split it further on impurity grounds alone.
Practice Question
Compute the Gini impurity of a node with 8 samples: 2 of class A, 6 of class B.
