Agglomerative Clustering

Agglomerative clustering is hierarchical clustering's standard bottom-up algorithm — the specific step-by-step merging process that builds the dendrogram, and the version you'll actually run in practice.

The Algorithm, As Explicit Steps

This is exactly the process worked through by hand in Hierarchical Clustering — this note focuses on running it directly in code and choosing the final cluster count.

Python Implementation

from sklearn.cluster import AgglomerativeClustering
import numpy as np

X = np.array([[1,1],[2,1],[5,5],[6,5]])

model = AgglomerativeClustering(n_clusters=2, linkage="single")
labels = model.fit_predict(X)
print(labels)   # [0 0 1 1] -- {A,B} and {C,D}, matching the dendrogram cut at height 5

Choosing n_clusters vs distance_threshold

# Option 1: specify the number of clusters directly (like K-Means)
model_by_k = AgglomerativeClustering(n_clusters=2, linkage="ward")

# Option 2: let the algorithm decide how many clusters, by cutting the dendrogram
# at a specific DISTANCE instead of a specific COUNT
model_by_distance = AgglomerativeClustering(n_clusters=None, distance_threshold=3, linkage="single")
labels = model_by_distance.fit_predict(X)
print(labels)

Specifying distance_threshold instead of n_clusters lets the data itself determine how many clusters result — useful when you have a meaningful sense of "how different is too different" but no fixed target cluster count.

Comparing Linkage Criteria on the Same Data

for linkage_method in ["single", "complete", "average", "ward"]:
    model = AgglomerativeClustering(n_clusters=2, linkage=linkage_method)
    labels = model.fit_predict(X)
    print(linkage_method, labels)
# On this clean, well-separated 4-point example, all four linkage methods
# likely agree -- but on messier real data, the choice of linkage can matter a lot

Practical Use Cases

• Small-to-medium datasets where a dendrogram's full structure adds interpretive value
• Situations where the natural number of clusters is genuinely unclear, and exploring the dendrogram helps decide

Common Mistakes

• Applying agglomerative clustering to a very large dataset without considering its quadratic-or-worse computational cost — it doesn't scale the way K-Means does.
• Not comparing multiple linkage criteria — the "best" linkage genuinely depends on the data's shape, and there's no universal default that's always correct.

Interview Relevance

Q: "How would you decide the number of clusters from a dendrogram?" Look for the tallest vertical gap between merge heights — cutting through that gap (rather than through a cluster of closely-spaced merges) typically produces the most natural, stable grouping; this is a visual analogue of the elbow method's "diminishing returns" logic.

Practice Question

Modify the code above to use linkage="complete" and compare the resulting labels to the single-linkage result on the same 4-point dataset.

Related ML Notes

• Hierarchical Clustering
• DBSCAN
• Clustering Evaluation