Hierarchical Clustering — Tutorial

Why hierarchical over K-Means?

K-Means requires you to specify k — the number of clusters — before you run it. Hierarchical clustering builds a full tree of possible groupings, letting you choose k after the fact by cutting the tree at different levels.

Two strategies:

Agglomerative (bottom-up): start with every point as its own cluster. Merge the two closest clusters repeatedly until one remains. This is the common default.

Divisive (top-down): start with all points in one cluster. Recursively split the least cohesive cluster. Rarely used in practice because it is more expensive.

The output is a dendrogram — a tree diagram where the vertical axis shows the distance (or dissimilarity) at which clusters were merged. Tall branches mean the two clusters were far apart when merged; short branches mean they were close.

Linkage criteria

How you measure the distance between two clusters (each containing multiple points) is the linkage criterion. It changes the shape of the clusters dramatically:

Single linkage: distance between clusters = distance between their two nearest points. Creates elongated, chain-like clusters. Very sensitive to outliers.

Complete linkage: distance = distance between two furthest points. Produces compact, roughly equal-sized clusters. More robust to outliers.

Average linkage: distance = average of all pairwise distances between the two clusters. Good general-purpose choice.

Ward linkage: merges the pair of clusters that minimises the increase in total within-cluster variance. Tends to produce compact, similarly-sized clusters. The most popular choice for numerical data.

The choice of linkage + distance metric (Euclidean, cosine, Manhattan) fully determines the clustering result.

Reading a dendrogram and choosing k

To read a dendrogram: draw a horizontal line across the tree. The number of vertical lines it crosses is the number of clusters k at that cut height. The longer the vertical branches at the cut point, the more distinct the clusters.

A practical heuristic: look for the largest vertical gap (the tallest undivided branch). Cut just below the merge that creates that gap — it represents the most significant jump in dissimilarity between levels.

Agglomerative clustering scales as O(n² log n) for most linkages (O(n³) for single and complete without optimisation). For datasets above ~10,000 points, it becomes slow. Alternatives: BIRCH for large datasets, or mini-batch hierarchical methods.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Hierarchical Clustering — Tutorial.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path