Agglomerative vs Divisive Clustering: Understanding Hierarchical Clustering Approaches

Hierarchical Clustering Explained

A clear guide to agglomerative and divisive clustering

Clustering is one of the most fascinating techniques in data science. It helps uncover natural groupings within data by organizing similar data points together.

Among many clustering approaches, hierarchical clustering stands out because it builds clusters step by step, forming a hierarchy.

What Is Hierarchical Clustering?

Hierarchical clustering is a method that builds a tree-like structure of clusters, similar to organizing books into categories and subcategories.

There are two main approaches:

• Agglomerative clustering (bottom-up)
• Divisive clustering (top-down)

Agglomerative Clustering

🔼 Building from the Ground Up

Agglomerative clustering starts with each data point as its own cluster. The closest clusters are repeatedly merged until only one cluster remains or a stopping condition is reached.

How It Works

1. Each data point starts as its own cluster
2. The two closest clusters are identified
3. Those clusters are merged
4. The process repeats

📏 Distance Measurement (Linkage Methods)

Cluster distance can be measured in different ways:

• Single linkage: Closest points between clusters
• Complete linkage: Farthest points between clusters
• Average linkage: Average distance between all points

📊 Simple Example

Given three data points:

• A to B = 2 units
• A to C = 5 units
• B to C = 4 units

Agglomerative clustering would merge A and B first because they are closest.

Advantages

• Easy to understand and implement
• No need to predefine number of clusters

Drawbacks

• Computationally expensive for large datasets
• Early mistakes cannot be undone

Divisive Clustering

🔽 Splitting from the Top Down

Divisive clustering begins with all data points in one cluster and repeatedly splits clusters into smaller groups.

How It Works

1. Start with one large cluster
2. Find the most dissimilar data points
3. Split the cluster
4. Repeat until stopping criteria are met

🌳 Intuition

Divisive clustering is like pruning a tree. You start with the whole tree and trim branches until distinct groups of leaves remain.

Advantages

• Considers the global structure of data
• Can avoid early poor decisions
• Useful for clearly separated datasets

Drawbacks

• More computationally expensive
• Less intuitive than agglomerative methods

Agglomerative vs Divisive

Aspect	Agglomerative	Divisive
Approach	Bottom-up	Top-down
Starting Point	Individual data points	One large cluster
Early Decisions	Irreversible merges	More global evaluation
Complexity	Moderate to high	High
Typical Use	Small to medium datasets	Well-separated data

Conclusion

Agglomerative clustering is often the go-to choice due to its simplicity and intuition, especially for smaller datasets.

Divisive clustering, while more computationally demanding, can provide better results when the data naturally forms large, distinct groups.

Both approaches are valuable tools in hierarchical clustering and can reveal meaningful patterns in your data when used appropriately.

💡 Key Takeaways

• Hierarchical clustering builds a tree of clusters
• Agglomerative = bottom-up merging
• Divisive = top-down splitting
• Distance metrics strongly influence results
• Choice depends on data size and structure

Educational guide to hierarchical clustering in data science