Source

Adapted from my own handwritten and hand-typed notes from Year 1 to Year 3 of my EEE degree

The elbow method is a way to choose the number of clusters for k-means.

Within Cluster Sum of Squares

WSS adds up, over all clusters, the squared distances of each point to its own centroid:

Lower WSS means tighter clusters. For Pokemon Total vs Speed with , WSS was about 2,118,651.

Behaviour as K Grows

Two extremes:

  • : one cluster holds everything, WSS is at its maximum
  • (number of Pokemon): every point is its own centroid, WSS is 0

So WSS always falls as grows, and the minimum WSS is not the answer.

Elbow Plot

Plot WSS against . The curve drops quickly at first, then flattens. The bend (the elbow) is where adding a cluster stops giving much gain, a good choice of .

Results in the course:

  • Total and Speed: elbow guess
  • six stats (HP, Attack, Defense, Sp. Atk, Sp. Def, Speed): elbow guess

The elbow is a guess. The final choice should also make the clusters interpretable.

Code

min_clust, max_clust = 1, 40
within_ss = []
for k in range(min_clust, max_clust + 1):
    kmeans = KMeans(n_clusters=k, n_init=10)
    kmeans.fit(X)
    within_ss.append(kmeans.inertia_)
 
plt.figure(figsize=(16, 6))
plt.plot(range(min_clust, max_clust + 1), within_ss, marker='o')
plt.xlabel('Number of Clusters K')
plt.ylabel('Within SS')