A High Silhouette Score Does Not Mean Your Clusters Are Real

A common clustering workflow looks harmless:

1. fit several values of $k$,
2. compute the average silhouette score,
3. choose the largest value,
4. describe the resulting groups as the structure discovered in the data.

The first three steps can be useful.

The fourth does not follow from them.

Silhouette width measures how well a partition separates observations under a chosen distance. It does not test whether the population was generated by distinct latent groups, whether the groups correspond to different mechanisms, or whether the selected partition is unique.

A high silhouette score answers a geometric question.

It is often interpreted as an ontological answer.

Those are not the same thing.

What the Silhouette Score Actually Measures

Suppose observation $i$ has been assigned to cluster $C_i$.

Let

be the average distance from observation $i$ to the other observations in its own cluster.

For every other cluster $C$, compute the average distance from $i$ to observations in $C$. Let the smallest of those averages be

The silhouette width is

Its range is

If

then observation $i$ is much closer, on average, to its assigned cluster than to the nearest competing cluster and $s(i)$ approaches one.

If

the assignment is geometrically ambiguous.

If

the observation is, on average, closer to another cluster than to its assigned one, so the silhouette becomes negative.

The average silhouette,

is therefore a measure of cohesion and separation under the distance function used to construct the calculation.

That is already useful.

But notice what is absent from the definition.

There is no term for:

• a population mixture,
• a latent class,
• a generative mechanism,
• a density valley,
• scientific relevance,
• reproducibility under new sampling,
• or an external outcome.

The score cannot establish properties it never tries to measure.

A Single Uniform Distribution Can Have a Strong Silhouette

Consider the simplest possible continuous population,

There is one uniform density.

No mixture components are present. There is no discontinuity at zero. There is no density valley separating two populations.

The density is simply

Now impose the partition

Nothing in the distribution selects zero as a latent class boundary. We selected it.

Nevertheless, the silhouette score likes the partition.

Let $x\in[-1,0]$.

For a point in the left cluster, the expected within-cluster distance is

Splitting the integral at $x$ gives

Because the interval has length one, no additional normalising factor is needed.

After simplification,

The average distance to the other cluster is

Since $V\geq0\geq x$,

so

For $x\in[-1,0]$ we have

and therefore

Substituting the expressions above,

The expected silhouette within the left cluster is

The integral has the closed form

By symmetry, the right cluster has the same expected silhouette.

Hence the population average is

for a distribution with no density separation whatsoever.

A number around $0.63$ would often be described informally as evidence of reasonably well-separated clusters.

Here it comes from cutting one flat continuous population in half.

Why the Counterexample Works

The silhouette coefficient does exactly what it was designed to do.

Once we choose the boundary at zero, points in the left half are usually closer to other points in the left half than to points in the right half. The same is true on the other side.

For example, at

the average distance to the left half is

while the average distance to the right half is

Therefore

That is a strong local silhouette.

Yet the density immediately to the left of zero is exactly the same as the density immediately to the right:

There is no statistical discontinuity for the silhouette to discover.

The score rewards the geometry of the imposed partition.

It does not test whether the boundary corresponds to a change in the data-generating mechanism.

Every Partition Creates Within and Between Distances

Once observations are partitioned, two classes of pairwise distances exist automatically:

and

A method that deliberately forms compact groups will tend to make the first smaller than the second.

An internal metric then measures exactly that difference.

There is nothing circular about using such a metric to compare candidate partitions under the same geometric objective. The problem appears when the score is given a stronger interpretation than the construction supports.

The logic

is reasonable.

The logic

is not.

The second statement needs additional evidence.

K-Means and Silhouette Share Geometric Preferences

K-means minimises

This encourages low within-cluster dispersion under Euclidean geometry.

Silhouette also rewards small within-cluster distances relative to distances to competing clusters.

The objectives are not identical, but they are aligned.

That alignment makes silhouette a useful diagnostic for K-means-like partitions.

It also means that a strong score is partly evidence that the partition has the geometry both procedures prefer.

If the scientific question concerns elongated structures, density-connected regions, temporal shapes or graph communities, Euclidean silhouette may reward the wrong notion of a cluster.

The metric is not neutral.

It inherits the meaning of distance.

Scaling Can Change the Validation Result

Suppose observations have two coordinates,

Under Euclidean distance,

Now rescale the second coordinate:

Distances become

The within-cluster quantities $a(i)$ and between-cluster quantities $b(i)$ change.

Therefore the silhouette changes.

This means that the same observations and the same hard labels can receive a different validation score after a defensible change of units or preprocessing.

That is not a bug.

It is a reminder that internal validation is conditional on a representation.

A statement such as

the clustering has silhouette 0.71

is incomplete without the geometry under which the score was computed.

A Maximum Over $k$ Is Still Conditional on the Candidate Family

Suppose we evaluate

and select

This answers

Which of these candidate partitions has the strongest average silhouette?

It does not answer

Does the population consist of exactly $\hat{k}$ groups?

The distinction is easy to miss because the optimisation produces a single integer.

That integer looks like an estimate of a population parameter.

But unless the statistical model actually defines a population quantity called the number of clusters, $\hat{k}$ is better understood as a tuning choice selected by a criterion.

The candidate family matters too.

If only K-means solutions are compared, the result selects the best K-means geometry among the evaluated values of $k$.

A density-based method, Gaussian mixture, hierarchical method or spectral construction may produce a different partition because it encodes a different cluster concept.

Internal Validation Is Not a Hypothesis Test

A silhouette value does not come with the null hypothesis

Nor does it automatically provide a sampling distribution under such a null.

So a threshold such as

is not analogous to rejecting a statistical hypothesis at a predefined significance level.

It is a rule of thumb.

Rules of thumb can be useful operationally. Problems begin when the threshold is interpreted as inferential evidence with a meaning it was never calibrated to have.

If the scientific question is whether observed clustering structure exceeds what could arise under an appropriate null model, then the null model has to be introduced explicitly.

Null Models Ask a Better Question

The gap statistic is one example of this philosophy.

Rather than considering within-cluster dispersion in isolation, it compares the observed value with what would be expected under a reference distribution with no comparable clustering structure.

Write

for a within-cluster dispersion measure using $k$ groups.

The gap statistic has the form

where the expectation is taken under a reference distribution.

The important idea is not the exact formula.

It is the comparison

That is closer to the inferential question many users mistakenly ask silhouette to answer.

But the gap statistic does not remove modelling choices either.

The reference distribution matters.

A uniform null over a bounding box, a principal-component-aligned reference, a unimodal Gaussian null and a process-specific null can lead to different conclusions because they define "absence of clustering" differently.

There is no model-free null hypothesis for cluster existence.

A Null Must Preserve What Is Not Under Test

Suppose the observed data have strong marginal skewness and feature correlation.

If the null simulation destroys both, then an observed clustering score may look unusual simply because the null data have unrealistic geometry.

A useful null model should preserve aspects of the data that are not meant to count as clustering evidence.

Depending on the application, that may include:

• marginal distributions,
• covariance structure,
• temporal autocorrelation,
• spatial dependence,
• varying measurement precision,
• known seasonality,
• graph degree structure,
• or sampling constraints.

The correct null for a time series is rarely independent Gaussian noise.

The correct null for spatial measurements is rarely a random cloud with no spatial dependence.

The validation question determines which structure should remain.

Stability Answers Yet Another Question

A clustering can also be evaluated by perturbing the data.

For example:

1. resample observations,
2. perturb measurements,
3. rerun the clustering,
4. align the resulting groups,
5. measure how often pairs remain together.

Define the co-clustering probability

Values close to one indicate that observations $i$ and $j$ repeatedly appear in the same group.

This is useful information.

But stability asks

Would a similar sample produce a similar partition?

It does not ask

Is the partition a set of true latent populations?

The uniform example can again be instructive.

A midpoint-type split of a symmetric continuous distribution can be highly reproducible under resampling. Its stability would tell us that the geometry consistently supports a similar split.

That still would not create a density gap that is absent from the population.

Stable does not mean discrete.

External Validation Changes the Source of Evidence

Suppose independent labels or outcomes exist after the clustering is constructed.

One can compare cluster assignments with those external variables.

For two partitions, common measures include the adjusted Rand index, normalized mutual information and Fowlkes-Mallows index.

That moves the evaluation beyond the geometry used to fit the clusters.

It can provide stronger evidence if the external variable has a clear scientific interpretation and was not used to manufacture the representation.

For example, a clustering of disease trajectories might later be compared with treatment response, complications or a biomarker measured independently.

If the groups differ reproducibly on such quantities, the clustering may have practical or scientific value even if its internal silhouette is moderate.

Conversely, a high silhouette with no external consequences may simply describe a convenient geometric partition.

External Agreement Is Still Question-Dependent

Known categories are not automatically ground truth for every clustering problem.

Suppose customer trajectories are clustered by purchasing dynamics.

Demographic segments may be available, but there is no reason the trajectory clusters must reproduce age groups or geographic categories.

A low adjusted Rand index against demographics does not invalidate a clustering designed for another purpose.

Similarly, if diagnostic categories are broad but the clustering is intended to identify treatment-response phenotypes, reproducing the diagnosis codes may not be the relevant goal.

External validation is strongest when the external target corresponds to a pre-specified interpretation of the proposed groups.

Otherwise, it is another comparison between two partitions with different meanings.

Predictive Validation Can Be More Useful Than Internal Geometry

In some applications, a cluster is useful because membership predicts something that matters later.

Let

be the estimated cluster label and let

be an independently observed future outcome.

A practical question is whether

differs meaningfully across groups and whether that difference replicates out of sample.

This changes the role of clustering.

The groups no longer need to be interpreted as immutable natural kinds. They can be treated as a useful low-dimensional summary of heterogeneity.

That is often a more defensible claim.

For example:

This partition is reproducible and separates populations with different future behaviour.

is more precise than

The algorithm discovered the three natural types of observation.

The first statement can be tested.

The second usually requires much stronger assumptions.

Validation Should Match the Claim

Different claims require different evidence.

If the claim is

the clusters are compact and separated under Euclidean distance,

then internal metrics such as silhouette are directly relevant.

If the claim is

the grouping is reproducible,

then resampling and perturbation stability are relevant.

If the claim is

the data contain more grouping structure than expected from a continuous null,

then explicit null-model comparisons are needed.

If the claim is

the clusters correspond to known classes,

then external agreement metrics are appropriate.

If the claim is

the groups identify populations with different future outcomes,

then out-of-sample predictive or inferential validation is needed.

If the claim is

the clusters correspond to distinct causal mechanisms,

none of the preceding evidence is sufficient by itself.

The validation design should be derived from the interpretation we intend to make.

One Metric Cannot Validate the Entire Pipeline

A clustering result depends on more than the final partition.

A simplified pipeline is

Here,

• $\phi$ is the representation,
• $d$ is the similarity geometry,
• $\mathcal{A}$ is the clustering algorithm,
• $\lambda$ denotes tuning choices,
• and $V$ is the validation criterion.

If all of these are selected using the same dataset and the final score is reported as if it were an independent assessment, the apparent evidence is optimistic.

This is the unsupervised analogue of tuning a supervised model on a test set.

The analogy is not exact because there may be no labels, but the selection problem is real.

Trying many representations, distances, algorithms and values of $k$ and then reporting only the highest silhouette creates a selection bias.

The winning score is the maximum of many noisy comparisons.

Report the Search, Not Only the Winner

Suppose an analyst tries

representations,

distance definitions,

algorithms or parameterisations,

and

candidate cluster counts.

That is already

candidate analyses.

Even if many are correlated, selecting the largest internal score from that search changes how the result should be interpreted.

A responsible report should retain information about the search space.

At minimum:

• which representations were considered,
• which distance functions were considered,
• which algorithms were compared,
• which hyperparameters were varied,
• whether the final validation score participated in selection,
• and how sensitive the conclusion is to reasonable alternatives.

Unsupervised learning does not remove researcher degrees of freedom.

It can increase them.

A More Defensible Validation Stack

For clustering that will carry substantive interpretation, I would not rely on one number.

I would use several layers.

Layer 1: internal geometry

Report quantities such as silhouette, within-cluster dispersion or likelihood-based diagnostics.

These answer whether the partition fits the geometry assumed by the method.

Layer 2: null comparison

Compare the observed structure with data generated under a scientifically defensible no-cluster or reduced-structure model.

This asks whether the observed grouping exceeds what the null geometry can produce.

Layer 3: stability

Repeat the procedure under resampling, noise perturbation, alternative initialisation and modest preprocessing changes.

This asks whether the partition is reproducible.

Layer 4: assignment uncertainty

Retain co-clustering probabilities, posterior membership probabilities or variation across plausible pipelines.

This prevents hard labels from hiding ambiguous cases.

Layer 5: external validation

Where possible, test relationships with variables not used in fitting the clusters.

This asks whether the groups carry information outside the clustering objective.

Layer 6: domain meaning

Finally, ask whether the interpretation of the groups is coherent with what is known about the process that generated the data.

This is not a substitute for quantitative validation.

It is the step that prevents a mathematically neat partition from being given a scientific meaning it does not support.

The Uniform Example Is the Main Warning

The uniform calculation is deliberately simple because it removes distractions.

There are no outliers.

There are no nonlinear manifolds.

There is no high-dimensional concentration.

There is no optimisation failure.

There is no misspecified mixture model.

There is only

and a partition at zero.

Yet

The number is not wrong.

Our interpretation can be.

The score says that, conditional on the imposed two-group partition and Euclidean distance, observations tend to be closer to members of their assigned half than to observations in the other half.

That is true.

It does not say that the uniform population secretly contains two latent classes.

That claim is not encoded in the statistic.

Conclusion

The silhouette score is a useful clustering diagnostic.

It should remain one.

Its strength is that it summarises a clear geometric idea:

Its weakness appears only when we ask it to answer a different question.

A high silhouette does not establish that groups are natural, unique, generative, scientifically meaningful or even separated by a density valley.

The uniform counterexample shows this directly:

One continuous population can therefore produce a strong-looking silhouette under a chosen partition.

The useful question is not

Is the silhouette high?

It is

What claim about the clustering is this validation procedure capable of supporting?

Once the claim is stated precisely, the correct validation design becomes much clearer.

References

• Hennig, C. (2015). What are the true clusters? Pattern Recognition Letters, 64, 53–62. https://doi.org/10.1016/j.patrec.2015.04.009
• Rousseeuw, P. J. (1987). Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics, 20, 53–65. https://doi.org/10.1016/0377-0427(87)90125-7
• Tibshirani, R., Walther, G., & Hastie, T. (2001). Estimating the number of clusters in a data set via the gap statistic. Journal of the Royal Statistical Society: Series B, 63(2), 411–423. https://doi.org/10.1111/1467-9868.00293
• von Luxburg, U. (2010). Clustering stability: An overview. Foundations and Trends in Machine Learning, 2(3), 235–274. https://doi.org/10.1561/2200000008