Bhattacharyya Distance: Measuring Distribution Overlap

The Bhattacharyya coefficient measures overlap between two probability distributions.

For discrete distributions,

For densities,

The Bhattacharyya distance is

Basic properties

The coefficient satisfies

If $P=Q$, then

and

If the distributions have disjoint support, then

and the distance is infinite.

The distance is symmetric:

It is not generally a metric because the triangle inequality need not hold.

Relation to Hellinger distance

The squared Hellinger distance is

under one common convention.

Thus Bhattacharyya and Hellinger measures are monotone transformations of the same overlap coefficient.

Gaussian case

For one-dimensional normal distributions

the Bhattacharyya distance is

This separates location and scale differences.

Classification connection

The Bhattacharyya coefficient appears in upper bounds on Bayes classification error.

Greater overlap between class-conditional distributions makes classification intrinsically harder.

This is a theoretical connection.

It does not imply that Bhattacharyya distance is itself a training loss for every classifier.

Histogram comparison

In computer vision, normalized histograms can be compared with Bhattacharyya-type measures.

The result depends on binning.

Changing histogram resolution can change the estimated overlap substantially.

Bhattacharyya distance versus KL divergence

Bhattacharyya distance is symmetric.

KL divergence is directional.

KL is based on log density ratios.

Bhattacharyya is based on square-root overlap.

The two should not be treated as interchangeable.

Distribution distance versus loss function

The original article mixed Bhattacharyya distance with a catalogue of regression and classification losses.

These are different objects.

A loss function such as

scores a prediction against an outcome.

Bhattacharyya distance compares two distributions.

The fact that both can appear in machine learning does not make them conceptually one family.

Estimation error

In practice, $P$ and $Q$ are often estimated from data.

The resulting Bhattacharyya distance inherits sampling uncertainty and density-estimation error.

Plug-in estimates in high dimensions can be unstable.

Bootstrap or model-based uncertainty assessment may be appropriate.

Conclusion

Bhattacharyya distance is best understood as an overlap measure between probability distributions.

Its strongest interpretations concern:

• distribution similarity
• class separability
• histogram overlap
• parametric distribution comparison

It should not be bundled indiscriminately with predictive loss functions.

References

• Bhattacharyya, A. (1943). On a Measure of Divergence Between Two Statistical Populations.
• Kailath, T. (1967). The Divergence and Bhattacharyya Distance Measures in Signal Selection.