Bhattacharyya coefficient

In statistics, the Bhattacharyya distance measures the similarity of two probability distributions. It is closely related to the Bhattacharyya coefficient which is a measure of the amount of overlap between two statistical samples or populations.

It is not a metric, despite named a "distance", since it does not obey the triangle inequality.

Definition

For probability distributions $P$ and $Q$ on the same domain $\mathcal$, the Bhattacharyya distance is defined as
:$D_B(P,Q) = -\ln \left( BC(P,Q) \right)$
where
:$BC(P,Q) = \sum_ \sqrt$
is the Bhattacharyya coefficient for discrete probability distributions.

For continuous probability distributions, with $P(dx) = p(x)dx$ and $Q(dx) = q(x) dx$ where $p(x)$ and $q(x)$ are the probability density functions, the Bhattacharyya coefficient is defined as
:$BC(P,Q) = \int_ \sqrt\, dx$.

More generally, given two probability measures $P, Q$ on a measurable space $(\mathcal X, \mathcal B)$, let $\lambda$ be a ( sigma finite) measure such that $P$ and $Q$ are absolutely continuous with respect to $\lambda$ i.e. such that $P(dx) = p(x)\lambda(dx)$, and $Q(dx) = q(x)\lambda(dx)$ for probability density functions $p, q$ with respect to $\lambda$ defined $\lambda$-almost everywhere. Such a measure, even such a probability measure, always exists, e.g. $\lambda = \tfrac12(P + Q)$. Then define the Bhattacharrya measure on $(\mathcal X, \mathcal B)$ by
:$bc(dx , P,Q) = \sqrt\, \lambda(dx) = \sqrt\lambda(dx).$
It does not depend on the measure $\lambda$, for if we choose a measure $\mu$ such that $\lambda$ and an other measure choice $\lambda'$ are absolutely continuous i.e. $\lambda = l(x)\mu$ and $\lambda' = l'(x) \mu$, then
:$P(dx) = p(x)\lambda(dx) = p'(x)\lambda'(dx) = p(x)l(x) \mu(dx) = p'(x)l'(x)\mu(dx)$,
and similarly for $Q$. We then have
:$bc(dx , P,Q) = \sqrt\, \lambda(dx) = \sqrt\, l(x)\mu(x) = \sqrt\mu(dx) = \sqrt\, \mu(dx) = \sqrt\,\lambda'(dx)$.
We finally define the Bhattacharyya coefficient
:$BC(P,Q) = \int_ bc(dx, P,Q) = \int_ \sqrt\, \lambda(dx)$.
By the above, the quantity $BC(P,Q)$ does not depend on $\lambda$, and by the Cauchy inequality $0\le BC(P,Q) \le 1$. In particular if $P(dx) = p(x)Q(dx)$ is absolutely continuous wrt to $Q$ with Radon Nikodym derivative $p(x) = \frac(x)$, then BC(P, Q) = \int_ \sqrt Q(dx) = \int_ \sqrt Q(dx) = E_Q\left sqrt\right

Properties

$0 \le BC \le 1$ and $0 \le D_B \le \infty$.

$D_B$ does not obey the triangle inequality, though the Hellinger distance $\sqrt$ does.

Let $p\sim\mathcal(\mu_p,\sigma_p^2)$, $q\sim\mathcal(\mu_q,\sigma_q^2)$, where $(\mu ,\sigma ^)$ is the normal distribution with mean $\mu$ and variance $\sigma^2$, then

:$D_B(p,q) = \frac \frac + \frac 1 2 \ln\left(\frac\right)$

And in general, given two multivariate normal distributions $p_i=\mathcal(\boldsymbol\mu_i,\,\boldsymbol\Sigma_i)$,
: $D_B(p_1, p_2)=(\boldsymbol\mu_1-\boldsymbol\mu_2)^T \boldsymbol\Sigma^(\boldsymbol\mu_1-\boldsymbol\mu_2)+\ln \,\left(\right),$

where $\boldsymbol\Sigma=.$ Note that the first term is a squared Mahalanobis distance.

Applications

The Bhattacharyya coefficient quantifies the "closeness" of two random statistical samples.

Given two sequences from distributions $P, Q$, bin them into $n$ buckets, and let the frequency of samples from $P$ in bucket $i$ be $p_i$, and similarly for $q_i$, then the sample Bhattacharyya coefficient is

:$BC(\mathbf,\mathbf) = \sum_^n \sqrt,$

which is an estimator of $BC(P, Q)$. The quality of estimation depends on the choice of buckets; too few buckets would overestimate $BC(P, Q)$, while too many would underestimate..

A common task in classification is estimating the separability of classes. Up to a multiplicative factor, the squared Mahalanobis distance is a special case of the Bhattacharyya distance when the two classes are normally distributed with the same variances. When two classes have similar means but significantly different variances, the Mahalanobis distance would be close to zero, while the Bhattacharyya distance would not be.

The Bhattacharyya coefficient is used in the construction of polar codes.

The Bhattacharyya distance is used in feature extraction and selection,Euisun Choi, Chulhee Lee, "Feature extraction based on the Bhattacharyya distance", ''Pattern Recognition'', Volume 36, Issue 8, August 2003, Pages 1703–1709 image processing,François Goudail, Philippe Réfrégier, Guillaume Delyon, "Bhattacharyya distance as a contrast parameter for statistical processing of noisy optical images", ''JOSA A'', Vol. 21, Issue 7, pp. 1231−1240 (2004) speaker recognition,Chang Huai You, "An SVM Kernel With GMM-Supervector Based on the Bhattacharyya Distance for Speaker Recognition", ''Signal Processing Letters'', IEEE, Vol 16, Is 1, pp. 49-52 and phone clustering.Mak, B., "Phone clustering using the Bhattacharyya distance", ''Spoken Language'', 1996. ICSLP 96. Proceedings., Fourth International Conference on, Vol 4, pp. 2005–2008 vol.4, 3−6 Oct 1996

A "Bhattacharyya space" has been proposed as a feature selection technique that can be applied to texture segmentation.Reyes-Aldasoro, C.C., and A. Bhalerao, "The Bhattacharyya space for feature selection and its application to texture segmentation", ''Pattern Recognition'', (2006) Vol. 39, Issue 5, May 2006, pp. 812–826

History

Both the Bhattacharyya distance and the Bhattacharyya coefficient are named after Anil Kumar Bhattacharyya, a statistician who worked in the 1930s at the Indian Statistical Institute. He developed the method to measure the distance between two non-normal distributions and illustrated this with the classical multinomial populations as well as probability distributions that are absolutely continuous with respect to the Lebesgue measure. The latter work appeared partly in 1943 in the Bulletin of the Calcutta Mathematical Society, while the former part, despite being submitted for publication in 1941, appeared almost five years later in Sankhya.

See also

* Bhattacharyya angle
* Kullback–Leibler divergence
* Hellinger distance
* Mahalanobis distance
* Chernoff bound
* Rényi entropy
* F-divergence
* Fidelity of quantum states

References

*

*

*

* For a short list of properties, see: http://www.mtm.ufsc.br/~taneja/book/node20.html

External links

*

Bhattacharyya's distance measure as a precursor of genetic distance measures
Journal of Biosciences, 2004

Statistical Intuition of Bhattacharyya's distance

{{DEFAULTSORT:Bhattacharyya Distance
Statistical distance
Statistical deviation and dispersion
Anil Kumar Bhattacharya