Mahalanobis and Euclidean Distance

The Gaussian Discriminant Functions contain a distance between the pattern $\mathbf{x}$ and the class mean ${\mathbf{\mu }}_i$. This distance is the Mahalanobis distance. It scales the distance by the covariance of the class. The Euclidean distance does not.

Main idea

All points on one ellipse of equal probability density have the same Mahalanobis distance to the mean, but different Euclidean distances.

---

Definitions

Square of the Mahalanobis distance between $\mathbf{x}$ and ${\mathbf{\mu }}_i$:

$$d_{{\Sigma }_i}(\mathbf{x},{\mathbf{\mu }}_i)=(\mathbf{x}-{\mathbf{\mu }}_i{)}^T{\mathbf{\Sigma }}_i^{-1}(\mathbf{x}-{\mathbf{\mu }}_i)$$

Square of the Euclidean distance:

$$d_{Eu}(\mathbf{x},{\mathbf{\mu }}_i)=(\mathbf{x}-{\mathbf{\mu }}_i{)}^T(\mathbf{x}-{\mathbf{\mu }}_i)$$

Both $d_{{\Sigma }_i}$ and $d_{Eu}$ are squared distances. When ${\mathbf{\Sigma }}_i=\mathbf{I}$, the two are the same.

---

1D Case

$$d_{Eu}(x,{\mu }_i)=(x-{\mu }_i{)}^2$$ $$d_{\Sigma }(x,{\mu }_i)=\frac{(x-{\mu }_i{)}^2}{{\sigma }^2}$$

In 1D, the Mahalanobis distance measures the distance in units of the standard deviation $\sigma$. For example, the points $\mu \pm 2\sigma$ have $d_{\Sigma }=4$ for any $\sigma$. The slide marks about 2.5% of the probability in each red tail near $\mu \pm 2\sigma$ (the exact 2.5% point is $\mu \pm 1.96\sigma$).

---

2D Case

$$p(\mathbf{x}|{\omega }_i)=\frac{1}{(2\pi {)}^{d/2}|{\mathbf{\Sigma }}_i{|}^{1/2}}\exp \left[-\frac{1}{2}(\mathbf{x}-{\mathbf{\mu }}_i{)}^T{\mathbf{\Sigma }}_i^{-1}(\mathbf{x}-{\mathbf{\mu }}_i)\right]$$

What the figure shows:

• samples from a 2D Gaussian lie in a cloud centred on the mean ${\mathbf{\mu }}_i$
• the ellipses are lines of equal probability density
• the density depends on $\mathbf{x}$ only through $d_{{\Sigma }_i}(\mathbf{x},{\mathbf{\mu }}_i)$, so each ellipse is a line of constant Mahalanobis distance
• points on one ellipse have different Euclidean distances to the mean

	Mahalanobis $d_{{\Sigma }_i}$	Euclidean $d_{Eu}$
Uses covariance	Yes, through ${\mathbf{\Sigma }}_i^{-1}$	No
Lines of equal distance	Ellipses (ellipsoids)	Circles (spheres)
Classifier that uses it	Quadratic and linear Gaussian classifiers	Minimum distance classifier