Time Series Classification by Class-Specific Mahalanobis Distance Measures

To classify time series by nearest neighbors, we need to specify or learn one or several distance measures. We consider variations of the Mahalanobis distance measures which rely on the inverse covariance matrix of the data. Unfortunately — for time series data — the covariance matrix has often low rank. To alleviate this problem we can either use a pseudoinverse, covariance shrinking or limit the matrix to its diagonal. We review these alternatives and benchmark them against competitive methods such as the related Large Margin Nearest Neighbor Classification (LMNN) and the Dynamic Time Warping (DTW) distance. As we expected, we find that the DTW is superior, but the Mahalanobis distance measures are one to two orders of magnitude faster. To get best results with Mahalanobis distance measures, we recommend learning one distance measure per class using either covariance shrinking or the diagonal approach.

💡 Research Summary

The paper investigates how to employ Mahalanobis‑based distance measures for nearest‑neighbor (1‑NN) time‑series classification and how to overcome the low‑rank covariance problem that is typical for high‑dimensional, short‑sample series. Three strategies for handling a singular or ill‑conditioned covariance matrix are examined: (1) using the Moore‑Penrose pseudoinverse, (2) applying Ledoit‑Wolf shrinkage to blend the empirical covariance with its diagonal, and (3) discarding all off‑diagonal terms and retaining only the diagonal variance (a purely uncorrelated metric). In addition to a global distance learned from all training instances, the authors propose learning a separate Mahalanobis matrix for each class, thereby capturing class‑specific variance patterns.

The experimental protocol uses 85 benchmark data sets from the UCR time‑series archive. The proposed variants are compared against three strong baselines: (i) Euclidean 1‑NN, (ii) Large Margin Nearest Neighbor (LMNN), a metric‑learning method that optimises a linear transformation, and (iii) Dynamic Time Warping (DTW), the de‑facto standard for elastic alignment. Accuracy, classification time, and memory consumption are reported.

Results show that DTW remains the most accurate method, especially on data sets where temporal mis‑alignments dominate. However, Mahalanobis‑based classifiers are dramatically faster—typically an order of magnitude (10×) to two orders (100×) faster than DTW—because after a one‑time covariance estimation the distance reduces to a simple quadratic form. Among the Mahalanobis variants, the class‑specific shrinkage (Mahalanobis‑Shrink) and the class‑specific diagonal (Mahalanobis‑Diag) achieve the best trade‑off: they retain enough covariance information to improve accuracy over a global Mahalanobis matrix, yet they avoid the numerical instability of the pseudoinverse. The diagonal version is the cheapest computationally (O(d) per distance) and is especially attractive for real‑time or resource‑constrained scenarios, while shrinkage offers a modest accuracy boost at a slightly higher cost.

LMNN, while competitive in accuracy on some data sets, suffers from a costly training phase and does not scale well to long series. DTW, despite its superior classification performance, requires O(n²) time and substantial memory for the warping matrix, making it unsuitable for large‑scale or streaming applications.

The authors conclude that if raw classification speed is the primary concern—such as in sensor‑network monitoring, high‑frequency financial tick analysis, or embedded systems—class‑specific Mahalanobis distances with either shrinkage or diagonal approximation should be the default choice. For applications where the highest possible accuracy is mandatory and computational resources are abundant, DTW remains preferable. The paper also highlights future research directions, including kernelised Mahalanobis metrics, integration with deep feature extractors, and more sophisticated regularisation schemes to further mitigate low‑rank covariance issues.