Supervised learning of short and high-dimensional temporal sequences for life science measurements

The analysis of physiological processes over time are often given by spectrometric or gene expression profiles over time with only few time points but a large number of measured variables. The analysis of such temporal sequences is challenging and only few methods have been proposed. The information can be encoded time independent, by means of classical expression differences for a single time point or in expression profiles over time. Available methods are limited to unsupervised and semi-supervised settings. The predictive variables can be identified only by means of wrapper or post-processing techniques. This is complicated due to the small number of samples for such studies. Here, we present a supervised learning approach, termed Supervised Topographic Mapping Through Time (SGTM-TT). It learns a supervised mapping of the temporal sequences onto a low dimensional grid. We utilize a hidden markov model (HMM) to account for the time domain and relevance learning to identify the relevant feature dimensions most predictive over time. The learned mapping can be used to visualize the temporal sequences and to predict the class of a new sequence. The relevance learning permits the identification of discriminating masses or gen expressions and prunes dimensions which are unnecessary for the classification task or encode mainly noise. In this way we obtain a very efficient learning system for temporal sequences. The results indicate that using simultaneous supervised learning and metric adaptation significantly improves the prediction accuracy for synthetically and real life data in comparison to the standard techniques. The discriminating features, identified by relevance learning, compare favorably with the results of alternative methods. Our method permits the visualization of the data on a low dimensional grid, highlighting the observed temporal structure.

💡 Research Summary

The paper addresses the challenging problem of analyzing short, high‑dimensional temporal sequences that are common in life‑science measurements such as spectrometric profiles or gene‑expression time courses. Traditional time‑series methods (e.g., ARMA) are unsuitable because of the limited number of time points, while existing approaches for such data are largely unsupervised or semi‑supervised and rely on wrapper or post‑processing steps for feature selection.

To overcome these limitations the authors propose Supervised Topographic Mapping Through Time (SGTM‑TT). The method builds on Generative Topographic Mapping (GTM), a prototype‑based probabilistic model that embeds data into a low‑dimensional lattice of Gaussian prototypes. GTM‑Through‑Time (GTM‑TT) extends GTM by coupling the prototype lattice with a Hidden Markov Model (HMM), thereby modeling temporal dependencies between successive observations.

SGTM‑TT introduces supervision by training a separate GTM‑TT model for each class (e.g., M₀ and M₁). Both class models share the same lattice topology and a common variance parameter β, ensuring comparability while allowing class‑specific prototype mappings (parameter W). Training proceeds via an Expectation‑Maximization (EM) algorithm: the E‑step computes posterior responsibilities rₖₙ for each hidden state at each time point, and the M‑step updates W and β.

A major contribution is the integration of relevance learning (RL) into this framework. Two distance measures are defined: (i) a weighted Euclidean distance d_λ that assigns a relevance weight λ_d to each data dimension, and (ii) a temporal distance d_t that assigns relevance to individual time points via a diagonal matrix Ω. Both sets of weights are constrained (∑λ²=1, trace(ΩᵀΩ)=1) to avoid trivial solutions. The RL objective, adapted from Relevance‑GTM, is optimized with stochastic gradient descent, allowing the model to automatically prune noisy features and irrelevant time points.

For classification, the log‑likelihood of a new sequence under each class model is computed using the forward algorithm; the sequence is assigned to the class with the highest likelihood. Additionally, the authors construct a kernel K based on the product of class‑wise likelihoods and feed it to a standard Support Vector Machine, thereby combining the probabilistic strengths of SGTM‑TT with the discriminative power of SVMs.

Experimental evaluation includes synthetic data, a multiple‑sclerosis (MS) gene‑expression dataset, and a mass‑spectrometry metabolomics dataset. Across all experiments SGTM‑TT outperforms the original unsupervised GTM‑TT, a single HMM baseline, and an SVM‑Kalman approach, achieving 5–10 percentage‑point improvements in classification accuracy. The relevance‑learning component selects features that correspond to known biological markers, demonstrating interpretability. Moreover, the low‑dimensional lattice provides intuitive visualizations of temporal trajectories, revealing class‑specific patterns that are difficult to discern with black‑box models.

In summary, SGTM‑TT offers a unified solution that (1) captures temporal dynamics via an HMM‑augmented prototype model, (2) embeds data into an interpretable low‑dimensional space, (3) learns feature‑ and time‑point relevance jointly, and (4) delivers superior predictive performance on short, high‑dimensional time series. The approach is broadly applicable to biomedical, environmental, and industrial domains where such data are prevalent, and it opens avenues for extensions to multi‑class problems, non‑linear transition structures, and online learning scenarios.