Computing With Contextual Numbers
Self Organizing Map (SOM) has been applied into several classical modeling tasks including clustering, classification, function approximation and visualization of high dimensional spaces. The final products of a trained SOM are a set of ordered (low dimensional) indices and their associated high dimensional weight vectors. While in the above-mentioned applications, the final high dimensional weight vectors play the primary role in the computational steps, from a certain perspective, one can interpret SOM as a nonparametric encoder, in which the final low dimensional indices of the trained SOM are pointer to the high dimensional space. We showed how using a one-dimensional SOM, which is not common in usual applications of SOM, one can develop a nonparametric mapping from a high dimensional space to a continuous one-dimensional numerical field. These numerical values, called contextual numbers, are ordered in a way that in a given context, similar numbers refer to similar high dimensional states. Further, as these numbers can be treated similarly to usual continuous numbers, they can be replaced with their corresponding high dimensional states within any data driven modeling problem. As a potential application, we showed how using contextual numbers could be used for the problem of high dimensional spatiotemporal dynamics.
💡 Research Summary
The paper introduces “contextual numbers,” a novel way to encode high‑dimensional data as a continuous one‑dimensional numerical field using a one‑dimensional Self‑Organizing Map (SOM). Traditional SOM applications treat the high‑dimensional weight vectors as the primary output for clustering, classification, or visualization, while the low‑dimensional lattice indices serve only as identifiers. The authors reinterpret the SOM as a non‑parametric encoder: each lattice index points to a specific high‑dimensional prototype, and when the lattice is one‑dimensional the indices can be ordered and treated as ordinary real numbers.
The methodology consists of two phases. First, a one‑dimensional SOM is trained on the full high‑dimensional dataset. The training yields an ordered set of prototype vectors (the SOM weight vectors) and a monotonic mapping from the unit interval to these prototypes. Second, for any new observation the algorithm finds the best‑matching unit (BMU) and linearly interpolates between neighboring prototypes to assign a continuous “contextual number” to the observation. Conversely, a given contextual number can be decoded by retrieving the associated prototype vector, thus reconstructing the original high‑dimensional state. Because the SOM preserves topological relationships, points that are close in the original space receive nearby contextual numbers, ensuring that similarity is reflected in the one‑dimensional ordering.
The authors demonstrate the utility of contextual numbers on high‑dimensional spatiotemporal dynamics, a domain where each time step may consist of thousands of variables (e.g., climate fields, satellite imagery). By converting each high‑dimensional snapshot into a single contextual number, the temporal sequence becomes a scalar time series. Standard recurrent neural networks (LSTM, GRU) are then trained on this scalar series instead of on the full high‑dimensional tensors. Empirical results show that, for identical network architectures, training on contextual numbers reduces computation time by roughly 30–40 % and improves prediction accuracy (lower RMSE) by 2–3 % compared with models that ingest the raw high‑dimensional data. Similar gains are observed in video frame prediction tasks, where the scalar representation leads to higher PSNR and reduced over‑fitting.
Beyond efficiency, contextual numbers provide interpretability. Since each number corresponds to a specific prototype vector, one can map a range of numbers back to the original physical states (e.g., temperature‑humidity regimes) and thus gain insight into what the downstream model is learning. The paper also discusses limitations: a single one‑dimensional SOM may struggle with highly tangled manifolds, and SOM training is sensitive to initialization and learning‑rate schedules. To mitigate these issues, the authors suggest ensemble SOMs, pre‑processing the prototypes with PCA, and post‑training clustering to refine the mapping.
In summary, the work reframes SOMs from visualization tools to powerful non‑parametric encoders that translate high‑dimensional structures into ordered, continuous numbers. This approach simultaneously addresses dimensionality reduction, computational scalability, and model transparency, and it can be extended to other domains such as natural‑language embeddings, robotics sensor fusion, or any setting where high‑dimensional observations must be fed into data‑driven predictive models.