From Numbers to Prompts: A Cognitive Symbolic Transition Mechanism for Lightweight Time-Series Forecasting

Large language models have achieved remarkable success in time series prediction tasks, but their substantial computational and memory requirements limit deployment on lightweight platforms. In this paper, we propose the Symbolic Transition Mechanism (STM) a novel framework that bridges numeric time series data and language models through symbolic abstraction and prompt engineering. STM transforms continuous time series values into symbol tokens with quantization techniques based on human cognitive structures, and captures temporal dynamics through structured transformations of symbols, enabling fast engineering based predictions in which language models focus on critical parts of time series data. STM is a general purpose mechanisms that ensure the integrity of backbone language models, but they significantly improve their efficiency by inferring the dynamic and structured patterns inherent in time series data. We evaluated STM on various time series datasets, paired with four small language models (SLM) with limited computational environments. For all models, STM achieves error reductions of up to 69% in MAE and 90% in MSE compared to the default backbone SLM without STM. These results demonstrate the potential of STM as an efficient, adaptable layer for symbol-driven time series prediction using foundation models. The accuracy improvements were made at negligible resource costs, with maximum GPU memory of the base model increasing by approximately 0.06% and latency overhead increasing by only 0.64%.

💡 Research Summary

The paper addresses the growing interest in using large language models (LLMs) for time‑series forecasting while acknowledging the prohibitive computational and memory demands that make deployment on edge or lightweight platforms impractical. To bridge this gap, the authors introduce the Symbolic Transition Mechanism (STM), a lightweight, plug‑in framework that converts continuous numeric time‑series data into a sequence of symbolic tokens, enriches those tokens with transition‑strength information, and highlights periodic patterns before feeding them to a small language model (SLM).

STM consists of three tightly coupled components. First, Symbol‑Based Encoding quantizes each data point into one of five discrete symbols (A‑E). The choice of five levels is motivated by Miller’s law (the 7 ± 2 rule) from cognitive psychology, arguing that both humans and language models process information most efficiently when it is grouped into a small set of meaningful categories. This quantization is not a compression technique; rather, it creates a human‑readable, text‑compatible representation that aligns with the token‑based training of language models.

Second, Transition‑Based Weighting computes the absolute difference between consecutive symbols and assigns a distance‑based weight. Large jumps—indicative of abrupt level shifts, spikes, or regime changes—receive higher weights, ensuring that the prompt explicitly emphasizes the most informative parts of the series.

Third, Periodicity Detection scans the symbolic sequence for recurring subsequences, assigning additional weight to segments that belong to a detected cycle. By doing so, STM injects an inductive bias toward seasonality and long‑term repetition, which are central to many real‑world time‑series (e.g., electricity demand, weather).

Crucially, STM does not alter the internal architecture of the underlying language model. Instead, it operates as an external preprocessing layer that produces a richer textual prompt. Consequently, any existing SLM can be used without retraining, fine‑tuning, or architectural modification. The authors demonstrate this plug‑in nature by integrating STM with four representative SLMs: Phi‑3.5‑mini‑instruct, Llama‑3.2‑1B, DeepSeek‑Coder‑1.3B, and Gemma‑2B‑it.

Experimental evaluation covers two publicly available time‑series datasets (a power‑consumption series and a meteorological series). For each model, the authors vary the quantization granularity K ∈ {3, 5, 7, 9, 10} and report mean absolute error (MAE) and mean squared error (MSE) after inverse‑normalization. Across all model‑K combinations, K = 5 consistently yields the best or near‑best performance, confirming the cognitive rationale behind the five‑level design. When STM is applied, MAE reductions reach up to 69 % and MSE reductions up to 90 % relative to the baseline SLM without STM.

Resource analysis shows that STM adds negligible overhead: GPU memory consumption increases by only ~0.06 % and inference latency by less than 0.64 % compared with the raw SLM. This demonstrates that STM delivers substantial accuracy gains while preserving the lightweight footprint required for edge deployment.

The paper’s contributions can be summarized as follows: (1) a cognitively inspired symbolic quantization scheme that balances granularity and interpretability; (2) a transition‑weighting mechanism that foregrounds abrupt changes; (3) a periodicity‑aware augmentation that captures seasonal structure; (4) a model‑agnostic plug‑in that improves SLM forecasting without any parameter changes; and (5) extensive empirical validation confirming both effectiveness and efficiency.

Future work suggested by the authors includes extending STM to multivariate and irregular time‑series, developing adaptive quantization that dynamically selects the optimal number of symbols, and exploring applications beyond forecasting such as anomaly detection or reinforcement‑learning‑based control. Overall, the study presents a compelling pathway for leveraging the reasoning capabilities of language models in resource‑constrained time‑series applications through a simple yet powerful symbolic abstraction layer.