TradeFM: A Generative Foundation Model for Trade-flow and Market Microstructure

Foundation models have transformed domains from language to genomics by learning general-purpose representations from large-scale, heterogeneous data. We introduce TradeFM, a 524M-parameter generative Transformer that brings this paradigm to market microstructure, learning directly from billions of trade events across >9K equities. To enable cross-asset generalization, we develop scale-invariant features and a universal tokenization scheme that map the heterogeneous, multi-modal event stream of order flow into a unified discrete sequence – eliminating asset-specific calibration. Integrated with a deterministic market simulator, TradeFM-generated rollouts reproduce key stylized facts of financial returns, including heavy tails, volatility clustering, and absence of return autocorrelation. Quantitatively, TradeFM achieves 2-3x lower distributional error than Compound Hawkes baselines and generalizes zero-shot to geographically out-of-distribution APAC markets with moderate perplexity degradation. Together, these results suggest that scale-invariant trade representations capture transferable structure in market microstructure, opening a path toward synthetic data generation, stress testing, and learning-based trading agents.

💡 Research Summary

TradeFM introduces a 524‑million‑parameter generative Transformer that learns market microstructure directly from billions of trade‑level events across more than 9,000 US equities. The authors address three core challenges that have limited previous AI approaches to finance: (1) heterogeneity of assets, (2) the need for full limit‑order‑book (LOB) snapshots, and (3) poor out‑of‑distribution (OOD) generalization. To overcome (1), they construct scale‑invariant features – log‑transformed volumes, price depth normalized by a robust exponentially‑weighted VWAP estimate of the mid‑price, and relative price levels expressed in basis points – thereby mapping assets of vastly different price and liquidity regimes onto a common representation. For (2), they devise a universal tokenization pipeline that discretizes continuous features using a hybrid of quantile‑based bins (for price) and logarithmic/equal‑width bins (for volume and inter‑arrival time), and combines these with categorical tokens for order side, action type, and a market/participant flag. This yields a vocabulary of roughly 30 k tokens, allowing the event stream to be treated as a standard autoregressive sequence.

The model is trained in a decoder‑only fashion to predict the conditional distribution P(eₜ | e₍<ₜ₎) using cross‑entropy loss on a corpus of 10 billion tokens (training) and 8.7 billion tokens (test). A deterministic market simulator implements the price‑time priority matching rule; generated events are fed into this engine to produce roll‑outs of price evolution. The authors evaluate realism by checking four canonical stylized facts: heavy‑tailed returns, volatility clustering, near‑zero autocorrelation of raw returns, and the distribution of inter‑arrival times. TradeFM’s synthetic series reproduce these facts with quantitative fidelity: the return kurtosis matches empirical values (≈ 7), the autocorrelation of absolute returns decays slowly, and the return autocorrelation is statistically indistinguishable from zero. Compared to a state‑of‑the‑art Compound Hawkes baseline, TradeFM reduces Wasserstein distance on the joint distribution of price depth, volume, and inter‑arrival time by a factor of 2–3.

A notable contribution is zero‑shot geographic generalization. By holding out a month of Japanese and Chinese equity data (APAC markets) the authors show that perplexity rises modestly (≈ 15 % increase) yet the generated roll‑outs still exhibit the stylized facts, indicating that the scale‑invariant representation captures universal market dynamics beyond the US equity universe.

The paper’s contributions are fourfold: (i) a large‑scale generative foundation model for market microstructure, (ii) a methodology for learning from partial observations (Level 3 trade messages) rather than privileged LOB snapshots, (iii) a scale‑invariant feature and tokenization scheme that eliminates asset‑specific calibration, and (iv) integration with a deterministic simulator to enable realistic synthetic data generation and a testbed for learning‑based agents.

Limitations are acknowledged. The deterministic simulator does not model slippage, latency, or multi‑matching effects, which are important in real high‑frequency trading. The model also relies on a massive compute budget (GPU clusters for pre‑training) and may be costly to deploy in low‑latency production environments. Future work is outlined: coupling TradeFM with reinforcement‑learning agents for end‑to‑end policy learning, extending the tokenization to multimodal inputs (news, social media), and stress‑testing under extreme market shocks.

Overall, TradeFM demonstrates that a foundation‑model approach—large‑scale, heterogeneous data, and scale‑invariant representations—can capture transferable structure in financial markets, opening pathways for synthetic data generation, robust stress testing, and the development of more sophisticated, learning‑based trading systems.