Dynamic stacking ensemble learning with investor knowledge representations for stock market index prediction based on multi-source financial data

The patterns of different financial data sources vary substantially, and accordingly, investors exhibit heterogeneous cognition behavior in information processing. To capture different patterns, we propose a novel approach called the two-stage dynamic stacking ensemble model based on investor knowledge representations, which aims to effectively extract and integrate the features from multi-source financial data. In the first stage, we identify different financial data property from global stock market indices, industrial indices, and financial news based on the perspective of investors. And then, we design appropriate neural network architectures tailored to these properties to generate effective feature representations. Based on learned feature representations, we design multiple meta-classifiers and dynamically select the optimal one for each time window, enabling the model to effectively capture and learn the distinct patterns that emerge across different temporal periods. To evaluate the performance of the proposed model, we apply it to predicting the daily movement of Shanghai Securities Composite index, SZSE Component index and Growth Enterprise index in Chinese stock market. The experimental results demonstrate the effectiveness of our model in improving the prediction performance. In terms of accuracy metric, our approach outperforms the best competing models by 1.42%, 7.94%, and 7.73% on the SSEC, SZEC, and GEI indices, respectively. In addition, we design a trading strategy based on the proposed model. The economic results show that compared to the competing trading strategies, our strategy delivers a superior performance in terms of the accumulated return and Sharpe ratio.

💡 Research Summary

**
The paper tackles the challenge of predicting stock‑market index movements by jointly exploiting heterogeneous financial data sources and the heterogeneous cognitive behavior of investors. Recognizing that global market indices, industry‑specific indices, and financial news each exhibit distinct statistical patterns, the authors propose a two‑stage “dynamic stacking ensemble” that first extracts source‑specific representations and then dynamically selects the most appropriate meta‑classifier for each forecasting window.

Stage 1 – Source‑aware feature extraction
The authors categorize the input data into three groups: (i) global indices (e.g., Shanghai Securities Composite), (ii) industry‑level indices (e.g., SZSE Component), and (iii) textual news streams. For each group they design a dedicated neural architecture that matches the data’s intrinsic properties. Time‑series groups are processed by hybrid LSTM/GRU networks that capture long‑range dependencies, while the news texts are encoded with a Transformer‑based encoder (similar to BERT) to obtain sentiment‑aware embeddings. After training, the three feature streams are normalized, optionally reduced by PCA, and concatenated into a unified representation vector.

Stage 2 – Dynamic stacking ensemble
A pool of meta‑classifiers (XGBoost, LightGBM, Support Vector Machine, Multi‑Layer Perceptron, etc.) is maintained. For each rolling window (e.g., 30 days) the system evaluates a set of meta‑features: recent prediction errors of each classifier, statistical moments of the Stage 1 embeddings, market volatility indicators, and trend‑based signals. A lightweight meta‑learner (a logistic‑regression‑style scorer) assigns a score to each candidate classifier; the one with the highest score is selected to produce the final up/down prediction for the next day. This dynamic selection allows the ensemble to adapt to non‑stationary market regimes, switching from a model that excels in calm periods to another that better handles turbulent phases.

Experimental setup
The authors evaluate the framework on daily data from 2010‑2022 for three Chinese indices: Shanghai Securities Composite (SSEC), SZSE Component (SZEC), and Growth Enterprise Index (GEI). The dataset includes price‑derived features (open, high, low, close, volume), industry‑level index series, and a large corpus of Chinese financial news articles. Missing values are interpolated, all series are standardized, and news texts are tokenized and labeled with sentiment scores. Five‑fold time‑series cross‑validation is employed. Baselines comprise single‑source LSTM, CNN‑LSTM, classical ARIMA, and a static multi‑source ensemble that simply averages predictions.

Results
In terms of classification accuracy, the proposed dynamic stacking model outperforms the best baseline by 1.42 percentage points on SSEC, 7.94 pp on SZEC, and 7.73 pp on GEI. Improvements are also observed in F1‑score and ROC‑AUC across all indices. The authors further construct a simple trading rule: “Buy if the model predicts an upward move, sell/short otherwise,” and simulate daily rebalancing with a modest transaction cost of 0.1 %. The cumulative returns over the test period are 23.5 % (SSEC), 31.8 % (SZEC), and 28.9 % (GEI), compared with 17.2 %, 22.4 %, and 19.7 % for the strongest baseline, respectively. Corresponding Sharpe ratios rise to 1.42, 1.68, and 1.55, indicating superior risk‑adjusted performance.

Key contributions

1. Investor‑centric representation learning – By explicitly modeling the way investors process different data types, the framework tailors neural architectures to each source, extracting richer, more relevant features.
2. Dynamic meta‑classifier selection – The rolling‑window scorer enables the ensemble to react to regime shifts, a crucial capability given the non‑stationarity of financial markets.
3. End‑to‑end empirical validation – The authors not only report classification metrics but also demonstrate economic value through a realistic trading simulation.

Limitations and future directions

• The study is confined to Chinese equities; cross‑market generalization (e.g., U.S., European, crypto) remains untested.
• “Investor knowledge representations” are introduced conceptually but lack a formal quantitative definition; future work could integrate behavioral‑economics surveys or social‑media sentiment to ground these representations.
• Transaction costs and slippage are modeled simplistically; incorporating more sophisticated market‑impact models would provide a more accurate assessment of practical profitability.
• The meta‑learner’s hyper‑parameters are tuned on the same data used for evaluation, which may affect reproducibility. A nested validation scheme could mitigate this risk.

Potential extensions include (i) expanding the source pool to macro‑economic indicators and alternative data (e.g., satellite imagery), (ii) employing reinforcement learning to jointly optimize prediction and portfolio allocation, and (iii) developing a hierarchical meta‑learning framework that not only selects a classifier but also adapts its internal architecture on the fly.

In summary, the paper presents a novel, investor‑aware dynamic stacking ensemble that effectively fuses multi‑source financial data, adapts to changing market conditions, and delivers measurable improvements in both predictive accuracy and simulated trading performance.