IKNet: Interpretable Stock Price Prediction via Keyword-Guided Integration of News and Technical Indicators

The increasing influence of unstructured external information, such as news articles, on stock prices has attracted growing attention in financial markets. Despite recent advances, most existing news-based forecasting models represent all articles using sentiment scores or average embeddings that capture the general tone but fail to provide quantitative, context-aware explanations of the impacts of public sentiment on predictions. To address this limitation, we propose an interpretable keyword-guided network (IKNet), which is an explainable forecasting framework that models the semantic association between individual news keywords and stock price movements. The IKNet identifies salient keywords via FinBERT-based contextual analysis, processes each embedding through a separate nonlinear projection layer, and integrates their representations with the time-series data of technical indicators to forecast next-day closing prices. By applying Shapley Additive Explanations the model generates quantifiable and interpretable attributions for the contribution of each keyword to predictions. Empirical evaluations of S&P 500 data from 2015 to 2024 demonstrate that IKNet outperforms baselines, including recurrent neural networks and transformer models, reducing RMSE by up to 32.9% and improving cumulative returns by 18.5%. Moreover, IKNet enhances transparency by offering contextualized explanations of volatility events driven by public sentiment.

💡 Research Summary

The paper tackles the growing influence of unstructured external information—particularly news articles—on equity prices and the shortcomings of existing news‑driven forecasting models, which typically compress all articles into a single sentiment score or an averaged embedding. Such approaches obscure the nuanced, context‑aware impact of individual words on price movements and provide little quantitative explanation for investors. To overcome these limitations, the authors propose IKNet (Interpretable Keyword‑guided Network), a novel multimodal architecture that explicitly models the semantic association between individual news keywords and stock price dynamics while preserving interpretability.

Model Architecture

1. Keyword Extraction – Using FinBERT, the system tokenizes each news article and generates contextual embeddings for every token. A two‑step selection process identifies salient keywords: (a) an initial candidate pool is built by weighting TF‑IDF scores with sentiment‑lexicon values; (b) FinBERT‑derived attention scores are then computed to rank candidates by their correlation with subsequent price changes, and the top‑N tokens are retained.
2. Separate Non‑linear Projections – Each selected keyword passes through its own multilayer perceptron (MLP) projection layer, which performs dimensionality reduction and introduces non‑linearity. This design isolates the contribution of each keyword, preventing interference among them and allowing the model to learn keyword‑specific signals.
3. Technical Indicator Encoder – Conventional technical indicators (moving averages, RSI, MACD, etc.) are normalized and fed into a time‑series encoder (either an LSTM or a 1‑D convolutional network) to capture temporal dependencies.
4. Fusion via Multi‑Head Attention – The concatenated keyword projections and the encoded technical‑indicator representation are processed by a multi‑head attention module. This component learns dynamic weights that dictate how much each textual or numeric signal should influence the final prediction at each time step.
5. Prediction Head & Loss – A linear regression head outputs the next‑day closing price. The loss combines mean‑squared error with a volatility‑regularization term, encouraging accurate forecasts while penalizing excessive price‑swing predictions.

Interpretability with SHAP
To provide transparent explanations, the authors apply Shapley Additive Explanations (SHAP) to the trained model. SHAP values are computed for each keyword projection, yielding a quantitative attribution of how much a particular word (e.g., “Federal Reserve rate hike”) contributed to a specific price forecast. These attributions are visualized alongside the contributions of technical indicators, enabling investors to trace a prediction back to concrete textual events.

Experimental Setup

• Data: S&P 500 constituents from 2015‑2024, comprising ~2.3 million news articles, daily closing prices, volumes, and a suite of technical indicators.
• Splits: 70 % training, 15 % validation, 15 % testing.
• Baselines: (i) LSTM‑only (price + indicators), (ii) Transformer‑only (price + indicators), (iii) Sentiment‑average + LSTM, (iv) Multimodal BERT‑LSTM hybrid.
• Metrics: Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and cumulative return from a simple long‑only strategy based on predicted next‑day price direction.

Results
IKNet consistently outperforms all baselines. Across the test set, RMSE improves by an average of 27 % (maximum 32.9 % reduction) and MAE shows a comparable drop. In a back‑tested trading scenario, the model yields an 18.5 % higher cumulative return than the best baseline, with the most pronounced gains during high‑volatility periods (2020‑2022). SHAP analysis reveals that keywords such as “COVID‑19 surge,” “inflation concerns,” and “Fed rate hike” receive the highest positive attributions during sharp market moves, confirming the model’s ability to pinpoint sentiment‑driven volatility.

Limitations & Future Work
The current keyword selection relies on a fixed top‑N cutoff, which may omit emerging terms in real‑time streams. Moreover, processing each keyword through a separate MLP incurs additional computational overhead, challenging deployment in ultra‑low‑latency trading environments. Future research directions include (a) dynamic keyword pool updating using online learning, (b) replacing FinBERT with lighter models like DistilBERT to reduce latency, (c) extending the framework to other asset classes (cryptocurrencies, bonds), and (d) integrating the interpretability layer into portfolio‑optimization pipelines for end‑to‑end decision support.

Conclusion
IKNet introduces a principled way to fuse keyword‑level news semantics with traditional technical indicators while delivering transparent, quantifiable explanations for each prediction. Empirical evidence on a decade‑long S&P 500 dataset demonstrates that this interpretability does not come at the cost of performance; rather, it yields superior forecasting accuracy and tangible trading benefits. The work sets a new benchmark for explainable multimodal financial forecasting and opens avenues for more trustworthy AI‑driven investment tools.