Tracking Traders Understanding of the Market Using e-Communication Data

Tracking the volume of keywords in Internet searches, message boards, or Tweets has provided an alternative for following or predicting associations between popular interest or disease incidences. Here, we extend that research by examining the role of e-communications among day traders and their collective understanding of the market. Our study introduces a general method that focuses on bundles of words that behave differently from daily communication routines, and uses original data covering the content of instant messages among all day traders at a trading firm over a 40-month period. Analyses show that two word bundles convey traders’ understanding of same day market events and potential next day market events. We find that when market volatility is high, traders’ communications are dominated by same day events, and when volatility is low, communications are dominated by next day events. We show that the stronger the traders’ attention to either same day or next day events, the higher their collective trading performance. We conclude that e-communication among traders is a product of mass collaboration over diverse viewpoints that embodies unique information about their weak or strong understanding of the market.

💡 Research Summary

This paper investigates how day‑traders collectively understand market conditions by analyzing the content of their instant messages (IMs) over a 40‑month period. Using a unique dataset comprising more than three million IMs exchanged by all traders at a single firm (over 11 million words, 232 000 unique tokens), the authors develop a data‑driven method that does not rely on pre‑selected keywords.

The methodology proceeds in three steps. First, words that appear at least once per day on average and more than 1 000 times in total are retained, eliminating misspellings and rare terms. Second, the retained vocabulary is split into “routine” words—those whose daily frequency scales proportionally with the total daily word count—and “external‑factor” words, whose frequencies are statistically independent of overall volume. The latter set (459 words) is presumed to reflect reactions to external stimuli such as market events. Third, pairwise Pearson correlations of daily frequency changes (Δf) are computed for all external‑factor words. By comparing observed correlations to a null model generated through random shuffling, a z‑score is assigned to each pair. Words are then represented as nodes in a weighted network where edge weights are the z‑scores, and community‑detection algorithms (Extremal Optimization, Kernighan‑Lin refinement, and simulated annealing) are applied to maximize modularity. This yields three distinct word bundles. Bundles 1 and 2 contain 35 % and 45 % of the external‑factor words respectively and consist almost entirely of English terms; bundle 3 is dominated by foreign‑language words and shows no relation to market volatility. Bundle 1 includes words such as “negative,” “cuts,” “banks,” “oil,” indicating a focus on risk and downside. Bundle 2 includes “happy,” “excited,” “trend,” “China,” “Reuters,” reflecting optimism and forward‑looking sentiment.

Market volatility is quantified using the VIX (the “fear index”). For each bundle i the relative frequency γ_i(t) is defined as the proportion of bundle i’s words among all external‑factor words on day t. All time series are transformed to first differences (Δ) to ensure stationarity. Cross‑correlation analysis reveals that bundle 1 is significantly correlated only with same‑day VIX changes (Δt = 0, p < 0.001), whereas bundle 2 correlates only with next‑day VIX changes (Δt = +1, p < 0.001). Granger causality tests confirm that bundle 2 Granger‑causes next‑day volatility (p = 0.031).

To capture how volatility level shapes the dominance of each bundle, the authors normalize VIX to a z‑score and define high‑volatility days (z > 0) and low‑volatility days (z < 0). They compute C(t) = γ₁(t) − γ₂(t); positive C indicates dominance of bundle 1, negative C indicates dominance of bundle 2. Fisher’s exact test shows that on high‑volatility days bundle 1 dominates (p < 10⁻¹²), while on low‑volatility days bundle 2 dominates. This suggests that traders concentrate on interpreting current market turbulence when uncertainty is high, but shift attention to anticipated next‑day events when the market is calm.

Finally, the study links linguistic attention to actual trading performance. Collective performance p(t) is measured as the daily proportion of traders who end the day with a profit. An attention index A(t) = |γ₁(t) − γ₂(t)| quantifies the absolute difference in usage between the two bundles. The first differences Θ_A(t) and Θ_p(t) are correlated at 0.42 (p < 0.001), indicating that greater focus on either same‑day or next‑day information is associated with higher collective profitability.

The paper contributes (1) a generalizable, keyword‑free approach to extracting meaningful word bundles from large‑scale communication data, (2) empirical evidence that these bundles encode distinct temporal aspects of market volatility, and (3) a demonstration that the degree of collective linguistic focus predicts real‑world trading success, thereby extending the literature on collective wisdom. Limitations include reliance on data from a single firm, lack of external validation, and limited interpretation of the foreign‑language bundle. Future work could incorporate multi‑firm datasets, apply deeper semantic modeling, and explore real‑time predictive applications.