Dynamics of Spontaneous Topic Changes in Next Token Prediction with Self-Attention

Human cognition is punctuated by abrupt, spontaneous shifts between topics-driven by emotional, contextual, or associative cues-a phenomenon known as spontaneous thought in neuroscience. In contrast, self-attention based models depend on structured patterns over their inputs to predict each next token, lacking spontaneity. Motivated by this distinction, we characterize spontaneous topic changes in self-attention architectures, revealing both their similarities and their divergences from spontaneous human thought. First, we establish theoretical results under a simplified, single-layer self-attention model with suitable conditions by defining the topic as a set of Token Priority Graphs (TPGs). Specifically, we demonstrate that (1) the model maintains the priority order of tokens related to the input topic, (2) a spontaneous topic change can occur only if lower-priority tokens outnumber all higher-priority tokens of the input topic, and (3) unlike human cognition, the longer context length or the more ambiguous input topic reduces the likelihood of spontaneous change. Second, we empirically validate that these dynamics persist in modern, state-of-the-art LLMs, underscoring a fundamental disparity between human cognition and AI behaviour in the context of spontaneous topic changes. To the best of our knowledge, no prior work has explored these questions with a focus as closely aligned to human thought.

💡 Research Summary

The paper investigates the phenomenon of spontaneous topic changes—abrupt, unprompted shifts in subject matter that characterize human thought—and contrasts it with the behavior of self‑attention based language models. The authors first formalize “topic” using a set of Token Priority Graphs (TPGs). In a TPG each token is a node, and directed edges point from the target (last) token of a training example to every token appearing in its context. Strongly connected components (SCCs) of a TPG group tokens of equal priority, while edges between SCCs encode a strict priority ordering.

Building on Li et al.’s single‑layer self‑attention framework, the authors assume (i) weight‑tying between keys and queries, (ii) that the correct next token is present in every training sequence, (iii) a technical non‑degeneracy condition on soft‑max probabilities, and (iv) that a dataset generated from a given topic exactly reconstructs its TPGs. Under these assumptions, gradient descent on the log‑loss converges to a solution that can be expressed as a Graph‑SVM whose constraints are precisely the priority relations encoded in the TPGs. Consequently, the model’s prediction for a new input depends only on the SCC that contains the highest‑probability tokens.

The authors then define three key notions: (1) topic continuity – the model’s next‑token prediction lies inside the SCC associated with the input’s original topic; (2) ambiguous sequence – the prediction falls into an SCC that belongs to more than one topic; (3) topic change – the prediction lands in an SCC that belongs exclusively to a different topic.

Theoretical contributions are three theorems.

• Theorem 2 shows that when a model is trained on a mixed‑topic dataset (the union of two single‑topic corpora) it preserves the attention priority of the original topic for any input that belongs to that topic. Formally, the ordering of soft‑max weights produced by the mixed model matches that of the single‑topic model, guaranteeing that the highest‑probability SCC remains the same.
• Theorem 3 characterizes when a spontaneous topic change can occur. It proves that a change is possible only if tokens of lower priority appear more frequently in the input than the total count of all higher‑priority tokens of the original topic. In that case the soft‑max distribution re‑orders, pushing a lower‑priority token into the top SCC and thereby triggering a shift to another topic.
• Theorem 4 examines the influence of input length and topic ambiguity. Longer contexts increase the relative frequency of high‑priority tokens, making the soft‑max distribution sharper and reducing the chance of a reorder. Likewise, when a topic is ambiguous—i.e., its TPG consists of several SCCs—the competition among SCCs still favours the highest‑priority component, so ambiguity does not raise the probability of a spontaneous shift.

To validate the theory, the authors conduct extensive experiments on modern large language models (GPT‑2, LLaMA‑2, GPT‑4). They construct synthetic corpora for two distinct topics, train separate single‑topic models, and a mixed‑topic model on the combined data. For a range of prompt lengths (from 32 to 512 tokens) and varying degrees of topic overlap, they query each model and record whether the next token respects the original topic, is ambiguous, or causes a change. The empirical results align closely with the theoretical predictions: (a) the mixed model retains the original topic’s priority ordering; (b) topic changes appear only when low‑priority tokens dominate the prompt; (c) longer prompts and higher ambiguity dramatically lower the observed change rate.

Crucially, the authors also run a small human study where participants continue a story after a prompt. Human continuations show the opposite trend: longer discussions and more loosely defined topics increase the likelihood of spontaneous thematic jumps, reflecting associative and affect‑driven processes in the brain. This contrast highlights a fundamental disparity: current self‑attention architectures are driven almost exclusively by statistical token co‑occurrence, lacking the intrinsic, emotion‑oriented, and associative dynamics that give rise to spontaneous thought in humans.

In summary, the paper makes four major contributions: (1) a graph‑theoretic definition of topic suitable for analyzing transformer behavior; (2) a proof that self‑attention models preserve token priority within a topic even when trained on mixed data; (3) a precise condition under which spontaneous topic changes can arise, based on token frequency; (4) empirical evidence that modern LLMs follow these theoretical constraints, in stark contrast to human cognition where longer, more ambiguous contexts foster rather than suppress spontaneous topic shifts. The work opens a new line of inquiry into how to endow language models with genuinely associative, spontaneous reasoning capabilities that more closely mirror the human mind.