When humans and large language models (LLMs) process the same text, activations in the LLMs correlate with brain activity measured, e.g., with functional magnetic resonance imaging (fMRI). Moreover, it has been shown that, as the training of an LLM progresses, the performance in predicting brain activity from its internal activations improves more in the left hemisphere than in the right one. The aim of the present work is to understand which kind of competence acquired by the LLMs underlies the emergence of this left-right asymmetry. Using the OLMo-2 7B language model at various training checkpoints and fMRI data from English participants, we compare the evolution of the left-right asymmetry in brain scores alongside performance on several benchmarks. We observe that the asymmetry co-emerges with the formal linguistic abilities of the LLM. These abilities are demonstrated in two ways: by the model's capacity to assign a higher probability to an acceptable sentence than to a grammatically unacceptable one within a minimal contrasting pair, or its ability to produce well-formed text. On the opposite, the left-right asymmetry does not correlate with the performance on arithmetic or Dyck language tasks; nor with text-based tasks involving world knowledge and reasoning. We generalize these results to another family of LLMs (Pythia) and another language, namely French. Our observations indicate that the left-right asymmetry in brain predictivity matches the progress in formal linguistic competence (knowledge of linguistic patterns).
The success of large language models (LLMs) in natural language processing tasks has generated a lot of interest in understanding their internal representations and their alignment with human brain activity. Brain activations, measured with functional magnetic resonance, magnetoencephalography, or electrocorticography, in humans listening to or reading a text can be predicted from the internal activity of LLMs fed with the same text (Jain and Huth, 2018;Toneva and Wehbe, 2019;Schrimpf et al., 2021;Caucheteux and King, 2022;Goldstein et al., 2022;Pasquiou et al., 2023;Antonello et al., 2024). In the encoding model approach (see Dupré la Tour et al., 2025, for a review), brain activations are regressed on the hidden neural activations from an LLM and the resulting model is used to compute cross-validated correlations at each voxel (brain scores).
Early studies (Huth et al., 2016;Jain and Huth, 2018;Caucheteux et al., 2021;Pasquiou et al., 2023) reported brain score maps that were very symmetrical, with similar brain scores in the right and in the left hemisphere, an odd finding given all the evidence for left hemispheric dominance for language. For instance, in their seminal paper, Huth et al. (2016) noted that “One striking aspect of our atlas is that the distribution of semantically selective areas is relatively symmetrical across the two cerebral hemispheres. This finding is inconsistent with human lesion studies that support the idea that semantic representation is lateralized to the left hemisphere.”
Recently, however, we showed that these symmetrical results, observed with word embeddings or small, first-generation LLMs, disappear in larger and higher-performing models (Bonnasse-Gahot and Pallier, 2024). More precisely, brain score maps exhibited an increasing left-right hemispheric asymmetry when LLMs increased in number of parameters and in performance on NLP tasks. Furthermore, this left-right asymmetry also emerged for a given LLM alongside its training. The relationship between amount of training and left-right asymmetry showed a phase transition profile that is reminiscent of those that have been observed in LLMs’ performance on several benchmarks (Chen et al., 2023).
The present work aims at understanding what competence, acquired during training, drives the emergence of the left-right asymmetry in brain score. We conduct a series of experiments designed to track the evolution of linguistic and non-linguistic capabilities of LLMs as a function of training progression, and we study their relationship to left-right asymmetry in brain score. In our initial experiment, we systematically investigate how the performance of an LLM on a set of carefully constructed benchmarks evolves with training. This set includes two linguistic benchmarks (BLiMP, Warstadt et al., 2020 andZorro, Huebner et al., 2021) and two non-linguistic benchmarks (specifically, Arithmetic and Dyck language tasks), all designed as minimal pairs tasks to isolate specific compe-tencies. Our analyses reveal a striking correlation: as training progresses, the emergence of the left-right dominance in brain score maps closely mirrors the improvement in performance on the linguistic benchmarks, but not on the non-linguistic benchmarks.
BLiMP and Zorro essentially assess formal linguistic competence (knowledge of linguistic rules and patterns). In a follow-up experiment, focusing on text-based tasks, we contrast formal linguistic competence and functional linguistic competence (understanding and using language in the world), a distinction proposed by Mahowald et al. (2024). To assess the model’s formal competence beyond BLiMP and Zorro, we evaluate the linguistic acceptability of texts generated by the model at different checkpoints during training. To assess functional competence, we test the LLM on conceptual and reasoning benchmarks, namely, ARC (Clark et al., 2018), andHellaswag (Zellers et al., 2019). The results show that the trajectory of linguistic acceptability correlates with the left-right transition in brain scores, unlike the performance on functional benchmarks.
The results reported above are based on OLMo-2 7B model (OLMo et al., 2024), a recent model for which training checkpoints are available. We show that these results generalize to other models, namely the 2.8b and the 6.9b models from the Pythia family (Biderman et al., 2023). Finally, we generalize to another language, French, and replicate the finding that the left-right asymmetry aligns better with a formal test (grammar) than with a functional one (Hellaswag).
Collectively, these results support the hypothesis that the emergence of the left-right asymmetry in LLMs’ brain predictivity is a direct reflection of their formal linguistic abilities.
The experiments reported in this paper rely on functional magnetic resonance data provided by the multilingual project Le Petit Prince, in which English, French and Mandarin Chinese speakers were scanned while listening
This content is AI-processed based on open access ArXiv data.