The Language You Ask In: Language-Conditioned Ideological Divergence in LLM Analysis of Contested Political Documents

Large language models (LLMs) are increasingly deployed as analytical tools across multilingual contexts, yet their outputs may carry systematic biases conditioned by the language of the prompt. This study presents an experimental comparison of LLM-generated political analyses of a Ukrainian civil society document, using semantically equivalent prompts in Russian and Ukrainian. Despite identical source material and parallel query structures, the resulting analyses varied substantially in rhetorical positioning, ideological orientation, and interpretive conclusions. The Russian-language output echoed narratives common in Russian state discourse, characterizing civil society actors as illegitimate elites undermining democratic mandates. The Ukrainian-language output adopted vocabulary characteristic of Western liberal-democratic political science, treating the same actors as legitimate stakeholders within democratic contestation. These findings demonstrate that prompt language alone can produce systematically different ideological orientations from identical models analyzing identical content, with significant implications for AI deployment in polarized information environments, cross-lingual research applications, and the governance of AI systems in multilingual societies.

💡 Research Summary

The paper investigates whether the language in which a user prompts a large language model (LLM) can condition the ideological orientation of the model’s output, even when the underlying content is identical. The authors focus on a politically contested Ukrainian civil‑society joint statement from May 2019, providing the English version of the document to the same model (ChatGPT 5.2) and issuing parallel analytical prompts in Russian and Ukrainian. All model parameters (temperature, max‑tokens, etc.) are held constant, ensuring that any divergence in the generated analyses can be attributed solely to the language of the prompt.

The experimental results reveal a striking asymmetry. The Russian‑language analysis employs a lexicon that mirrors Russian state‑aligned discourse: terms such as “quasi‑elite,” “ideological supervision,” and “substitutes for the people’s mandate” appear repeatedly, casting the civil‑society signatories as illegitimate actors undermining democratic authority. By contrast, the Ukrainian‑language analysis adopts terminology typical of Western liberal‑democratic political science, describing the same actors as a “professionalized pro‑Western civic elite” engaged in “normative restraint of power.” Both outputs correctly identify factual details—document structure, the six policy “red lines,” the dual domestic‑international audience, and the list of signatories—yet they diverge markedly in evaluative tone, rhetorical framing, and inferred motivations.

To unpack these differences, the authors conduct a multi‑dimensional discourse analysis, focusing on (1) lexical choice, (2) rhetorical positioning, (3) interpretive conclusions, and (4) alignment with established discourse traditions in the respective linguistic communities. The Russian output aligns with narratives that portray NGOs as foreign‑funded threats, while the Ukrainian output aligns with narratives that view NGOs as legitimate watchdogs within a democratic system.

The paper attributes the observed language‑conditioned bias to two primary mechanisms. First, training‑data imbalance: Russian‑language corpora contain a higher proportion of state‑controlled media and pro‑government narratives, leading the model to associate “civil society” with negative connotations in Russian. Second, reinforcement learning from human feedback (RLHF): annotators for different languages may hold divergent political views, imprinting language‑specific alignment biases during the fine‑tuning stage.

The authors argue that such language‑dependent ideological framing has profound implications for AI deployment in polarized, multilingual environments—particularly in the Russia‑Ukraine information war, where language itself is a proxy for geopolitical allegiance. An LLM that silently reproduces language‑specific propaganda risks amplifying existing information asymmetries and deepening conflict.

Consequently, the paper recommends concrete mitigation strategies: (a) balancing multilingual training data to reduce language‑specific narrative dominance, (b) ensuring political neutrality and diversity among RLHF annotators across languages, and (c) instituting systematic multilingual evaluation frameworks that monitor ideological drift in model outputs. By documenting the phenomenon of “language‑conditioned ideological divergence,” the study contributes a novel empirical insight to the literature on political bias in LLMs, multilingual NLP, and AI‑mediated information warfare, and it underscores the need for robust governance mechanisms when deploying AI in multilingual societies.