Beyond Partisan Leaning: A Comparative Analysis of Political Bias in Large Language Models

As large language models (LLMs) become increasingly embedded in civic, educational, and political information environments, concerns about their potential political bias have grown. Prior research often evaluates such bias through simulated personas or predefined ideological typologies, which may introduce artificial framing effects or overlook how models behave in general use scenarios. This study adopts a persona-free, topic-specific approach to evaluate political behavior in LLMs, reflecting how users typically interact with these systems-without ideological role-play or conditioning. We introduce a two-dimensional framework: one axis captures partisan orientation on highly polarized topics (e.g., abortion, immigration), and the other assesses sociopolitical engagement on less polarized issues (e.g., climate change, foreign policy). Using survey-style prompts drawn from the ANES and Pew Research Center, we analyze responses from 43 LLMs developed in the U.S., Europe, China, and the Middle East. We propose an entropy-weighted bias score to quantify both the direction and consistency of partisan alignment, and identify four behavioral clusters through engagement profiles. Findings show most models lean center-left or left ideologically and vary in their nonpartisan engagement patterns. Model scale and openness are not strong predictors of behavior, suggesting that alignment strategy and institutional context play a more decisive role in shaping political expression.

💡 Research Summary

This paper investigates political bias in large language models (LLMs) by adopting a persona‑free, topic‑specific evaluation that mirrors how ordinary users interact with these systems. Rather than relying on simulated ideological personas or a single left‑right axis, the authors propose a two‑dimensional framework: (1) partisan orientation on highly polarized issues (e.g., abortion, immigration, presidential elections) and (2) sociopolitical engagement on less polarized topics (e.g., climate change, foreign policy, misinformation, “most important problem”).

The authors curated 45 survey‑style prompts drawn from the American National Election Studies (ANES) and Pew Research Center, covering nine political topics (four highly polarized, five less polarized). They then queried 43 LLMs released between April 2023 and September 2024, spanning the United States, Europe, China, and the Middle East. The sample includes both open‑source and proprietary models, with parameter counts ranging from 2 billion to 176 billion.

To quantify bias, the study introduces an entropy‑weighted bias score for polarized issues. This metric captures both the direction (Democratic vs. Republican) and the consistency of a model’s responses, weighting more certain, repeatable answers higher. For the less polarized set, a sociopolitical engagement score measures how strongly a model emphasizes issue importance, factual accuracy, and prioritization, independent of partisan cues.

Plotting each model in the two‑dimensional space and applying k‑means clustering yields four behavioral clusters: (1) left‑center / high engagement, (2) left‑center / low engagement, (3) centrist / high engagement, and (4) centrist / low engagement. The majority of models fall into clusters 1 or 2, indicating a general left‑center or left tilt, especially among U.S. and European systems. Model scale and open‑source status show little predictive power for either bias or engagement; large models are not systematically more neutral or more partisan. Instead, the alignment strategy (e.g., reinforcement learning from human feedback, policy‑based filters) and the geopolitical or regulatory context of the developer appear to drive the observed patterns. Chinese models tend toward state‑aligned positions on foreign policy, while Middle‑Eastern models display culturally sensitive avoidance on certain social issues.

The paper discusses several limitations: the prompts are U.S.-centric, the entropy‑weighted score may not map directly onto real‑world political behavior, and distinguishing genuine model stance from strategic evasion remains challenging. Future work is suggested to expand multilingual, cross‑cultural surveys, integrate real user interaction logs for external validation, and explore alignment techniques that explicitly mitigate undesirable political bias.

In conclusion, the study provides a novel, multidimensional measurement framework for political bias in LLMs, demonstrates that bias is more strongly linked to alignment methodology and institutional context than to model size or openness, and offers actionable insights for developers, policymakers, and researchers aiming to build more fair and transparent AI systems.