Inference-Time Reasoning Selectively Reduces Implicit Social Bias in Large Language Models

Drawing on constructs from psychology, prior work has identified a distinction between explicit and implicit bias in large language models (LLMs). While many LLMs undergo post-training alignment and safety procedures to avoid expressions of explicit social bias, they still exhibit significant implicit biases on indirect tasks resembling the Implicit Association Test (IAT). Recent work has further shown that inference-time reasoning can impair LLM performance on tasks that rely on implicit statistical learning. Motivated by a theoretical link between implicit associations and statistical learning in human cognition, we examine how reasoning-enabled inference affects implicit bias in LLMs. We find that enabling reasoning significantly reduces measured implicit bias on an IAT-style evaluation for some model classes across fifteen stereotype topics. This effect appears specific to social bias domains, as we observe no corresponding reduction for non-social implicit associations. As reasoning is increasingly enabled by default in deployed LLMs, these findings suggest that it can meaningfully alter fairness evaluation outcomes in some systems, while also raising questions about how alignment procedures interact with inference-time reasoning to drive variation in bias reduction across model types. More broadly, this work highlights how theory from cognitive science and psychology can complement AI evaluation research by providing methodological and interpretive frameworks that reveal new insights into model behavior.

💡 Research Summary

This paper investigates how inference‑time reasoning influences the expression of implicit social bias in large language models (LLMs). Drawing on psychological theory that distinguishes explicit from implicit bias, the authors note that while alignment and safety training can suppress overt (explicit) bias, LLMs still display implicit bias on tasks modeled after the Implicit Association Test (IAT). Recent work in cognitive science shows that prompting models to “think step‑by‑step” (chain‑of‑thought or other reasoning techniques) can impair performance on tasks that rely on implicit statistical learning. The authors hypothesize that enabling reasoning during inference will similarly dampen implicit bias in LLMs because implicit bias is thought to arise from automatic distributional learning.

To test this, they conduct two experiments. Experiment 1 adapts the “LLM Word Association Test” introduced by Bai et al., which mirrors the IAT by asking a model to assign target group names (e.g., male vs. female names) to a list of attribute words (e.g., career‑related vs. family‑related). Fifteen stereotype topics spanning race, gender, religion, and health are evaluated. For each model and condition, 50 random runs are performed, and a bias score ranging from –1 (counter‑stereotypical) to +1 (stereotypical) is computed. The models examined include OpenAI GPT‑4.1 (no built‑in reasoning) and o3 (built‑in reasoning), Anthropic Claude Opus 4.1, Google Gemini 2.5 Flash, and Meta Llama 3.3 70B Instruct. Reasoning is toggled via model‑specific flags (or via CoT prompting for Llama). Independent‑samples t‑tests compare standard inference to reasoning‑enabled inference for each model‑topic pair and for aggregated scores.

Results show that reasoning‑enabled inference significantly reduces bias scores for several model families, especially GPT‑4.1 vs. o3 and Claude Opus 4.1 vs. its reasoning‑enabled counterpart. Reductions range from modest (≈30 %) to dramatic (up to 91 % for certain topics). Some models, notably Llama with CoT prompting, exhibit little change, indicating that the effect depends on how reasoning is implemented. The overall pattern suggests that when models are forced to articulate intermediate steps, the automatic associative mechanisms that drive implicit bias are weakened, leading to more neutral or less stereotypical responses.

Experiment 2 examines whether this effect generalizes to non‑social implicit associations. Using a set of neutral words that carry positive or negative semantic prosody, the same models and reasoning conditions are tested. In contrast to the social bias tasks, reasoning does not produce a statistically significant change in bias scores, implying that the reasoning‑bias interaction is specific to stereotype‑laden content rather than a blanket suppression of all distributional associations.

The authors discuss several implications. First, the findings support the view that implicit bias in LLMs is rooted in statistical learning of co‑occurrence patterns, and that reasoning can interrupt this process. Second, the variability across model families highlights that architectural differences and the exact mechanism of reasoning (built‑in vs. prompting) matter. Third, as many deployed systems now enable reasoning by default, fairness evaluations that rely on implicit bias measures may be systematically affected, potentially under‑reporting bias. However, the authors caution that reduced implicit bias does not automatically equate to improved fairness; reasoning may also introduce other risks or mask biases in different contexts.

Limitations include the relatively small number of repetitions per condition, the heterogeneity of reasoning hyper‑parameters, and the focus on English‑language models and a limited set of stereotypes. Future work is suggested to explore a broader range of reasoning techniques (e.g., self‑consistency, self‑refine), to test multilingual models, to expand the set of social domains, and to assess how reasoning‑induced bias changes manifest in real‑world user interactions.

In conclusion, the study demonstrates that inference‑time reasoning can selectively reduce implicit social bias in certain LLMs, offering a novel lens for interpreting bias evaluations and informing the design of mitigation strategies in next‑generation language technologies.