Toward Revealing Nuanced Biases in Medical LLMs

Large language models (LLMs) used in medical applications are known to be prone to exhibiting biased and unfair patterns. Prior to deploying these in clinical decision-making, it is crucial to identify such bias patterns to enable effective mitigation and minimize negative impacts. In this study, we present a novel framework combining knowledge graphs (KGs) with auxiliary (agentic) LLMs to systematically reveal complex bias patterns in medical LLMs. The proposed approach integrates adversarial perturbation (red teaming) techniques to identify subtle bias patterns and adopts a customized multi-hop characterization of KGs to enhance the systematic evaluation of target LLMs. It aims not only to generate more effective red-teaming questions for bias evaluation but also to utilize those questions more effectively in revealing complex biases. Through a series of comprehensive experiments on three datasets, six LLMs, and five bias types, we demonstrate that our proposed framework exhibits a noticeably greater ability and scalability in revealing complex biased patterns of medical LLMs compared to other common approaches.

💡 Research Summary

**
The paper introduces a novel framework for systematically uncovering nuanced and intersectional bias in medical large language models (LLMs). Recognizing that existing bias‑evaluation methods either focus on single attributes or rely heavily on human judgment, the authors combine knowledge graphs (KGs) with auxiliary “attacker” LLMs to generate targeted perturbations of clinical questions and then probe the target LLM with a multi‑hop reasoning process.

Framework Overview

1. KG Construction – Starting from seed clinical text (question‑answer pairs, patient notes, etc.), entities such as patient demographics, symptoms, diseases, and social determinants are extracted using NLP tools (spaCy) and rule‑based pattern matching. The extracted information is organized into directed triples (head, relation, tail).
2. Perturbed Question Generation – A generator LLM first creates natural‑language sentences that summarize the KG context. An attacker LLM, prompted in a few‑shot fashion, systematically modifies selected attributes (e.g., age, gender, ethnicity, comorbidities) to produce a set of perturbed questions. Two bias categories are defined: (i) attribute bias (single‑attribute changes) and (ii) intersectional bias (simultaneous changes to multiple attributes).
3. Answer Generation with Multi‑Hop Reasoning – The perturbed questions are fed to the target medical LLM. The authors design a three‑step chain‑of‑thought (CoT) prompting pipeline:
   • Step 1 – Triplet Generation: Convert the question’s explicit information into KG triples.
   • Step 2 – Triplet Expansion: Leverage the target LLM’s internal knowledge to extend these triples, linking them to broader medical ontologies, demographic risk factors, and latent associations. This creates multi‑hop relationships that go beyond the surface text.
   • Step 3 – Answer Generation: Using the expanded multi‑hop context, the LLM produces a clinical response while the CoT prompt encourages it to articulate reasoning steps, making hidden biases more observable.

Experimental Setup

• Datasets: Three public medical QA corpora (MedQA, PubMedQA, and a MIMIC‑derived set).
• Models: Six LLMs covering general‑purpose (GPT‑4, Claude, LLaMA‑2‑70B) and domain‑specific (MedPaLM, BioGPT, ChatGPT‑3.5).
• Bias Types: Gender, age, race/ethnicity, socioeconomic status, and composite intersectional scenarios (e.g., elderly + minority + low‑income).

Results

• The proposed KG‑augmented perturbation pipeline generated more diverse and challenging question variants than manually crafted baselines, as confirmed by expert ratings.
• When evaluated with the multi‑hop CoT reasoning, the target LLMs exhibited measurable shifts in answer content, risk assessments, and sentiment for perturbed inputs.
• Compared to conventional “LLM‑as‑judge” evaluations, the new framework improved bias detection rates by an average of 18 percentage points, with a striking 25 percentage‑point gain for intersectional bias cases.
• Statistical analyses showed higher inter‑rater agreement (Cohen’s κ) for bias scores derived from the proposed method, indicating more reliable detection.

Limitations & Future Work

• KG quality depends on the accuracy of entity extraction; errors can propagate to downstream perturbations.
• The attacker LLM’s perturbation logic is constrained by a predefined attribute list, limiting exploration of emergent bias dimensions.
• Multi‑hop reasoning incurs additional computational overhead, which may hinder real‑time deployment.
  Future directions include automated entity normalization, dynamic expansion of attribute vocabularies, and efficient prompting techniques (e.g., lightweight CoT, knowledge distillation) to scale the framework.

Conclusion
By integrating structured knowledge from KGs with adversarial question generation and a three‑step multi‑hop reasoning pipeline, the authors provide a scalable, automated tool for revealing subtle and compound biases in medical LLMs. The framework outperforms existing evaluation approaches, especially in detecting intersectional bias, and offers a promising foundation for developing fairer, safer AI‑driven clinical decision support systems.