Mind the Confidence Gap: Overconfidence, Calibration, and Distractor Effects in Large Language Models

Large Language Models (LLMs) show remarkable proficiency in natural language tasks, yet their frequent overconfidence-misalignment between predicted confidence and true correctness-poses significant risks in critical decision-making applications. We present a comprehensive analysis on calibration in LLMs across nine LLMs and three factual Question-Answering (QA) datasets, systematically comparing standard free-generation settings against structured distractor-augmented prompts. Our evaluation reveals that explicitly incorporating distractors can substantially mitigate miscalibration, achieving relative accuracy improvements up to 460% and ECE reductions up to 90%. Despite general trends, we uncover nuanced findings: large RLHF-tuned models display inherent calibration strengths but can paradoxically suffer increased miscalibration on easier queries, whereas smaller models benefit disproportionately from distractor prompts but remain significantly miscalibrated. Through detailed analyses across question types, we identify persistent calibration failures, particularly in person-based queries. We conclude with concrete recommendations-targeted fine-tuning, structured prompting, and strategic model choice-to ensure reliable, trustworthy LLM deployments.

💡 Research Summary

This paper, “Mind the Confidence Gap: Overconfidence, Calibration, and Distractor Effects in Large Language Models,” presents a comprehensive empirical investigation into the miscalibration of LLMs, where a model’s expressed confidence fails to align with its actual accuracy. The work highlights the significant risks this poses for high-stakes deployments and introduces a novel prompting strategy to mitigate the issue.

Core Problem & Approach: The authors systematically evaluate calibration across nine state-of-the-art LLMs—spanning diverse scales (from ~8B to undisclosed large parameters), architectures (dense and MoE), and alignment techniques (SFT and RLHF). These include models from the GPT-4 family, the LLaMA-3 series, LLaMA-4-Scout-17B, Gemma2-9B-it, and Qwen-qwq-32B. The models are assessed on three factual QA datasets: SimpleQA, FaVIQ, and TriviaQA. The key methodological innovation is the comparison between two prompting regimes:

1. A Normal (free-generation) setting, where the model answers a question directly.
2. A Distractor-augmented setting, where the model is presented with the question alongside a structured list of one correct and three plausible incorrect answers (distractors) and must choose from among them while stating its confidence.

Confidence is measured via elicited self-reports (0-100) to ensure a uniform metric across black-box and open-weight models. Performance is judged by an LLM-based evaluator (GPT-4o-mini).

Key Findings:

1. Substantial Calibration Improvement with Distractors: The distractor-augmented setting consistently and dramatically improves both accuracy and Expected Calibration Error (ECE) compared to free-generation. In the most striking case on SimpleQA, GPT-4o-mini’s accuracy improved relatively by ~460% (from 8.46% to 47.43%), while its ECE dropped by over 90%. This demonstrates that forcing models to explicitly consider alternatives, mirroring a human “consider-the-opposite” cognitive strategy, effectively curbs overconfidence.
2. Nuanced Model-Specific Behaviors:
   • Paradox in Large RLHF-tuned Models: While large RLHF models like GPT-4o and LLaMA-3-70B generally showed strong inherent calibration (low ECE), they exhibited a counterintuitive increase in miscalibration on easier queries (TriviaQA) when distractors were introduced. This suggests that highly aligned models may become overconfident when presented with explicit, seemingly clear-cut choices.
   • Disproportionate Benefit for Smaller Models: Smaller models (e.g., LLaMA-3-8B, Gemma2-9B) saw massive absolute gains in accuracy from the distractor setting. However, they still maintained significantly higher ECE levels than their larger counterparts, indicating that while their performance improves, their ability to accurately gauge their own correctness remains deficient.
3. Persistent Failure Modes: Fine-grained analysis across question types revealed that person-based queries are a particular source of persistent calibration failure, likely due to the ambiguity and multiplicity of factual information about individuals.
4. Net Positive with Exceptions: The distractor setting “helped” (improved accuracy or confidence alignment) in the vast majority of instances (often >90%) but still “harmed” a non-negligible minority (up to 22.68%), causing incorrect answers or worse confidence estimation.

Conclusions and Recommendations: The study concludes that mitigating overconfidence requires moving beyond post-hoc calibration methods. It offers three concrete, actionable recommendations for reliable deployment:

• Targeted Fine-tuning: Incorporate calibration-aware objectives (e.g., in RLHF) and use data augmentation focused on identified failure domains like person queries.
• Structured Prompting: Where feasible, deploy models using prompts that present explicit answer choices, as this paradigm leads to better-calibrated confidence estimates than open-ended generation.
• Strategic Model Choice: Select models based on task difficulty and required reliability, being mindful of the potential for increased miscalibration in large RLHF models on easy tasks and the fundamental calibration limitations of smaller models.

By rigorously quantifying the effect of distractors and uncovering the complex interplay between model scale, alignment, and calibration, this research provides crucial insights for building more trustworthy and transparent LLM-based systems.