Evaluating Scoring Bias in LLM-as-a-Judge

The “LLM-as-a-Judge” paradigm, using Large Language Models (LLMs) as automated evaluators, is pivotal to LLM development, offering scalable feedback for complex tasks. However, the reliability of these judges is compromised by various biases. Existing research has heavily concentrated on biases in comparative evaluations. In contrast, scoring-based evaluations-which assign an absolute score and are often more practical in industrial applications-remain under-investigated. To address this gap, we undertake the first dedicated examination of scoring bias in LLM judges. We shift the focus from biases tied to the evaluation targets to those originating from the scoring prompt itself. We formally define scoring bias and identify three novel, previously unstudied types: rubric order bias, score ID bias, and reference answer score bias. We propose a comprehensive framework to quantify these biases, featuring a suite of multi-faceted metrics and an automatic data synthesis pipeline to create a tailored evaluation corpus. Our experiments empirically demonstrate that even the most advanced LLMs suffer from these substantial scoring biases. Our analysis yields actionable insights for designing more robust scoring prompts and mitigating these newly identified biases.

💡 Research Summary

The paper “Evaluating Scoring Bias in LLM‑as‑a‑Judge” addresses a largely unexplored aspect of large language model (LLM) evaluation: the systematic biases that arise when LLMs are used as automated judges that assign absolute scores rather than making pairwise comparisons. While prior work has catalogued biases such as positional, length, and self‑preference effects in comparative settings, the authors note that scoring‑based evaluations—common in industry because they produce a single, interpretable quality metric—have received far less scrutiny.

To fill this gap, the authors formally define “scoring bias” as the measurable shift in scores produced by an LLM judge when the scoring prompt is perturbed, while the target response remains unchanged. They introduce three novel bias categories that stem from the prompt itself:

1. Rubric Order Bias – the ordering of the rubric entries (e.g., 1→5, 5→1, or a random permutation) influences the judge’s scoring tendency.
2. Score ID Bias – using alternative identifiers for scores (alphabetic grades A‑E, Roman numerals i‑v) instead of the standard Arabic numerals (1‑5) leads to systematic deviations.
3. Reference Answer Score Bias – attaching a specific score to a reference answer (e.g., providing a reference answer labeled with a score of 3 instead of the usual 5) can destabilize the judge’s scoring behavior.

To quantify these biases, the authors propose a comprehensive evaluation framework comprising three families of metrics:

• Stability metrics (Flip Rate and Mean Absolute Deviation) measure consistency between scores obtained from a perturbed prompt and a baseline prompt.
• Accuracy metrics (Spearman’s ρ and Pearson’s r) assess alignment with “golden” scores derived from human annotations or high‑performing LLMs (GPT‑4.1 in this work).
• Scoring‑tendency metrics capture the distribution of scores across the five‑point scale, revealing over‑ or under‑representation of particular categories.

The experimental setup leverages four widely used LLM‑as‑a‑Judge benchmarks: BiGGen, FLASK, MT‑Bench, and Vicuna‑Bench. Each dataset contains thousands of response instances with human or GPT‑4 derived gold scores. To explore Reference Answer Score Bias, the authors build an automatic generation‑review pipeline that synthesizes reference answers for scores 1‑4, using GPT‑4.1 and GPT‑4o alternately as response generators and reviewers.

Four judge models are evaluated: GPT‑4.1, GPT‑4o, Qwen‑3, and Mistral‑Small. The results are striking: all models exhibit non‑trivial Rubric Order Bias (average Flip Rate 12‑18 %, MAD increase of 0.3‑0.5 points), Score ID Bias (pronounced skew toward lower scores when non‑numeric IDs are used), and Reference Answer Score Bias (average score drops of 0.6‑0.9 points when reference answers are labeled with scores other than 5). Importantly, these effects persist regardless of model size or architecture; even the strongest model (GPT‑4.1) shows substantial deviations under prompt perturbations despite achieving the highest correlation with human scores in the unperturbed baseline (Spearman ρ≈0.60, Pearson r≈0.64).

Based on these findings, the authors propose concrete mitigation strategies:

• Keep rubric entries in a fixed ascending order (1→5) and avoid re‑ordering.
• Use only Arabic numerals for score identifiers; avoid grades or Roman numerals.
• When providing a reference answer, attach only the maximal score (5) and omit lower‑score references.
• Add explicit meta‑instructions (e.g., “Scores are integers from 1 (lowest) to 5 (highest)”) to the prompt to reinforce the intended scoring schema.

Applying these guidelines reduces Flip Rate and MAD by up to 70 % and yields a more balanced score distribution.

In summary, this work delivers the first systematic taxonomy of scoring‑related biases in LLM‑as‑a‑Judge systems, introduces a robust quantitative framework for their measurement, and supplies an automatic data‑synthesis pipeline that can be reused by the community. The empirical evidence that even state‑of‑the‑art LLMs are vulnerable to such prompt‑level biases underscores the necessity of careful prompt engineering when deploying LLM judges in real‑world applications. The paper’s insights and recommendations constitute a valuable foundation for future research on bias mitigation and for building more reliable, scalable evaluation pipelines for LLMs.