arxiv-2608-25869 SRC-0638 · paperAnchoring Bias in LLM-as-a-Judge Systems: Prior Scores Compromise Evaluation Independence
Created: 2026-08-26 · Ingested: 2026-10-24
https://arxiv.org/abs/2608.25869(opens in a new tab)Large language models (LLMs) increasingly assess generated content, giving rise to the LLM-as-a-Judge paradigm. These systems now score outputs, filter content, and gate iterative refinement in production pipelines, where each judgment is often assumed to be independent of earlier evaluations. We test this assumption using three prompt conditions: no metadata, revision framing, and anchored metadata containing revision, attempt, and prior-score fields. We show that prior scores, even when included only as context metadata, anchor judgments and systematically shift ratings toward their values.
Referenced by
Claims in this catalog that draw on this source, and whether as support or counterpoint.