arxiv-2608-25869 SRC-0638 · paper

Anchoring Bias in LLM-as-a-Judge Systems: Prior Scores Compromise Evaluation Independence

Created: 2026-08-26 · Ingested: 2026-10-24

https://arxiv.org/abs/2608.25869(opens in a new tab)

Large language models (LLMs) increasingly assess generated content, giving rise to the LLM-as-a-Judge paradigm. These systems now score outputs, filter content, and gate iterative refinement in production pipelines, where each judgment is often assumed to be independent of earlier evaluations. We test this assumption using three prompt conditions: no metadata, revision framing, and anchored metadata containing revision, attempt, and prior-score fields. We show that prior scores, even when included only as context metadata, anchor judgments and systematically shift ratings toward their values.

Referenced by

Claims in this catalog that draw on this source, and whether as support or counterpoint.