Judge both orders and reconcile
Run the judge with candidates in both orders and count a preference only when it holds in both.
For pairwise judging, call the judge twice with the candidates swapped. If the verdicts disagree, record a tie. Add a length-normalized rubric and never use a judge from the same model family as a candidate when it can be avoided.
Contexts: LLM as judge
Does it work?
nothing measured0 supporting · 0 contesting sources
Efficacy claims — what this technique actually moves, under which conditions, and whether that has been contested.
No efficacy claim filed yet. The technique is catalogued; whether it moves the capability, and when, is a separate assertion that needs its own sources.
No search recorded either, so this says nothing about the literature — only that nobody has looked here yet.
Code
- https://github.com/lm-sys/FastChat — MT-Bench judging code with position swapping.verified 2026-09-02
Sources
Whether a technique works is a claim. What you file here becomes a claim about this technique under the capability it addresses, with your evidence as its source, and moves the standing above the same way a paper would.