putting-relevant-passages-first-mitigates-position-effects CLM-0123Where a passage sits in a long input changes how much the model uses it — accuracy is highest when the needed information is at the very start or the very end and lowest when it is in the middle — so ordering retrieved passages to put the most relevant ones first is a real lever on accuracy.
Evidence for: First impressions last (narrows)
Capability: Losing information in long inputs · Context, Retrieval-augmented QA
Observed on
2023, GPT-3.5 / Claude / open long-context models.
Applies when
- Effect on the capability:
- helps
- Needs:
- documents or an index to retrieve from
- Cost:
- low
- Fails when:
- Which passage matters is not known before the model reads them, or the skew sits in the representation so that reordering does not remove it. This claim is contested on exactly that point.
Sources
- Measures the U-shaped curve directly: multi-document QA and key-value retrieval accuracy drops sharply when the relevant document sits in the middle of the context, and is highest at the beginning or the end.
- Contests the remedy rather than the effect. When several passages genuinely conflict, whichever is placed first dominates the output, and randomly permuting the evidence does not move that dominance — the skew is already present in the residual-stream representation of the combined prompt, so reordering cannot be the fix.
- A third position, from the builder. Anthropic reports that earlier models "could sometimes need repeated instructions or be more likely to listen to instructions at the end of their context window than at the start", and that for the Claude 5 generation they deleted those repeats with no measurable loss. If the positional effect has weakened in newer models, the technique is a remedy for a property that is being trained away rather than a durable one. Unverifiable from outside: the evaluations are not published.
Suspected axis of disagreement(a guess, not verified)
Two axes now, because there are three positions. The first is what the other passages are. The supporting result places one relevant passage among distractors, where moving it out of the middle recovers accuracy that position was costing. The 2026 conflict result places several mutually incompatible passages of equal legitimacy, where position is not degrading access to a known answer but deciding which answer wins — and reordering only moves which passage occupies the winning slot. If that is right, both hold, and ordering is a remedy for distraction rather than for conflict. The second is model generation, and it cuts across the first. Anthropic says the end-of-context bias it used to prompt around is largely gone in the Claude 5 generation. So the effect may be real, real still under conflict, and being trained away for ordinary retrieval — three claims that can all hold at once about different models at different times. This is exactly what the mechanism-versus-observation split is for, and it is an argument that this claim is closer to observation than it is currently filed. I have not tested any of the three setups against each other.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- In LLM fact-checking over supplied evidence, high veracity accuracy does not imply the verdict depends on that evidence: when the model's own cited sentences are iteratively removed, off-the-shelf models (GPT-4o-mini, Gemini-2.5-Flash-Lite, Qwen-2.5-32B, Llama-3.1-8B) keep their SUPPORT/REFUTE labels at a non-zero plateau instead of abstaining, and training with paired supervision that requires NOT_ENOUGH_INFO on the evidence-ablated input removes most of that persistence without losing accuracy.Checking claims against evidence · reviewed by AI
- Given several conflicting but equally legitimate passages, a model's answer is driven disproportionately by whichever passage comes first, and the skew sits in the representation of the combined prompt rather than in decoding — which is why reordering the passages does not remove it and why steering the activations does.Checking claims against evidence
- When an 8B model verifies natural-language claims against knowledge-graph triples, scoring the evidence twice under opposing prompts (argue it supports, argue it refutes) and adjudicating between the two arguments avoids the bias toward a "Supported" verdict that a single-perspective assessment shows, and is the largest single contributor to accuracy in that pipeline.Checking claims against evidence · reviewed by AI
- In ESG-report RAG question answering with retrieval held fixed at a single top-1 chunk, open-weight 2B-30B generators vary widely in how well they stay anchored to the retrieved passage (faithfulness 0.607-0.822) yet all land in a narrow low band of factual correctness (0.387-0.449), and the most faithful model is the least factually correct, so groundedness to context and correctness of extracted ESG facts do not track each other.Competence in a specialist domain · reviewed by AI
- In retrieval-augmented answers, splitting an answer into claims, checking each against the retrieved source, and then acting on the flagged claims reduces the share of answers judged to contain unsupported content — but the strategies trade grounding against preservation: deleting unsupported claims reduces unsupported content most while retaining the least original text, and rewriting retains the most while reducing least.Stating false facts confidently · reviewed by AI
Notes
Filed deliberately as an incumbent position rather than as a held belief. The catalog previously recorded only claims it endorsed, which left contesting evidence with nothing to attach to — the reason the backtest grades known reversals as missed even when the reversing paper is on file. A claim someone reasonably holds, with the evidence against it attached, is more useful than its absence.