Performance on multi-document QA is highest when the answer is at the start or end of the context and drops sharply in the middle.

Capability: Losing information in long inputs

Sources

supportsLost in the Middle: How Language Models Use Long Contexts
Performance on multi-document QA is highest when the answer is at the start or end of the context and drops sharply in the middle.

Status: activeLast checked: 2026-09-03Evidence activity

Disagreeing is the most useful thing you can do here. Both sides of every contested claim in this catalog were assembled by the same person, which is its weakest point.

Related claims

On multi-document question answering with GPT-3.5 Turbo's 16k-context variant, accuracy dropped by more than twenty points when the document containing the answer was moved from the start or end of the context to the middle, with nothing else about the task changed.Losing information in long inputs