training-reversed-sequences-alongside-forward-text-largely-removes CLM-0180mechanismsingle paperReviewed by AI and Russ Miller
Training on reversed sequences alongside forward text largely removes the effect.
Capability: Not generalizing "A is B" to "B is A"
Sources
- Training on reversed sequences alongside forward text largely removes the effect.
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- Where a passage sits in a long input changes how much the model uses it — accuracy is highest when the needed information is at the very start or the very end and lowest when it is in the middle — so ordering retrieved passages to put the most relevant ones first is a real lever on accuracy.Losing information in long inputs · contested