decomposing-long-answers-into-atomic-facts-checking-eachmechanismsingle paper
Decomposing long answers into atomic facts and checking each with search catches most errors and beats human annotators on cost and agreement.
Capability: Stating false facts confidently
Sources
- Decomposing long answers into atomic facts and checking each with search catches most errors and beats human annotators on cost and agreement.
Disagreeing is the most useful thing you can do here. Both sides of every contested claim in this catalog were assembled by the same person, which is its weakest point.
Related claims
- In retrieval-augmented answers, splitting an answer into claims, checking each against the retrieved source, and then acting on the flagged claims reduces the share of answers judged to contain unsupported content — but the strategies trade grounding against preservation: deleting unsupported claims reduces unsupported content most while retaining the least original text, and rewriting retains the most while reducing least.Stating false facts confidently · unreviewed
- Long-form biographies from strong models contain a substantial share of unsupported atomic facts, rising for less well-known subjects.Stating false facts confidently
- Accuracy on a fact scales with how many pretraining documents mention it, so long-tail facts are systematically unreliable.Stating false facts confidently