decomposing-long-answers-into-atomic-facts-checking-each CLM-0039mechanismsingle paperReviewed by AI and Russ Miller
Decomposing long answers into atomic facts and checking each with search catches most errors and beats human annotators on cost and agreement.
Capability: Stating false facts confidently
Sources
- Decomposing long answers into atomic facts and checking each with search catches most errors and beats human annotators on cost and agreement.
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- In retrieval-augmented answers, splitting an answer into claims, checking each against the retrieved source, and then acting on the flagged claims reduces the share of answers judged to contain unsupported content — but the strategies trade grounding against preservation: deleting unsupported claims reduces unsupported content most while retaining the least original text, and rewriting retains the most while reducing least.Stating false facts confidently · reviewed by AI
- When claim-level uncertainty is estimated from a response-claim graph, anchoring the graph with trusted external priors (edges weighted by entailment from verified source sections) and scoring claims by weighted closeness centrality separates ungrounded claims from repeatedly-generated-but-false ones better than internal-consistency centrality alone, and yields retrieval whose answerability tracks its accuracy instead of the near-100% answer rate GraphRAG shows on the same long documents.Stating false facts confidently · reviewed by AI
- Long-form biographies from strong models contain a substantial share of unsupported atomic facts, rising for less well-known subjects.Stating false facts confidently
- Accuracy on a fact scales with how many pretraining documents mention it, so long-tail facts are systematically unreliable.Stating false facts confidently
- For factual paired-candidate QA where the model already holds the needed knowledge, hallucination can be detected by removing the query keyword that most drives the chosen answer and seeing whether the prediction systematically recovers to the alternative — erroneous answers show that recovery while correct answers merely become ambiguous — but the signal weakens when the knowledge is absent and largely fails on multi-step arithmetic reasoning, where errors are distributed across steps rather than bound to one keyword.Stating false facts confidently · reviewed by AI