claim-level-uncertainty-estimated-response-claim-graph-anchoring-graph-trusted-external CLM-0235When claim-level uncertainty is estimated from a response-claim graph, anchoring the graph with trusted external priors (edges weighted by entailment from verified source sections) and scoring claims by weighted closeness centrality separates ungrounded claims from repeatedly-generated-but-false ones better than internal-consistency centrality alone, and yields retrieval whose answerability tracks its accuracy instead of the near-100% answer rate GraphRAG shows on the same long documents.
Capability: Stating false facts confidently
Observed on
Long-document QA (QASPER, QuALITY) with GPT-4o-mini and LLaMA 3.1 8B, where trusted priors can be carved out of the document itself (abstract, intro, chunks); no gold key for claim.
Sources
- supportsGRACE: Graph-Grounded Reflective Agent Copilot Engine for Expert-in-the-Loop Knowledge ExpansionValidated by: QA accuracy and answerability against zero-shot partial/full context, RAG, and GraphRAG on 100 QASPER papers and 50 QuALITY articles; plus a confusion matrix against an LLM oracle showing 97.6% grounding precision and 85.5% of hallucinated claims placed in Boundary/Refuted.. Measured on two benchmarks and two backbones, but accuracy gains over GraphRAG are mixed (GraphRAG leads on QuALITY with GPT-4o-mini); [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For claim-level hallucination detection on long-document QA, estimating a claim's reliability from how central it is in a graph of sampled model responses cannot separate true claims from confidently repeated false ones, because both get high agreement; adding trusted source documents as anchor nodes with higher-weight edges shifts the basis from internal consensus to external grounding and yields a Grounded pool with few hallucinations (97.6% precision on QASPER), at the cost of conservatively leaving correct-but-novel claims on the boundary.Stating false facts confidently · reviewed by AI
- In retrieval-augmented answers, splitting an answer into claims, checking each against the retrieved source, and then acting on the flagged claims reduces the share of answers judged to contain unsupported content — but the strategies trade grounding against preservation: deleting unsupported claims reduces unsupported content most while retaining the least original text, and rewriting retains the most while reducing least.Stating false facts confidently · reviewed by AI
- Decomposing a response into atomic claims and applying post-hoc temperature or Platt scaling to per-claim self-verification/log-probability confidence lowers expected calibration error on straightforward factual QA (TriviaQA) in closed-box settings, but on adversarial false-premise questions (TruthfulQA) it is worse than response-level temperature scaling, because the per-claim score faithfully reports the model's own mistaken confidence rather than correcting it.Saying it does not know · reviewed by AI
- When an 8B model verifies natural-language claims against knowledge-graph triples, scoring the evidence twice under opposing prompts (argue it supports, argue it refutes) and adjudicating between the two arguments avoids the bias toward a "Supported" verdict that a single-perspective assessment shows, and is the largest single contributor to accuracy in that pipeline.Checking claims against evidence · reviewed by AI
- In LLM fact-checking over supplied evidence, high veracity accuracy does not imply the verdict depends on that evidence: when the model's own cited sentences are iteratively removed, off-the-shelf models (GPT-4o-mini, Gemini-2.5-Flash-Lite, Qwen-2.5-32B, Llama-3.1-8B) keep their SUPPORT/REFUTE labels at a non-zero plateau instead of abstaining, and training with paired supervision that requires NOT_ENOUGH_INFO on the evidence-ablated input removes most of that persistence without losing accuracy.Checking claims against evidence · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: low. Falsifier as drafted: An ablation removing the prior anchors and prior-weighted edges (leaving plain response-consistency centrality) that matches GRACE's grounding precision and calibrated answerability on the same documents. Drafted stance toward decomposing-long-answers-into-atomic-facts-checking-each: supports -- It again finds that atomic-claim decomposition plus external checking catches unsupported content that whole-response consistency does not.