Checking claims against evidence

Claims

• mechanismsingle paper↑1↓0
  When an 8B model verifies natural-language claims against knowledge-graph triples, scoring the evidence twice under opposing prompts (argue it supports, argue it refutes) and adjudicating between the two arguments avoids the bias toward a "Supported" verdict that a single-perspective assessment shows, and is the largest single contributor to accuracy in that pipeline.
• mechanismsingle paper↑2↓0
  Given several conflicting but equally legitimate passages, a model's answer is driven disproportionately by whichever passage comes first, and the skew sits in the representation of the combined prompt rather than in decoding — which is why reordering the passages does not remove it and why steering the activations does.
• observationsingle paper↑1↓0
  When an LLM agent must reconcile user-supplied identity credentials against database records before a sensitive read or write, frontier and open models frequently skip the cross-field consistency check and act anyway, and the failure rate barely moves whether the request is simple or has several parallel sub-requests and whether the forged field is visually near-identical to the true one or completely unrelated.
• mechanismsingle paper↑1↓0
  In LLM fact-checking over supplied evidence, high veracity accuracy does not imply the verdict depends on that evidence: when the model's own cited sentences are iteratively removed, off-the-shelf models (GPT-4o-mini, Gemini-2.5-Flash-Lite, Qwen-2.5-32B, Llama-3.1-8B) keep their SUPPORT/REFUTE labels at a non-zero plateau instead of abstaining, and training with paired supervision that requires NOT_ENOUGH_INFO on the evidence-ablated input removes most of that persistence without losing accuracy.
• mechanismsingle paper↑1↓0
  Models endorse widely held falsehoods, showing weak verification against what they know.
• mechanismsingle paper↑1↓0
  Models readily adopt a single counter-memory passage when it is coherent, and when sources conflict they follow the majority and show confirmation bias toward their own beliefs.
• mechanismsingle paper↑1↓0
  When a prompt contains several legitimate but conflicting evidence passages, models internally register the conflict — linear probes on mid-to-late-layer hidden states separate conflicting from consistent inputs at high AUC — while still resolving the conflict badly at the output, so the failure is in evidence integration rather than in conflict detection.
• mechanismsingle paper↑1↓0
  Training the model to decide when to retrieve and to critique whether passages support its output improves factuality and citation accuracy.
• mechanismsingle paper↑1↓0
  When training small instruction-tuned models with reinforcement learning plus factual supervision, routing each atomic fact's verification score only to the tokens that produced it, and down-weighting verifier judgements that do not change when their key evidence is removed, improves factuality benchmark scores over trajectory-level or reasoning- step-level factual rewards.
• observationsingle paper↑1↓0
  When VLM agents must verify claims against interactive visualizations where no claim is answerable from the initial viewport, giving them an interaction budget of ten actions does not reliably beat answering immediately from the first screenshot — some models score slightly lower with interaction — because partial, unplanned exploration leaves the model with enough evidence to abandon its prior but not enough to replace it.

Techniques

• Check long answers fact by facttooling
  Split a long answer into atomic claims and verify each against search or a corpus.
• Ground answers in retrieved documentsretrieval
  Retrieve relevant documents and generate from them, with the model deciding when retrieval is needed.
• neutral-center activation steering for evidence integrationprocess
  neutral-center activation steering for evidence integration, as introduced by the paper it was drafted from.
• Supervise abstention on evidence-ablated inputstraining
  Fine-tune a verifier on paired inputs — full context with the gold label and evidence, and the same context with gold evidence removed and target NOT_ENOUGH_INFO — with gold evidence sets first expanded by majority vote of three other LLMs.

Sources

Every paper, post or observation behind this capability. Ones cited by a claim show the claim and its stance; ones filed directly are about the capability but have no claim drafted from them yet. Open one to add support or contest it.

• DARE: Dialectical Agentic Reasoning for Structured Knowledge Fact Checking2026-09-12
  ↑ supports a claim here
• Evaluating and Improving Evidence-Grounded Fact-Checking in LLMs via Multi-Round Evidence Ablation2026-09-08
  ↑ supports a claim here
• KC-Bench: A Dynamic Interactive Benchmark for Evaluating Knowledge Conflicts in LLM Agents2026-09-03
  ↑ supports a claim here
• Large Language Models in Resolving Contextual Knowledge Conflicts2026-09-02
  ↑ supports a claim here · ↑ supports a claim here
• InSight: A Benchmark for Agentic Claim Verification in Interactive Visualizations2026-09-01
  ↑ supports a claim here
• FARCA: Fact-Aligned Reliability-Aware Credit Assignment for Reinforcement Learning with Factual Supervision2026-08-25
  ↑ supports a claim here
• Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge ConflictsThis source: Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large …How much the field cites it — heavily cited in the last 12 months148 in the last 12 months · 382 totalpublished 2023checked 2026-09-042023
  ↑ supports a claim here · ↑ supports a claim here
• TruthfulQA: Measuring How Models Mimic Human FalsehoodsThis source: TruthfulQA: Measuring How Models Mimic Human FalsehoodsHow much the field cites it — very heavily cited in the last 12 months1000+ in the last 12 months · 3851 totalpublished 2021checked 2026-09-04count capped at 1000 by the fetch2021
  ↑ supports a claim here
• Self-RAG: Learning to Retrieve, Generate, and Critique through Self-ReflectionThis source: Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Re…How much the field cites it — very heavily cited in the last 12 months1000+ in the last 12 months · 2517 totalpublished 2023checked 2026-09-04count capped at 1000 by the fetch2023
  ↑ supports a claim here