arxiv-2608-29307 · paper

Detecting and Repairing Hallucinations in Retrieval-Augmented Generation

Created: 2026-08-29 · Ingested: 2026-09-09

Language models increasingly answer questions by consulting retrieved documents rather than memory alone, a design now common in search assistants and enterprise knowledge tools. Grounding a model in retrieved text reduces unsupported statements but does not eliminate them, and a reader cannot tell a grounded sentence from an invented one. Most research on this problem stops at detection, yet flagging a faulty answer changes nothing for the person reading it, and little is known about which action should follow. Using RAGTruth, a benchmark whose unsupported passages are annotated by hand, we s

In brief

Flagging a hallucination in a RAG answer is only the start; the repair action you choose trades grounding against how much of the answer survives, and there is no dominant option.

Using RAGTruth, where unsupported passages are hand-annotated, each flagged answer is decomposed into individual factual claims, each claim checked against the retrieved source, and 3 repair strategies of increasing richness are compared against leaving the answer untouched: deleting an unsupported claim, replacing it with source text, and rewriting it. 3 language models from different families judged the 916 repaired answers.

All 3 strategies cut the proportion of answers judged to contain unsupported content, and the 3 judges agree on the ordering. Deletion reduces the most while retaining the least original text, at 64.3%; rewriting retains 80.1% and reduces the least. The pipeline also edits 83.5% of answers annotated clean, so repair is not confined to faulty ones.

The grounding effect is measured, with 3 independent judge families as the main cross-check, but there is no human judgment of answer usefulness and only 1 benchmark. Whether an edited answer still helps a reader is untested, and the over-editing rate on clean answers is a direct cost.

If you are building RAG guardrails, treat repair choice as a trade-off you must evaluate downstream, not a solved ranking.

Written from the abstract by claude-opus-5 on 2026-09-09, with every figure checked against it. Not a substitute for the paper.

Referenced by

Claims in this catalog that draw on this source, and whether as support or counterpoint.

supportsIn retrieval-augmented answers, splitting an answer into claims, checking each against the retrieved source, and then acting on the flagged claims reduces the share of answers judged to contain unsupported content — but the strategies trade grounding against preservation: deleting unsupported claims reduces unsupported content most while retaining the least original text, and rewriting retains the most while reducing least.