A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions

Hallucination in large language models is reframed here not as a single failure mode but as two distinct ones: factuality hallucination (contradicting or fabricating real-world facts) and faithfulness hallucination (deviating from the user's instruction, the provided context, or the model's own reasoning chain). This replaces the older intrinsic/extrinsic split inherited from task-specific NLG, which the authors argue no longer fits open-ended general-purpose models.

The survey covers causes across 3 stages — data, training, inference — plus detection methods, benchmarks, mitigation strategies, and the limits of retrieval-augmented generation. Named causes include imitative falsehoods memorised from flawed pre-training corpora, long-tail, up-to-date and copyright-restricted knowledge boundaries, exposure bias and the snowball effect, sycophancy induced by RLHF preference models, sampling temperature, softmax bottleneck, and the Reversal Curse.

One cited result is load-bearing and counterintuitive: supervised fine-tuning on instruction pairs containing facts outside the model's pre-training knowledge boundary correlates with more hallucination, not less. Task-format-focused and overly complex instructions also raise hallucination rates.

This is a literature synthesis, not an experiment. No new measurements, no baselines, no models run. Every quantitative claim is second-hand and the reader must chase the primary sources.

Use it as a map of the causal taxonomy and detection literature, not as evidence for any specific mitigation.