arxiv-2609-09754 SRC-0233 · paper

LexAgentHallu: A Hierarchical Benchmark for Profiling Hallucinations in Legal Agents

Yujin Zhou, Mingxuan Zheng, Chuxue Cao, Huang Yidan, Jiale Chen, Yike Guo

Created: 2026-09-09 · Ingested: 2026-09-19

As large language models are increasingly deployed as tool-augmented legal agents, they introduce agentic hallucinations where tool-call and reasoning errors cascade into fabricated holdings and miscited authority. However, existing legal benchmarks evaluate only single-turn QA with outcome-level metrics, while agentic hallucination benchmarks lack legal-specific diagnostic capability. Neither answers to what extent and how a legal agent hallucinates along its trajectory. To address these limitations, we introduce LexAgentHallu, a legal agentic hallucination benchmark designed to evaluate to w

Referenced by

Claims in this catalog that draw on this source, and whether as support or counterpoint.

supportsFor tool-augmented legal agents on multi-step Chinese-law tasks, a correct final answer does not indicate a clean reasoning trajectory: across 18 agent configurations, trajectories that reach the right answer still contain at least one substantive legal hallucination about 68% of the time on average, and that rate barely moves with backbone scale or agentic framework (0.61-0.78), while procedural-process hallucination rates vary widely (0.07-0.85).