2m-parameter-classifier-head-reads-hidden-states-already-computed-during CLM-0001A ~2M-parameter classifier head that reads hidden states already computed during generation (tapping shallow, middle and deep layers of the generating model) detects response-level hallucination at least as well as specialized hidden-state hallucination detectors, and its detection quality rises with the scale of the base model it is attached to.
Capability: Stating false facts confidently
Observed on
Two Ling-3.0 base models (tiny and flash); response-level ROC AUC on six offline hallucination benchmarks and seven free-generation datasets; head trained on curated factuality/hal.
Sources
- supportsSingProbe Technical ReportMeasured AUC, not accuracy at a threshold. Macro-average offline AUC 0.7765 (tiny) and 0.8012 (flash) vs best baseline DRIFT 0.7408/0.8000; online free-generation average 0.6786/0.7271 vs DRIFT 0.6316/0.6546, with labels from an LLM judge. Absolute AUCs are moderate and per-benchmark ranking is mixed (FAVA, RAGTruth, BBH favor baselines). Layer ablation shows hallucination gains more from multi-layer taps than safety does, supporting the mechanism reading. Only one model family, so scale claim rests on two points.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For training-free inference-time hallucination mitigation in 7B vision- language models, reported gains on object-hallucination benchmarks are largely inseparable from reduced informativeness: hallucination rate and object recall/coverage move together, so a lower score can mean the model mentioned fewer visual entities rather than grounded better.Whether the measurement made the finding · reviewed by AI
- In 7-8B open-weight instruct models (Mistral-7B, Qwen2.5-7B, Llama-3-8B) answering closed-book factual questions, a small set of feed-forward neurons (top 1% or fewer) selected on one dataset by class-conditional separation of their residual-space contribution at the final prompt token transfers to other factual QA datasets, where probes on those neuron activations detect hallucination about as well as probes on full residual-stream or MLP-output states.Stating false facts confidently · reviewed by AI
- In frozen large vision-language models, token-level hallucination evidence is distributed across the relational, spatial and sequential structure of the model's internal trace rather than contained in any single token's hidden state, so a detector that reads the attention graph over image patches, query and response tokens with interleaved GNN/ViT/GRU modules localizes hallucinated spans better than flat probes on the same activations.Grounding answers in the image · reviewed by AI
- In open-weight 7-8B instruct models, a supervised detector built on how hidden states change between adjacent layers separates factually wrong from correct answers better than a probe on any single layer, and shuffling the layer order destroys most of the signal — so the ordering of the depth trajectory, not just the content of one layer, carries the information.Stating false facts confidently · reviewed by AI
- In open-source vision-language models (LLaVA-1.5, Qwen-VL-Chat, Qwen2.5-VL, LLaVA-NeXT), a training-free decoding intervention that detects large layer-to-layer and step-to-step shifts in hidden states and nudges the diverging states back toward their prior value lowers object-hallucination rates, but the lower rate comes with fewer objects mentioned and shorter outputs.Grounding answers in the image · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A probe trained the same way on reused hidden states scores no better than chance, or falls clearly below external detectors of similar training budget, or its AUC fails to improve when moved from a small to a larger base model. Proposed technique, not catalogued: intrinsic hidden-state probe for hallucination risk during decoding.