open-source-vision-language-models-llava-15-qwen-vl-chat-qwen25-vl-llava-next-training-free-decodingIn open-source vision-language models (LLaVA-1.5, Qwen-VL-Chat, Qwen2.5-VL, LLaVA-NeXT), a training-free decoding intervention that detects large layer-to-layer and step-to-step shifts in hidden states and nudges the diverging states back toward their prior value lowers object-hallucination rates, but the lower rate comes with fewer objects mentioned and shorter outputs.
Ingested from a paper but not yet reviewed by a human. It is deliberately inert: it does not move any technique’s standing, does not count toward the backtest, and is excluded anywhere a claim would carry weight. Read the source before relying on it.
Capability: Grounding answers in the image
Observed on
Open-source image-input LVLMs at 3B-7B scale, standard hallucination benchmarks (POPE, CHAIR, HallusionBench); not tested on closed models, video, or multi-image input..
Sources
- Measured on nine benchmarks across six backbones against vanilla decoding and training-free baselines (VCD, ICD, MemVR, OPERA). CHAIR shows Cs 47.6->28.2 and Ci 13.3->7.3 on LLaVA-1.5 with recall 80.6->72.0 and length 99.7->88.9. Authors report a length-normalized analysis in an appendix not included here, so the informativeness confound is claimed to be addressed but not verifiable from the given text. Single group, fixed thresholds, no seed variance reported.
Disagreeing is the most useful thing you can do here. Both sides of every contested claim in this catalog were assembled by the same person, which is its weakest point.
Related claims
- For training-free inference-time hallucination interventions in 7B-class vision-language models, increasing intervention strength (finer regional evidence partitioning, longer attention-routing windows, stronger textual perturbation) keeps lowering CHAIR object-hallucination scores while producing empty or reduced-coverage responses, so lower hallucination numbers at high intervention strength partly reflect the model saying less rather than grounding better.Grounding answers in the image · unreviewed
- For training-free inference-time hallucination mitigation in 7B vision- language models, reported gains on object-hallucination benchmarks are largely inseparable from reduced informativeness: hallucination rate and object recall/coverage move together, so a lower score can mean the model mentioned fewer visual entities rather than grounded better.Whether the measurement made the finding · unreviewed
- In 7B vision-language models that prune visual tokens during decoding for efficiency, hallucination rises as the generated caption gets longer, because pruning is irreversible and discards fine-grained evidence needed at later decoding steps; keeping the pruned tokens as a retrievable bank and pulling them back when next-token entropy spikes recovers the grounding without a second forward pass.Grounding answers in the image · unreviewed
- For short-answer object-existence questions in open multimodal models (LLaVA v1.5, MiniGPT-4, Qwen2.5-VL), object hallucination is driven partly by the visual encoder itself rather than only by language priors: hallucinated cases show lower image-text embedding similarity and inverted attention, and swapping in a weaker or noisier visual encoder lowers POPE accuracy while a stronger one raises it.Grounding answers in the image · unreviewed
- A ~2M-parameter classifier head that reads hidden states already computed during generation (tapping shallow, middle and deep layers of the generating model) detects response-level hallucination at least as well as specialized hidden-state hallucination detectors, and its detection quality rises with the scale of the base model it is attached to.Stating false facts confidently · unreviewed
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A replication where the same compensation leaves CHAIR recall and response length unchanged while hallucination rates still fall, or where hallucination rates do not fall at all on a held-out backbone. Drafted stance toward training-free-inference-time-hallucination-mitigation-7b-vision-language-models-reported-gains: supports -- On CHAIR the method cuts sentence- and instance-level hallucination rates while recall and average response length both fall, the same coupling of lower hallucination with reduced informativeness. Proposed technique, not catalogued: Hidden-state divergence compensation at decoding time.