training-free-inference-time-hallucination-interventions-7b-class-vision-language-models-increasing-interventionFor training-free inference-time hallucination interventions in 7B-class vision-language models, increasing intervention strength (finer regional evidence partitioning, longer attention-routing windows, stronger textual perturbation) keeps lowering CHAIR object-hallucination scores while producing empty or reduced-coverage responses, so lower hallucination numbers at high intervention strength partly reflect the model saying less rather than grounding better.
Ingested from a paper but not yet reviewed by a human. It is deliberately inert: it does not move any technique’s standing, does not count toward the backtest, and is excluded anywhere a claim would carry weight. Read the source before relying on it.
Capability: Grounding answers in the image
Observed on
LLaVA-1.5-7B, InstructBLIP-Vicuna-7B and Qwen3-VL-4B with a training-free decoding-time intervention, measured on POPE, CHAIR and MMHal-Bench with object recall and response length.
Sources
- Ablation on LLaVA-1.5-7B over REA slot counts 1/4/9/16/25 and routing lengths 4-12 layers, with recall, response length and empty-response counts reported. Main tables also report recall and length, and at the chosen 3x3 setting recall is preserved, so the confound is a property of the strength dial, not of the method at default. Single paper, no independent replication.
Disagreeing is the most useful thing you can do here. Both sides of every contested claim in this catalog were assembled by the same person, which is its weakest point.
Related claims
- In open-source vision-language models (LLaVA-1.5, Qwen-VL-Chat, Qwen2.5-VL, LLaVA-NeXT), a training-free decoding intervention that detects large layer-to-layer and step-to-step shifts in hidden states and nudges the diverging states back toward their prior value lowers object-hallucination rates, but the lower rate comes with fewer objects mentioned and shorter outputs.Grounding answers in the image · unreviewed
- For training-free inference-time hallucination mitigation in 7B vision- language models, reported gains on object-hallucination benchmarks are largely inseparable from reduced informativeness: hallucination rate and object recall/coverage move together, so a lower score can mean the model mentioned fewer visual entities rather than grounded better.Whether the measurement made the finding · unreviewed
- For short-answer object-existence questions in open multimodal models (LLaVA v1.5, MiniGPT-4, Qwen2.5-VL), object hallucination is driven partly by the visual encoder itself rather than only by language priors: hallucinated cases show lower image-text embedding similarity and inverted attention, and swapping in a weaker or noisier visual encoder lowers POPE accuracy while a stronger one raises it.Grounding answers in the image · unreviewed
- In LLaVA-1.5-7B captioning, restricting a LoRA adapter to attention heads picked by a hallucination diagnosis (attention-to-image drop around hallucinated object words, then ablation screening) reduces object hallucination, while an identically trained layer-matched random-head adapter does not — so the selection, not the added adapter capacity, carries the effect.Grounding answers in the image · unreviewed
- In 7-8B vision-language models, fine-tuning visual and text embeddings toward perturbation-averaged and ground-truth anchors before cross-modal contrastive alignment lowers object and attribute hallucination rates on captioning and existence benchmarks without reducing object coverage or general VQA accuracy.Grounding answers in the image · unreviewed
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: An ablation sweeping intervention strength on the same backbones and benchmarks in which hallucination metrics fall while object recall and response length hold constant at every strength setting. Drafted stance toward training-free-inference-time-hallucination-mitigation-7b-vision-language-models-reported-gains: supports -- The paper's own ablation shows 16- and 25-slot configurations reach lower CHAIRi while producing 8 and 25 empty responses and recall dropping from 77.38 to 71.48 and 54.98, and the authors say the lower hallucination scores are partly due to reduced output coverage. Proposed technique, not catalogued: Regional visual evidence anchoring with decision-conditioned attention routing.