adding-image-level-preference-pairs-multimodal-dpo-llava-1-5-7b CLM-0015When adding image-level preference pairs to multimodal DPO for LLaVA-1.5 7B/13B, constructing the rejected image by selecting, among many equal-size random masks, the variant with lowest CLIP similarity to the original lowers object-hallucination rates more than coarse alternatives (blockwise masking, rotation, cropping, or a fully blacked-out image), at the cost of slightly lower object coverage.
Capability: Grounding answers in the image
Observed on
LLaVA-v1.5 7B and 13B fine-tuned on 22K RLAIF-V preference instances; hallucination measured on MMHalBench, Object HalBench, AMBER.
Sources
- One ablation table comparing five rejected-image strategies on MMHalBench and AMBER, single backbone (LLaVA-7B), single seed, no variance reported; recall check on Object HalBench against two baselines argues the CHAIR gain is not purely shorter outputs, though coverage on AMBER does fall relative to the base model.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- In LLaVA-1.5-7B captioning, restricting a LoRA adapter to attention heads picked by a hallucination diagnosis (attention-to-image drop around hallucinated object words, then ablation screening) reduces object hallucination, while an identically trained layer-matched random-head adapter does not — so the selection, not the added adapter capacity, carries the effect.Grounding answers in the image · reviewed by AI
- In LLaVA-1.5-7B captioning, restricting a LoRA adapter to attention heads picked by a hallucination diagnosis (attention-to-image drop around hallucinated object words, then ablation screening) reduces object hallucination, while an identically trained layer-matched random-head adapter does not — so the selection, not the added adapter capacity, carries the effect.Grounding answers in the image · reviewed by AI
- In RL-trained multimodal image-quality models, supervising reasoning with the rating reward alone raises rating alignment while making the stated reasoning less faithful — rating-only training reached an average PLCC/SRCC of 0.836/0.815 yet its reasoning-guided edits lowered judged image quality (mean gain -0.260, worse than the untrained baseline's -0.136).Saying what actually drove the answer · reviewed by AI
- In 7B-32B vision-language models, applying a bounded decoding-time logit penalty to candidate tokens whose rank-pooled visual-state evidence falls below a per-token reference estimated offline from unlabeled images lowers object-hallucination rates (CHAIR_I down up to 21.3% on four 7B backbones) while largely preserving descriptive and general multimodal scores, at about 1.33% added per-token latency once image evidence is cached at prefill.Grounding answers in the image · reviewed by AI
- For multimodal LLMs, a behavioral perturbation signature (answer/confidence change under visual degradation, blank-image replacement, and grounding/relation probes) predicts which failure family an error belongs to far better than scalar confidence, yet folding the same signature into confidence as a direct penalty makes answer-versus-abstain ranking worse than scalar confidence alone.Saying it does not know · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: low. Falsifier as drafted: An ablation on the same backbones and data where blackness, blockwise, rotation or crop rejected images match or beat the CLIP-min-similarity selection on CHAIR and AMBER hallucination rates. Proposed technique, not catalogued: CLIP-selected context-removed rejected images for multimodal DPO.