training-free-contrastive-decoding-7b-class-medical-vlms-spatial-temporal-contrast CLM-0649In training-free contrastive decoding for 7B-class medical VLMs, the spatial and temporal contrast terms act on different answer types: anatomy-conditioned classifier-free guidance is what raises closed-ended accuracy (68.5 to 81.5 on SLAKE with Phi-3.5V-Med) while step-wise contrast against the previous step's logits is what preserves open-ended recall (37.8 to 44.9), so dropping either term costs about five overall points.
Capability: Grounding answers in the image
Observed on
Three Med-VLMs (Phi-3.5V-Med, HuatuoGPT-Vision-7B, LLaVA-Med-7B) on SLAKE and MIMIC-CXR Med-VQA; ablation reported only for Phi-3.5V-Med on SLAKE..
Sources
- Measured, one ablation figure on one model and one dataset (SLAKE, Phi-3.5V-Med); the full-vs-baseline comparison spans three models and two datasets. The decomposition into spatial-for-closed and temporal-for-open rests on a single ablation, and the paper does not test whether the split holds on MIMIC-CXR or on the other two backbones. Hyperparameters (gamma, attention weights) were grid-searched, and the search set relative to the test split is not described.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- In region-guided contrastive decoding for medical VLMs, the effectiveness of the guidance depends on how much of the image the guiding mask covers: coarse ground-truth boxes (mean ~39% coverage on SLAKE) suppress so much of the unconditional branch that the contrast weakens, while compact automatically discovered masks (~7% coverage) selected by the drop in answer likelihood under occlusion give a sharper signal and higher closed-ended accuracy.Grounding answers in the image · reviewed by AI
- In 4-7B open vision-language models (LLaVA-1.5-7B, Qwen2.5-VL-7B, InternVL-2.5-4B), how the cross-modal attention distribution over visual tokens shifts between adjacent decoder layers separates hallucinated from grounded object predictions better than the model's sensitivity to masking the attended region, and better than attention drift measured against a fixed first or last layer anchor.Grounding answers in the image · reviewed by AI
- For binary visual question answering with open LVLMs (InstructBLIP-7B, Qwen2.5-VL, InternVL-4B), a hallucinated answer can be held with high response consistency, so a single consistency/uncertainty score misses it; separating consistency into four probe groups (image vs. text perturbation crossed with affirmative vs. negated statement) and classifying on the joint within- and between-group agreement pattern detects hallucination better than any single-score black-box or white-box baseline.Stating false facts confidently · reviewed by AI
- In 7B open vision-language models (LLaVA-1.5, Shikra), the per-token log-probability gap between a normal forward pass and one with visual features zeroed shrinks as a caption gets longer, and while that gap ranks hallucinated tokens slightly better than entropy or max-probability, its absolute separation is weak (AUROC around 0.57-0.59) and it fails entirely when aggregated to sentence level.Grounding answers in the image · reviewed by AI
- When vision-language models are given an explicit abstention option plus a stated payoff for correct, wrong and declined answers, quadrupling the penalty for a wrong answer barely changes how often they answer, so their abstention rate is largely insensitive to the stated cost of error across text, synthetic video and natural video recall tasks.Saying it does not know · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: low. Falsifier as drafted: An ablation on other Med-VLMs or datasets where removing the temporal contrast leaves open-ended recall unchanged, or where removing spatial CFG leaves closed-ended accuracy unchanged, would show the two terms are not acting on separate failure types. Automatic check flagged: figures not in the source: 81.5.