panoramic-dental-x-ray-vqa-frozen-vision-language-models-replacing-model CLM-0216For panoramic dental X-ray VQA with frozen vision-language models, replacing the model's own spatial judgement with external detector findings that are deterministically bound to tooth identities before synthesis raises answer accuracy over the same two-pass pipeline without detectors, and the resulting accuracy is bounded by detector quality — degrading the identity/attribution detector costs more downstream accuracy than a much larger degradation of a task-specific pathology detector.
Capability: Grounding answers in the image
Observed on
Panoramic dental radiographs (MMOral-OPG-Bench, 100 images, 491 closed / 578 open questions); four instruction-following VLM backbones plus Qwen2.5-VL-7B; synthesis prompted to pri.
Sources
- Matched-pipeline ablation: same VLM, prompts, two-pass structure, detectors removed. Closed-ended overall +11.8 to +23.7 points, open-ended +14.0 to +22.4 points across four backbones, consistent under two LLM judges; 43 of 44 category comparisons favor tools. Tool-quality ablation on one backbone (mistral-large-3): FDI 92.4->88.1 mAP@50 cost 3.3 mean points, dental-history 87.6->59.0 cost 2.7. Weakest category is closed-ended Jaw, where a low-precision bone-loss detector propagates false positives. Single run per config; no repeated-run variance; [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For 7-13B multimodal models, verifying each object claim with a frozen open-vocabulary detector lowers object hallucination only if evidence is pooled across semantically equivalent phrasings and checked for spatial agreement (the preferred boxes of different queries overlapping on one instance); a single-query maximum score is a weaker signal, and shuffling the spatial evidence across samples drives its discrimination to chance.Grounding answers in the image · reviewed by AI
- In frozen large vision-language models, token-level hallucination evidence is distributed across the relational, spatial and sequential structure of the model's internal trace rather than contained in any single token's hidden state, so a detector that reads the attention graph over image patches, query and response tokens with interleaved GNN/ViT/GRU modules localizes hallucinated spans better than flat probes on the same activations.Grounding answers in the image · reviewed by AI
- For detecting findings-conclusion inconsistencies in oncology PET/CT radiology reports, a 15M-parameter BERT pre-trained on in-domain reports reaches higher balanced accuracy and lower false-positive rate than zero- or few-shot prompted 27B-70B open-weight LLMs, and LoRA fine-tuning the 70B model only matches the small model rather than beating it.Spotting an error in a given artifact · reviewed by AI
- In claim-level hallucination detection pipelines over image, video and audio inputs (both comprehension and generation directions), decomposing an output into atomic claims contributes more to detection accuracy than ensembling several verifier models: removing decomposition costs about 6.6-7.9 macro-F1 across six cross-modal tasks while removing multi-expert majority voting costs about 4.5-5.3.Stating false facts confidently · reviewed by AI
- On long-video (10-150 minute) audio-visual reasoning questions, multiple-choice accuracy overstates grounding: a large open model can answer about 55% of items correctly while scoring 8.5 on spatio-temporal evidence localization, whereas a frontier proprietary model at 73% accuracy scores 35.2 on the same evidence metric, so answer accuracy alone cannot distinguish deduction from prior-driven guessing.Reasoning about time in video · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A matched comparison where adding deterministically ID-bound detector evidence fails to beat the detector-free pipeline, or where weakening the identity/grounding detector by a few mAP points costs less downstream accuracy than a far larger degradation of a task-specific detector. Proposed technique, not catalogued: Deterministic grounding of expert-detector findings to entity IDs before VLM synthesis.