multimodal-model-asked-object-existence-question-over-set-images-once CLM-0104When a multimodal model is asked an object-existence question over a set of images at once, it is less accurate than when the same images are queried one at a time and the answers combined, so cross-image integration adds errors beyond the per-image perception errors.
Capability: Grounding answers in the image
Observed on
Object-centric questions (existence, counting, attribute, position) over sets of 2-10 natural images; shown for existence in a controlled decomposition ablation across 29 MLLMs inc.
Sources
- Ablation on the existence task comparing comprehensive multi-image questions against per-image decomposed questions over identical content; the paper reports the single-image route wins consistently across all models and scales, but the gap size is only shown in a figure. Overall benchmark average was 36.1% across 29 models; accuracy fell as image count rose (existence 62.4% easy to 30.0% at larger image counts).
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For 4-8B multimodal models on long-video QA, RL training on questions authored from short scene segments but answered over the whole video improves benchmark accuracy more than RL on questions authored from the whole video, because segment-authored questions target details a whole-video view overlooks; including the segment timestamp in the training instruction adds a further gain by removing referential ambiguity, and is not needed at inference.Reasoning about time in video · reviewed by AI
- On multiple-choice audio questions where each answer choice is paired with a generated image, adding the images helps native multimodal models most on environmental-sound questions and not at all on speech (speech is best audio-only), and in cascaded pipelines that convert audio to transcripts or captions first the images lower average accuracy.Understanding speech and audio · reviewed by AI
- For 7-8B multimodal models answering multiple-choice long-video questions under a fixed 16-frame budget, selecting keyframes by embedding similarity to the query concatenated with each candidate answer beats query-only scoring, query-agnostic diversity selection, and uniform sampling across Qwen2-VL, LLaVA-Video and InternVL3.5, with the largest margins on the longest video splits — but per-category results still invert, including uniform sampling winning some categories.Reasoning about time in video · reviewed by AI
- When routing each image to one of several domain-specialist vision backbones trained on a shared label space, a prompted MLLM router (Gemma-4-12B) reaches nearly the accuracy of a trained neural-network router and beats majority-vote fusion, because routing only needs coarse domain identification, which a general multimodal model already does zero-shot.Getting more from several models than from one · reviewed by AI
- When a document image states multi-step selection constraints in narrative text and the supporting numbers sit in several charts, multimodal models degrade steadily as either the number of logical hops (2 to 5) or the number of charts (2 to 6) grows, with the best model reaching 62.83% overall against over 90% for human annotators.Carrying a chain of inference through to its conclusion · reviewed by AI
Notes
Ingested unreviewed on 2026-09-07 and deliberately inert until a human endorses it: it does not move a technique standing, does not count toward the backtest, and is excluded anywhere a claim would carry weight. Drafted confidence: medium. Falsifier as drafted: A model whose accuracy on joint multi-image existence questions matches or exceeds the combined per-image decomposed queries on the same images. Proposed technique, not yet catalogued: Decompose multi-image questions into per- image queries.