multimodal-cultural-interpretation-multiple-choice-questions-pipelines-pool-several-same-backbone-evidence CLM-0619On multimodal cultural-interpretation multiple-choice questions, pipelines that pool several same-backbone evidence streams (perception, retrieval, web verification) and weight them by scored relevance beat majority voting, self-consistency and agent debate mainly on the items where the streams disagree, and the margin shrinks sharply as the backbone gets stronger.
Capability: Getting more from several models than from one
Observed on
442 expert-verified 4-way cultural MCQs; Qwen2.5-VL 3B/7B, llava-onevision-7b, GPT4.1-mini; all modules run on the same backbone; prompts and budgets fixed.
Sources
- Validated by: 4-way accuracy on 442 CM2D items against CoT, Self-Refine, CoT-SC and Agent Debate on four backbones (e.g. llava-onevision-7b 23.53 to 50.68, best baseline 43.21; GPT4.1-mini 41.63 to 55.20, best baseline 54.30), plus module ablations and a conflict/consensus split where conflict-subset accuracy is 50.0/53.4 vs 41.2/43.3 majority vote.. Measured on one 442-item self-built dataset by the framework's authors, four backbones, with module ablations and a conflict/consensus partition. [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- In multi-agent debate with 3 open-weight agents over 3 rounds, giving each agent a memory of past debates helps only if retrieval is conditioned on the current consensus ratio and peer responses are annotated with memory-derived reliability labels; randomly inserting past cases is the weakest retrieval policy and plain task-similarity retrieval trails every consensus-conditioned variant.Getting more from several models than from one · reviewed by AI
- When an 8B model verifies natural-language claims against knowledge-graph triples, scoring the evidence twice under opposing prompts (argue it supports, argue it refutes) and adjudicating between the two arguments avoids the bias toward a "Supported" verdict that a single-perspective assessment shows, and is the largest single contributor to accuracy in that pipeline.Checking claims against evidence · reviewed by AI
- On month-level egocentric recordings (20-120 days per participant) with human-written multiple-choice questions, current multimodal models degrade as the question moves from inferring recurring habits, to retrieving one specific episode, to composing evidence across days — the best model reached 71.8% macro-average against a 94.2% human baseline, with several models near or below the 25% chance level on route reasoning, cross-view spatial reasoning and direction judgement.Reasoning about time in video · reviewed by AI
- When a document image states multi-step selection constraints in narrative text and the supporting numbers sit in several charts, multimodal models degrade steadily as either the number of logical hops (2 to 5) or the number of charts (2 to 6) grows, with the best model reaching 62.83% overall against over 90% for human annotators.Carrying a chain of inference through to its conclusion · reviewed by AI
- When extracting structured rules from multimodal enterprise guideline documents, wrapping the extraction in deterministic schema validation, an LLM semantic judge, and a duplicate/contradiction consistency filter over a shared rule store lowers the share of extracted rules with no matching source sentence from about 15.7% to 3.2% relative to a one-pass pipeline using the same parser and models.Stating false facts confidently · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: low. Falsifier as drafted: On the same dataset, matched-compute majority voting over the three streams matching gated fusion on the conflict subset, or the margin over the best baseline failing to shrink with backbone strength across more models.