long-video-qa-under-fixed-frame-budget-frozen-7b-multimodal CLM-0606observationsingle paperReviewed by AI
For long-video QA under a fixed frame budget with a frozen 7B multimodal model, organizing frames into a question-agnostic hierarchical temporal tree and searching it coarse-to-fine (answer, then issue a search statement, then expand the relevant branch) raises accuracy over uniform sampling and over agent methods that score frames independently, with the gain largest on the longest-video benchmarks.
Capability: Reasoning about time in video
Observed on
Training-free, Qwen2.5-VL-7B, same frame budget as the baselines; evaluated on VideoMME, LongVideoBench, LVBench..
Sources
- Validated by: Accuracy on VideoMME, LongVideoBench and LVBench with Qwen2.5-VL-7B at matched frame budget: +0.5%, +4.6%, +4.4% over the baseline.. Measured on three long-video QA benchmarks with one 7B backbone; gains are small on VideoMME (0.5%) and larger on the two longer-video benchmarks. No ablation reported in the source text isolating the tree construction from the answer-retrieve-explore loop, and only one backbone is shown.
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For long-video QA under a fixed 32-frame budget with 7B-8B multimodal backbones, selecting frames from the structural shape of the frame-query similarity curve (peak, rising, falling, transition regions) beats ranking frames by similarity magnitude, and the margin grows with video length — a few points on short clips and roughly 8 points on hour-long videos — while on short/medium-clip benchmarks it is near-indistinguishable from other selectors.Reasoning about time in video · reviewed by AI
- For 7-8B multimodal models answering multiple-choice long-video questions under a fixed 16-frame budget, selecting keyframes by embedding similarity to the query concatenated with each candidate answer beats query-only scoring, query-agnostic diversity selection, and uniform sampling across Qwen2-VL, LLaVA-Video and InternVL3.5, with the largest margins on the longest video splits — but per-category results still invert, including uniform sampling winning some categories.Reasoning about time in video · reviewed by AI
- For 4-8B multimodal models on long-video QA, RL training on questions authored from short scene segments but answered over the whole video improves benchmark accuracy more than RL on questions authored from the whole video, because segment-authored questions target details a whole-video view overlooks; including the segment timestamp in the training instruction adds a further gain by removing referential ambiguity, and is not needed at inference.Reasoning about time in video · reviewed by AI
- In training-free video temporal grounding with frozen vision-language embedding backbones, matching a query only against frames sampled at a fixed rate leaves temporal context unused: adding clip-level (sliding-window) similarities projected onto the frame timeline raises localization accuracy even though clip-only similarity is worse than frame-only on its own.Reasoning about time in video · reviewed by AI
- On multi-hop video questions written so no single frame or timestamp lookup suffices, single-turn MLLMs answer poorly (0.07-0.51 accuracy across 12 proprietary and open models), and wrapping the same models in an agentic loop helps only when the loop lets the agent choose what, where and at what granularity to look: frameworks giving adaptive fps/resolution/interval control raised accuracy and trajectory scores for nearly every model, while a fixed-schedule framework with weak perception tools (STAR) fell below the single-turn baseline for all of them.Reasoning about time in video · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: At a matched frame budget, the tree search fails to beat uniform sampling or independent-frame agent selection on long-video QA, or the gain disappears once frame count is equalized across methods.