4-8b-multimodal-models-long-video-qa-rl-training-questions-authored CLM-0002For 4-8B multimodal models on long-video QA, RL training on questions authored from short scene segments but answered over the whole video improves benchmark accuracy more than RL on questions authored from the whole video, because segment-authored questions target details a whole-video view overlooks; including the segment timestamp in the training instruction adds a further gain by removing referential ambiguity, and is not needed at inference.
Capability: Reasoning about time in video
Observed on
4-8B open multimodal backbones (Qwen3-VL-4B/8B, MiMo-VL-7B), 10K synthesized multiple-choice samples, GRPO with a binary answer-accuracy reward, evaluated on LongVideoBench, Video-.
Sources
- Single paper. Ablation on one backbone (Qwen3-VL-4B) isolates the three factors: base 56.2/66.1 (LVB/MLVU), whole-video-authored 57.7/67.0, segment-authored without timestamp 60.6/69.3, full S2V 62.3/71.9. Gains hold across three backbones and three benchmarks, but the causal ablation is on one model only, and data synthesis used a much larger teacher (Qwen3-VL-235B), so distillation from the teacher is not separated from the segment-authoring mechanism.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For 7-8B multimodal models answering multiple-choice long-video questions under a fixed 16-frame budget, selecting keyframes by embedding similarity to the query concatenated with each candidate answer beats query-only scoring, query-agnostic diversity selection, and uniform sampling across Qwen2-VL, LLaVA-Video and InternVL3.5, with the largest margins on the longest video splits — but per-category results still invert, including uniform sampling winning some categories.Reasoning about time in video · reviewed by AI
- For long-video QA under a fixed frame budget with a frozen 7B multimodal model, organizing frames into a question-agnostic hierarchical temporal tree and searching it coarse-to-fine (answer, then issue a search statement, then expand the relevant branch) raises accuracy over uniform sampling and over agent methods that score frames independently, with the gain largest on the longest-video benchmarks.Reasoning about time in video · reviewed by AI
- For 7B multimodal models on long-video QA, a multi-turn agentic exploration harness (adaptive viewing speed, cross-turn textual notes, rewatch) lowers accuracy below plain uniform-frame single-pass answering unless the model is trained to use it, because the untrained model commits to an answer before gathering evidence; with cold-start SFT on verified exploration turns plus trajectory-level RL, the same harness beats the single-pass baseline.Reasoning about time in video · reviewed by AI
- For 2B-8B video multimodal models fine-tuned for temporal reasoning, placing a small minority of reversed-video rollouts in each GRPO group (1 reversed out of 7) beats both all-forward GRPO and larger reversed shares (2 or 3 of 7), because more reversed samples dilute the main forward-video answering objective.Reasoning about time in video · reviewed by AI
- For VLM-based judging of camera-conditioned world-model videos, splitting a long video pair into action-aligned chunks with structured visual evidence (source image, frame-grid overview, per-action first/last-frame panels) and voting chunk verdicts into video-level preferences agrees with human preferences better than judging the whole video pair at once, because short-lived action evidence is otherwise diluted by the full context.Judging another's work as well as a careful reviewer would · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: An ablation where whole-video-authored VQA (V2V) at the same data scale and reward matches or beats segment-authored VQA, or where removing segment timestamps from training instructions makes no difference to final accuracy. Proposed technique, not catalogued: segment-authored questions trained on whole video.