streaming-video-question-answering-frozen-video-llm-appending-retrieved-historicalFor streaming video question answering with a frozen video-LLM, appending retrieved historical frames as extra visual context helps less than optimizing a small set of latent memory tokens at test time and then dropping the retrieved tokens before decoding; the gain is largest on questions requiring backward tracing over past history.
Ingested from a paper but not yet reviewed by a human. It is deliberately inert: it does not move any technique’s standing, does not count toward the backtest, and is excluded anywhere a claim would carry weight. Read the source before relying on it.
Capability: Reasoning about time in video
Observed on
Frozen Qwen2.5-VL 3B/7B backbones, streaming and long-video benchmarks with a bounded memory budget; test-time optimization only, no parameter updates..
Sources
- Ablation on OVO-Bench/VideoMME: retrieved-evidence-appended 59.7/65.6 vs evolved latent tokens 64.2/66.6, unoptimized latent tokens 56.5/65.2, baseline 56.9/65.4. Backward Tracing rose 44.7 to 60.0 while real-time perception rose 63.3 to 68.5. Single paper, authors' own system, two backbones; does not isolate which of the three components carries the effect beyond the reported staged ablation.
Disagreeing is the most useful thing you can do here. Both sides of every contested claim in this catalog were assembled by the same person, which is its weakest point.
Related claims
- In 7B vision-language models that prune visual tokens during decoding for efficiency, hallucination rises as the generated caption gets longer, because pruning is irreversible and discards fine-grained evidence needed at later decoding steps; keeping the pruned tokens as a retrievable bank and pulling them back when next-token entropy spikes recovers the grounding without a second forward pass.Grounding answers in the image · unreviewed
- When the preference needed to answer a query lies in a different concept space from the cues in a user's dialogue history (e.g. inferring an emotion-regulation strategy from personality cues), retrieval-based and profile-summarization personalization methods barely beat a no-history baseline, and dumping the full history into the prompt does worse than no history at all.Remembering across sessions · unreviewed
- In open-source vision-language models (LLaVA-1.5, Qwen-VL-Chat, Qwen2.5-VL, LLaVA-NeXT), a training-free decoding intervention that detects large layer-to-layer and step-to-step shifts in hidden states and nudges the diverging states back toward their prior value lowers object-hallucination rates, but the lower rate comes with fewer objects mentioned and shorter outputs.Grounding answers in the image · unreviewed
- In diffusion vision-language models generating free-form captions, object tokens that are hallucinated commit at later denoising steps and with lower per-token confidence than grounded object tokens, so the commit-step trajectory is an available internal hallucination signal that has no counterpart in autoregressive decoding.Grounding answers in the image · unreviewed
- For short-answer object-existence questions in open multimodal models (LLaVA v1.5, MiniGPT-4, Qwen2.5-VL), object hallucination is driven partly by the visual encoder itself rather than only by language priors: hallucinated cases show lower image-text embedding similarity and inverted attention, and swapping in a weaker or noisier visual encoder lowers POPE accuracy while a stronger one raises it.Grounding answers in the image · unreviewed
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: low. Falsifier as drafted: An ablation on the same benchmarks where appending the retrieved evidence directly matches or beats the evolved latent memory tokens, or where the backward-tracing gain does not exceed the real-time perception gain. Proposed technique, not catalogued: latent working memory internalization for streaming video.