memory-augmented-vla-manipulation-policies-must-actively-move-viewpoint-find CLM-0440For memory-augmented VLA manipulation policies that must actively move the viewpoint to find task-relevant evidence, writing one memory frame at each fixed action-chunk boundary generalizes better than event-triggered writing (on predicted subtask change or on gripper/head/torso motion), and extra memory capacity helps only under the regular schedule — event-triggered policies degrade further out of distribution because mistimed triggers omit informative views and control jitter fills memory with redundant frames.
Capability: Turning plans into physical action
Observed on
35 simulated active-perception manipulation tasks on a dual-arm humanoid with controllable head/torso, 100 training trajectories per task, ID and OOD (new backgrounds, denser clutt.
Sources
- Validated by: Task success rate across 35 tasks: chunk writing 62.97 ID / 47.60 OOD at 12 frames vs subtask writing 28.11 / 16.00 and motion writing 32.80 / 22.69; memory-free baseline 27.37 / 6.12. Capacity 6 to 12 helped chunk writing but hurt motion writing in most settings.. Measured: six write-policy/capacity variants plus a memory-free baseline, 50 seeds per task per setting, all other components held fixed (same instruction, 18-D proprioception, OFT head, subtask supervision). [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For frozen VLA policies executing repetitive, perceptually aliased manipulation tasks (counting pick-and-place), a progress memory that advances after each attempted action converts transient grasp or placement failures into persistent task-state errors, so writes must be gated on observed physical evidence of the intended transition; gating this way raised success far above the frozen policy and above memory-augmented baselines.Turning plans into physical action · reviewed by AI
- For manipulation policies whose large VLA backbone cannot run within the control period, letting the slow model propose action chunks from a stale observation while a small RL-trained edit policy adjusts and Q-selects among those chunks using the latest observation recovers more performance than applying RL on top of asynchronous imitation-learning schemes alone, and the gap widens as inference delay or environment speed grows.Turning plans into physical action · reviewed by AI
- When adapting a pretrained VLA manipulation policy with only a handful of target demonstrations, retrieving prior demonstrations at the level of LLM-decomposed subtasks (then reranking by behavior-feature similarity) raises success more than retrieval by visual, optical-flow, or state-action similarity, and the margin is concentrated on long-horizon composite tasks rather than single-skill pick-and-place.Turning plans into physical action · reviewed by AI
- For post-training a pretrained vision-language-action manipulation policy under a fixed per-round frame budget, collecting handheld demonstrations only at states where the human action chunk diverges from the policy's sampled action distribution, and weighting them by a learned task-progress advantage, raises task progress more than collecting the same number of frames of ordinary demonstrations and fine-tuning on them.Turning plans into physical action · reviewed by AI
- For manipulation policies (diffusion-policy and VLA backbones) trained on few demonstrations, decomposing trajectories implicitly via a Mixture-of-Experts gate and storing the per-segment gating profiles as retrievable memory, then blending retrieved profiles into the current gating distribution at inference, raises success most on unseen object-receptacle recombinations rather than on in-distribution tasks.Turning plans into physical action · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: On the same or comparable active-perception manipulation benchmarks, an event-triggered write policy (with stable triggers) matching or beating fixed chunk-boundary writing OOD, or increasing capacity improving event-triggered variants as much as chunk-based ones.