manipulation-policies-whose-large-vla-backbone-cannot-run-within CLM-0437mechanismsingle paperReviewed by AI
For manipulation policies whose large VLA backbone cannot run within the control period, letting the slow model propose action chunks from a stale observation while a small RL-trained edit policy adjusts and Q-selects among those chunks using the latest observation recovers more performance than applying RL on top of asynchronous imitation-learning schemes alone, and the gap widens as inference delay or environment speed grows.
Capability: Turning plans into physical action
Observed on
Dynamic, reactive manipulation at ~30 Hz with sparse binary rewards, small offline demo set, delay no longer than the chunk execution horizon; shown on Kinetix sim tasks and four r.
Sources
- Validated by: Success rate against RLPD, DSRL, DSRL+RTC, EXPO-FT, EXPO-FT+RTC and SFT baselines: best in 10/10 Kinetix environments (96.2% average under 4-step delay, above the 81.4% zero-delay RLPD baseline) and 29/30 average over four real tasks versus 25/30 for the strongest baseline, plus delay and speed sweeps.. Measured on 10 Kinetix tasks (4 seeds) and 4 real robot tasks at 30 trials each, against six baselines including two RTC-augmented ones. Baselines were given more gradient updates, which the authors say favors them. [truncated]
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- When adapting a pretrained VLA manipulation policy with only a handful of target demonstrations, retrieving prior demonstrations at the level of LLM-decomposed subtasks (then reranking by behavior-feature similarity) raises success more than retrieval by visual, optical-flow, or state-action similarity, and the margin is concentrated on long-horizon composite tasks rather than single-skill pick-and-place.Turning plans into physical action · reviewed by AI
- For frozen VLA policies executing repetitive, perceptually aliased manipulation tasks (counting pick-and-place), a progress memory that advances after each attempted action converts transient grasp or placement failures into persistent task-state errors, so writes must be gated on observed physical evidence of the intended transition; gating this way raised success far above the frozen policy and above memory-augmented baselines.Turning plans into physical action · reviewed by AI
- For real-world online RL post-training of large pretrained vision-language-action policies on high-precision manipulation, optimizing the action head toward a paired advantage margin against a frozen reference action, rather than maximizing absolute Q values, yields higher autonomous success on long multi-step tasks, because absolute-Q maximization amplifies early optimistic value errors into policy drift that behavior-cloning regularization alone does not stop.Turning plans into physical action · reviewed by AI
- For a small agent policy (Qwen3.5-4B) repeatedly deployed on the same task family with environment-level feedback, distilling only the trajectories whose textual-state guidance is newly or persistently useful relative to state-free execution raises the state-free policy's accuracy more than distilling all state-conditioned trajectories, and the resulting policy keeps improving over alternating rounds of state editing and consolidation.Improving its own scaffolding from failures · reviewed by AI
- For memory-augmented VLA manipulation policies that must actively move the viewpoint to find task-relevant evidence, writing one memory frame at each fixed action-chunk boundary generalizes better than event-triggered writing (on predicted subtask change or on gripper/head/torso motion), and extra memory capacity helps only under the regular schedule — event-triggered policies degrade further out of distribution because mistimed triggers omit informative views and control jitter fills memory with redundant frames.Turning plans into physical action · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A latency-matched comparison where RL fine-tuning on an asynchronous imitation policy (e.g. training-time RTC) matches or beats the slow-propose/fast-edit split, or where the edit policy's advantage disappears as delay grows.