post-training-pretrained-vision-language-action-manipulation-policy-under-fixed-per-round-frame CLM-0466mechanismsingle paperReviewed by AI
For post-training a pretrained vision-language-action manipulation policy under a fixed per-round frame budget, collecting handheld demonstrations only at states where the human action chunk diverges from the policy's sampled action distribution, and weighting them by a learned task-progress advantage, raises task progress more than collecting the same number of frames of ordinary demonstrations and fine-tuning on them.
Capability: Turning plans into physical action
Observed on
Four real-world Franka tasks (two long-horizon, two precision) with a pi-0.5 policy adapted per task; 10 trials per checkpoint; partial-credit task progress score; policy predictio.
Sources
- Validated by: Mean task progress score across rounds on four real tasks versus supervised fine-tuning on equal-budget ordinary UMI demonstrations, plus an ablation removing the advantage model and a comparison to real-robot HG-DAgger on Clean Up Table (higher score, 73.4 vs 412.99 ms per collected frame).. Measured on four tasks, 10 trials per checkpoint, one policy backbone, one lab, one robot; ablation isolates the advantage component and a threshold sweep on one task shows insensitivity, but the comparison is against a single SFT baseline and evaluation counts are small.
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- When adapting a pretrained VLA manipulation policy with only a handful of target demonstrations, retrieving prior demonstrations at the level of LLM-decomposed subtasks (then reranking by behavior-feature similarity) raises success more than retrieval by visual, optical-flow, or state-action similarity, and the margin is concentrated on long-horizon composite tasks rather than single-skill pick-and-place.Turning plans into physical action · reviewed by AI
- When a pretrained vision-language-action policy is fine-tuned sequentially on a stream of real-robot manipulation tasks, the choice of action-normalization statistics is itself a determinant of continual-learning success: statistics that are task-derived, first-task-derived, or recomputed each stage produce forgetting or poor transfer, while statistics estimated once from a small task-independent calibration set covering the embodiment's motion range and then frozen give the best acquisition, retention and task-order stability.Turning plans into physical action · reviewed by AI
- For real-world online RL post-training of large pretrained vision-language-action policies on high-precision manipulation, optimizing the action head toward a paired advantage margin against a frozen reference action, rather than maximizing absolute Q values, yields higher autonomous success on long multi-step tasks, because absolute-Q maximization amplifies early optimistic value errors into policy drift that behavior-cloning regularization alone does not stop.Turning plans into physical action · reviewed by AI
- For vision-language-action models pretrained on multi-embodiment video, latent actions learned only from RGB frame pairs miss finger-level end-effector articulation, and adding a shared discrete latent over 3D end-effector point-cloud transitions improves fine-grained motion probing and downstream manipulation success, but only when the geometric latent is forced to encode motion rather than morphology (shared bimanual code, pair-consistent augmentation, bidirectional transitions).Turning plans into physical action · reviewed by AI
- For manipulation policies whose large VLA backbone cannot run within the control period, letting the slow model propose action chunks from a stale observation while a small RL-trained edit policy adjusts and Q-selects among those chunks using the latest observation recovers more performance than applying RL on top of asynchronous imitation-learning schemes alone, and the gap widens as inference delay or environment speed grows.Turning plans into physical action · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: low. Falsifier as drafted: Under matched frame budgets on comparable manipulation tasks, ordinary demonstration collection plus fine-tuning matches or beats disagreement-targeted collection with advantage conditioning, or the gain disappears once the advantage model is given to both arms.