vla-world-action-manipulation-policies-score-near-ceiling-standard-libero-recovery CLM-0250observationsingle paperReviewed by AI
For VLA and world-action manipulation policies that score near-ceiling on standard LIBERO, recovery success after a real execution failure falls sharply as recovery requires deeper state reasoning \u2014 retrying or adapting the immediate action works far more often than restoring a displaced task object or clearing an environmental obstruction \u2014 and rankings on the standard benchmark do not predict recovery rankings.
Capability: Turning plans into physical action
Observed on
Six VLA/WAM policies (pi0, pi0-Fast, GR00T-N1.5, OpenVLA-OFT, Wan2-Policy, Cosmos-Predict2-Policy) in LIBERO simulation, on 2178 scenarios built from failures the policies themselv.
Sources
- Validated by: Applied to six policies; all showed over 50% drops from standard LIBERO success to recovery success, with monotone decline across the four recovery levels and rank inversions relative to standard LIBERO (Wan2-Policy +14.4 on LIBERO-100 but -5.0 under failures).. Measured across four LIBERO suites and four difficulty levels; level labels and failure localization were assigned by an LLM over rendered videos, so the difficulty hierarchy is partly annotation-defined rather than independently validated, and level counts are imbalanced (L4 rates are often 0.0).
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For vision-language-action and world-action manipulation policies evaluated on LIBERO-style tasks, near-ceiling one-shot task success does not predict the ability to resume a task from a failure state that arose during the model's own execution: all six tested policies lost over half their success rate on recovery scenarios, rankings inverted between the two settings, and recovery success fell steeply as the failure altered object or environment state rather than just the last action.Turning plans into physical action · reviewed by AI
- For LLM agents that score their own actions during exploration in text-based games and then reuse that experience, the accuracy of those self-scores against the environment verifier does not predict later gains on held-out stricter configurations: block-level correlations between judgment agreement or calibration and the next checkpoint's score change sit near zero, so recognizing a good action is not the bottleneck — converting it into a transferable policy is.Improving its own scaffolding from failures · reviewed by AI
- For long-horizon embodied task planning where a symbolic model defines action preconditions and effects, correcting a violated plan by edits the action model itself implies (inserting the missing navigate or open) raises plan success more per LLM call than returning the violation to the LLM for replanning, and the gain persists for frontier models as well as compact ones.Planning a sequence of actions before acting · reviewed by AI
- For frozen VLA policies executing repetitive, perceptually aliased manipulation tasks (counting pick-and-place), a progress memory that advances after each attempted action converts transient grasp or placement failures into persistent task-state errors, so writes must be gated on observed physical evidence of the intended transition; gating this way raised success far above the frozen policy and above memory-augmented baselines.Turning plans into physical action · reviewed by AI
- For a small agent policy (Qwen3.5-4B) repeatedly deployed on the same task family with environment-level feedback, distilling only the trajectories whose textual-state guidance is newly or persistently useful relative to state-free execution raises the state-free policy's accuracy more than distilling all state-conditioned trajectories, and the resulting policy keeps improving over alternating rounds of state editing and consolidation.Improving its own scaffolding from failures · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A policy set whose standard LIBERO ranking reproduces its recovery ranking, or whose L3/L4 recovery rates match its L1/L2 rates on failures drawn from its own rollouts.