arxiv-2609-05178 SRC-0208 · paperLIBERO-RECOVER: Beyond Task Success Towards Failure Recovery in Robotic Manipulation Models
Lin Liu, Zhicheng Bao, Lu Zhang, Ziying Song, Wu Yang, Shuai Tao
Created: 2026-09-04 · Ingested: 2026-09-19
https://arxiv.org/abs/2609.05178(opens in a new tab)Vision-Language-Action (VLA) or World Action (WAM) models have recently demonstrated remarkable performance in robotic manipulation. On LIBERO, SOTA method have achieved nearly 100\% success rates, seemingly suggesting that the models are ready for deployment in real world. However, near perfect performance on existing benchmarks can be misleading: success under ideal conditions does not imply real world robustness. Existing benchmarks primarily evaluate task completion from predefined initial states, while real world interactions inevitably involve failures such as failed grasps, collisions,
In brief
Near-perfect LIBERO scores do not indicate that manipulation policies can recover from their own mistakes. Placed in failure states that actually occurred during execution, every model tested loses more than 50% of its performance, and standard benchmark rankings stop predicting who does better.
LIBERO-Recover builds 2178 recovery scenarios from 130 subtasks across 4 LIBERO suites, with failures harvested from runs of pi-0, pi-0.5, OpenVLA, GR00T, Wan-Policy and Cosmos-Policy rather than injected by hand, then sorted by Qwen3.5-27B-Instruct into 4 recovery levels from action retry to environmental recovery. 6 policies are evaluated, 10 trials per task.
Recovery success falls steeply with level: pi-0 goes 40.0 at L1 to 6.7 at L4 on LIBERO-Spatial, and no model exceeds 12.5 at L4 anywhere. Averaged over models, success drops from 26.2% to 0.3% on LIBERO-100 post-failure, a 100.0% degradation rate. Shrinking action chunks from 32 to 4 helps; adding the initial frame as temporal context raises OpenVLA-OFT from 20.8% to 26.8%.
The transfer result is the sharpest: mixing 3,184 teleoperated recovery trajectories into training lifts recovery (GR00T 17.8% to 21.2%) but leaves standard LIBERO flat or worse (OpenVLA-OFT 97.1% to 96.6%). That is a matched comparison on the same 2 models.
Level labels come from an LLM, not human audit, and higher levels are sparse. Treat saturated LIBERO numbers as uninformative about deployment robustness.
Written from the abstract by claude-opus-5 on 2026-09-18, with every figure checked against it. Not a substitute for the paper.
Referenced by
Claims in this catalog that draw on this source, and whether as support or counterpoint.
- supportsFor vision-language-action and world-action manipulation policies evaluated on LIBERO-style tasks, near-ceiling one-shot task success does not predict the ability to resume a task from a failure state that arose during the model's own execution: all six tested policies lost over half their success rate on recovery scenarios, rankings inverted between the two settings, and recovery success fell steeply as the failure altered object or environment state rather than just the last action.
- supportsFor VLA and world-action manipulation policies that score near-ceiling on standard LIBERO, recovery success after a real execution failure falls sharply as recovery requires deeper state reasoning \u2014 retrying or adapting the immediate action works far more often than restoring a displaced task object or clearing an environmental obstruction \u2014 and rankings on the standard benchmark do not predict recovery rankings.