language-conditioned-vla-manipulation-tasks-instantiated-graded-difficulty-levels-add CLM-0202When language-conditioned VLA manipulation tasks are instantiated at graded difficulty levels that add spatial ambiguity (fine-grained spatial relations, precise placement) and procedural length (multi-stage plans requiring memory of completed steps), representative VLA models degrade on both axes, so success on predefined single-stage settings does not indicate spatial or long-horizon competence.
Capability: Turning plans into physical action
Observed on
Representative VLA models evaluated on a simulated/collected manipulation benchmark with 10 task categories, 56 base tasks, 5 difficulty levels each, across multiple embodiments..
Sources
- Benchmark paper: 527K trajectories, 280 task variants, diagnostic metrics beyond binary success. The abstract reports qualitative failure directions (complex spatial relations, precise low-level execution, memory-intensive planning) without per-level numbers in the text given, and does not isolate which of perception, control, or memory causes the drop.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- On language-conditioned manipulation tasks where the same base task is instantiated at increasing levels of spatial ambiguity and procedural length, current VLA models degrade on fine-grained spatial relations, precise low-level execution, and memory-dependent multi-step planning, so success measured only on predefined simple settings overstates their competence.Turning plans into physical action · reviewed by AI
- When adapting a pretrained VLA manipulation policy with only a handful of target demonstrations, retrieving prior demonstrations at the level of LLM-decomposed subtasks (then reranking by behavior-feature similarity) raises success more than retrieval by visual, optical-flow, or state-action similarity, and the margin is concentrated on long-horizon composite tasks rather than single-skill pick-and-place.Turning plans into physical action · reviewed by AI
- For frozen VLA policies executing repetitive, perceptually aliased manipulation tasks (counting pick-and-place), a progress memory that advances after each attempted action converts transient grasp or placement failures into persistent task-state errors, so writes must be gated on observed physical evidence of the intended transition; gating this way raised success far above the frozen policy and above memory-augmented baselines.Turning plans into physical action · reviewed by AI
- For multi-step assembly task planning from an action history and a workstation image, prompting a multimodal LLM to emit a structured (JSON) state representation and propagate it through each predicted action — then truncating the plan at the predicted state closest to the inferred goal — raises action executability and lowers final-state error relative to predicting actions alone, at the cost of shorter plans (lower overlap with recorded human action sequences) and roughly eight times the latency.Planning a sequence of actions before acting · reviewed by AI
- When a pretrained vision-language-action policy is fine-tuned sequentially on a stream of real-robot manipulation tasks, the choice of action-normalization statistics is itself a determinant of continual-learning success: statistics that are task-derived, first-task-derived, or recomputed each stage produce forgetting or poor transfer, while statistics estimated once from a small task-independent calibration set covering the embodiment's motion range and then frozen give the best acquisition, retention and task-order stability.Turning plans into physical action · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A VLA model that holds near-constant success rate across the five difficulty levels on these task families, with no drop on the spatial-ambiguity or memory-intensive planning categories. Proposed technique, not catalogued: Graded spatial-procedural difficulty benchmark for VLA models.