fine-tuning-vision-language-action-policy-deployment-shift-parameter-change-needed-concentrates CLM-0569mechanismsingle paperReviewed by AI
When fine-tuning a vision-language-action policy for a deployment shift, the parameter change needed concentrates in the functional region matching the shift's modality — appearance in the vision encoder, rephrased instructions in the language backbone, novel objects in the vision encoder plus action head — so allocating LoRA rank per region by a pre-training diagnostic beats uniform-rank LoRA at matched budget, with the advantage largest at small budgets and absent when the shift's demand spreads across regions rather than concentrating.
Capability: Turning plans into physical action
Observed on
Five VLA architectures from 93M to 7B across autoregressive, diffusion and flow-matching decoders; LIBERO and CALVIN plus one xArm-7 embodiment; ~20 target demonstrations per task;.
Sources
- supportsNot All Layers Need Tuning: Diagnosing and Directing Adaptation in Vision-Language-Action ModelsValidated by: Diagnostic scores ranked regions against measured region-isolated parameter displacement at median Spearman 0.91; at a matched budget of roughly half uniform rank-32, the allocation beat uniform-matched LoRA, DoRA, (IA)^3, AdaLoRA, LoRA-SP, random and dominant-region-only allocation on all five architectures on LIBERO and CALVIN (5 seeds, 50 rollouts/task), and on the xArm-7 language shift reached 21/30 vs 22/30 for full fine-tuning at 0.04% of its trainable parameters; random allocation at the same budget collapsed to 42.8% vs 73.4%.. [truncated]
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- In 4-7B open vision-language models (LLaVA-1.5-7B, Qwen2.5-VL-7B, InternVL-2.5-4B), how the cross-modal attention distribution over visual tokens shifts between adjacent decoder layers separates hallucinated from grounded object predictions better than the model's sensitivity to masking the attended region, and better than attention drift measured against a fixed first or last layer anchor.Grounding answers in the image · reviewed by AI
- When a pretrained vision-language-action policy is fine-tuned sequentially on a stream of real-robot manipulation tasks, the choice of action-normalization statistics is itself a determinant of continual-learning success: statistics that are task-derived, first-task-derived, or recomputed each stage produce forgetting or poor transfer, while statistics estimated once from a small task-independent calibration set covering the embodiment's motion range and then frozen give the best acquisition, retention and task-order stability.Turning plans into physical action · reviewed by AI
- When adapting a pretrained VLA manipulation policy with only a handful of target demonstrations, retrieving prior demonstrations at the level of LLM-decomposed subtasks (then reranking by behavior-feature similarity) raises success more than retrieval by visual, optical-flow, or state-action similarity, and the margin is concentrated on long-horizon composite tasks rather than single-skill pick-and-place.Turning plans into physical action · reviewed by AI
- For real-world online RL post-training of large pretrained vision-language-action policies on high-precision manipulation, optimizing the action head toward a paired advantage margin against a frozen reference action, rather than maximizing absolute Q values, yields higher autonomous success on long multi-step tasks, because absolute-Q maximization amplifies early optimistic value errors into policy drift that behavior-cloning regularization alone does not stop.Turning plans into physical action · reviewed by AI
- For vision-language-action and world-action manipulation policies evaluated on LIBERO-style tasks, near-ceiling one-shot task success does not predict the ability to resume a task from a failure state that arose during the model's own execution: all six tested policies lost over half their success rate on recovery scenarios, rankings inverted between the two settings, and recovery success fell steeply as the failure altered object or environment state rather than just the last action.Turning plans into physical action · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A VLA family where region-isolated adaptation cost does not track the shifted modality, or where diagnostic-driven variable-rank LoRA fails to beat uniform LoRA at matched low budgets once random-allocation and checkpoint-selection controls are held equal.