arxiv-2609-18084 SRC-0564 · paper

Not All Layers Need Tuning: Diagnosing and Directing Adaptation in Vision-Language-Action Models

Shahram Najam Syed, Arthur Jakobsson, Prayuj Sachdev, Jeffrey Ichnowski

Created: 2026-09-16 · Ingested: 2026-09-27

Fine-tuning a Vision-Language-Action (VLA) model for a new deployment environment is expensive, yet most methods apply uniform-capacity adapters to every network region as if every region requires equal adjustment. This paper tests that assumption on five architecturally diverse VLAs (OpenVLA-OFT, $π_0$, SmolVLA, DTP, Octo; 93M-7B parameters). Measuring per-region adaptation cost as normalized parameter displacement under region-isolated fine-tuning reveals an adaptation spectrum in which appearance shifts concentrate cost in the vision encoder, instruction shifts in the language backbone, and

Referenced by

Claims in this catalog that draw on this source, and whether as support or counterpoint.

supportsWhen fine-tuning a vision-language-action policy for a deployment shift, the parameter change needed concentrates in the functional region matching the shift's modality — appearance in the vision encoder, rephrased instructions in the language backbone, novel objects in the vision encoder plus action head — so allocating LoRA rank per region by a pre-training diagnostic beats uniform-rank LoRA at matched budget, with the advantage largest at small budgets and absent when the shift's demand spreads across regions rather than concentrating.