Finetuning Vision-Language-Action Models Requires Fewer Layers Than You Think

Researchers demonstrate that Vision-Language-Action (VLA) models used in robotic manipulation contain significant layer-wise redundancy, enabling a training-free compression method that reduces model depth by up to 50% while improving downstream fine-tuning speed by 40-50% and inference speed by 30%. This finding suggests advanced robotics foundation models can operate effectively with substantially fewer parameters than currently assumed.

The robotics and AI communities have increasingly relied on massive Vision-Language-Action models pre-trained on diverse video-robot datasets to advance robotic manipulation capabilities. However, these multi-billion parameter architectures create practical constraints for real-world deployment, where computational efficiency directly impacts both training costs and real-time operational speed. This research identifies a critical inefficiency: despite their diverse training, VLA models exhibit severe representational redundancy across layers, meaning many layers perform functionally similar operations.

The breakthrough lies in the proposed training-free compression pipeline, which uses Centered Kernel Alignment to identify redundant features without requiring full model loading or optimization. By removing twin layers, researchers achieve substantial compression while maintaining or exceeding baseline performance across comprehensive evaluations. This approach differs fundamentally from existing methods requiring expensive fine-tuning or dynamic layer selection mechanisms.

For the robotics and AI infrastructure industries, this finding carries significant implications. Organizations developing or deploying VLA models can immediately reduce computational overhead for fine-tuning and inference, lowering operational costs and enabling deployment on resource-constrained robotic systems. The results demonstrate that model depth—a commonly cited architectural characteristic—may not be the limiting factor for performance.

Future work should explore whether similar redundancy patterns exist in other foundation model classes and whether selective layer removal can be combined with other compression techniques. The validation across multiple simulation environments and real-world tasks suggests broad applicability, though understanding why these models develop such redundancy could further optimize architecture design for robotics applications.

• →VLA models exhibit severe layer-wise redundancy despite training on diverse physical trajectories, enabling 50% depth reduction without performance loss.
• →Training-free compression using Centered Kernel Alignment eliminates the need to load full-scale models or learn dynamic layer selectors.
• →Compressed models achieve 40-50% faster fine-tuning and up to 30% faster real-time inference while matching baseline performance.
• →Validation spans three simulation benchmarks and 10 real-world tasks across four robotic embodiments, demonstrating broad applicability.
• →Results suggest advanced robotics foundation models require significantly fewer layers than currently assumed, improving scalability of robot learning.