ActQuant: Sub-4-bit Action-Guided Quantization for Vision-Language-Action Models

ActQuant introduces a novel post-training quantization framework that compresses Vision-Language-Action models to sub-4-bit weights while maintaining 94-95% performance, enabling practical deployment on edge devices. The method combines action-guided bit allocation with curvature-aware optimization, achieving 5.3× compression on major VLA models and validated performance on physical robotic hardware.

ActQuant addresses a critical bottleneck in embodied AI deployment: Vision-Language-Action models deliver impressive capabilities but remain computationally prohibitive for edge platforms. The framework's innovation lies in action-awareness—rather than applying uniform quantization across all weights, it identifies which parameters most directly influence action prediction and concentrates precision there. This targeted approach sidesteps the severe performance cliffs that plague naive aggressive quantization methods, which typically degrade accuracy substantially below 4-bit precision. The accompanying OmniModel.cpp runtime bridges the gap between academic optimization and practical deployment, translating quantized architectures into efficient C/C++ implementations with specialized low-bit kernels.

The competitive landscape reveals why this matters: existing post-training quantization methods fail catastrophically at sub-4-bit regimes, forcing practitioners to accept either impractical model sizes or unacceptable performance loss. ActQuant's demonstration of 95% retention at 3 bits-per-weight represents a qualitative leap. Real-world validation on a UR3 robotic arm confirms the method's robustness beyond simulation benchmarks, maintaining baseline success rates while halving memory footprint.

For the broader AI-on-edge ecosystem, this work enables deployment scenarios previously infeasible: smaller robots, mobile platforms, and resource-constrained environments can now run sophisticated vision-language models. The 5.3× compression ratio transforms a 14.3GB model into 2.7GB, unlocking deployment possibilities at scales from autonomous systems to embedded robotics. As edge deployment becomes commercially critical for robotics and autonomous applications, techniques that preserve capability under extreme compression gain strategic value.

• →ActQuant achieves sub-4-bit quantization with 94-95% performance retention, solving a critical bottleneck in edge deployment of VLA models.
• →Action-guided mixed-precision allocation intelligently assigns different bit-widths to different layers based on their contribution to control decisions.
• →Real-world validation on robotic hardware demonstrates practical viability beyond simulation, maintaining baseline success rates with 2.5× memory reduction.
• →OmniModel.cpp enables production-ready deployment with specialized low-bit kernels, bridging research and practical edge implementation.
• →5.3× compression (14.3GB to 2.7GB) at 3 bits-per-weight opens deployment opportunities for robotics and autonomous systems previously constrained by model size.