CT-VAM: A Cerebello-Thalamic-Inspired Vision-Action Model for Efficient Visuomotor Control

Researchers introduce CT-VAM, a compact 68M-parameter neural network inspired by cerebellar-thalamic brain architecture for robotic manipulation tasks. The model processes visual inputs and proprioception to predict action sequences efficiently on edge devices, matching larger vision-language-action models while reducing latency and enabling practical deployment on resource-constrained robots.

CT-VAM represents a meaningful advancement in embodied AI by decoupling high-level semantic reasoning from low-level motor control. Traditional vision-language-action models process language tokens continuously during execution, creating computational overhead unnecessary for repetitive control loops. This research demonstrates that language input primarily serves task specification at initialization, allowing the authors to create a specialized execution layer that operates independently on local hardware.

The architecture introduces TARS (Thalamic Action Routing Stream), a novel attention mechanism that prevents dense sensory data from overwhelming task-relevant signals. This design choice addresses a fundamental challenge in multimodal AI: balancing heterogeneous input streams without architectural bloat. By maintaining only 68M parameters—orders of magnitude smaller than comparable VLA models—CT-VAM achieves competitive performance on LIBERO benchmarks while dramatically reducing inference latency.

The cloud-edge paradigm the authors propose has significant practical implications for robotics deployment. Large language models handle abstract reasoning and task planning in the cloud, while local hardware executes real-time control loops. This separation enables faster response times critical for manipulation tasks and reduces bandwidth requirements and computational costs. The integration of flow-consistent inpainting for asynchronous chunk execution further optimizes performance on edge devices with limited resources.

As robot manufacturers increasingly deploy systems in real-world environments, efficiency bottlenecks become business constraints rather than academic concerns. This work demonstrates that model specialization—rather than scaling general-purpose models—may be the practical path forward for embodied AI applications.

• →CT-VAM achieves LIBERO benchmark performance competitive with much larger vision-language-action models using only 68M parameters
• →The TARS mechanism enables effective fusion of visual, proprioceptive, and task-condition inputs through stream-separated attention routing
• →Cloud-edge architecture separates semantic reasoning from real-time control, enabling deployment on resource-constrained robotic platforms
• →Reduced inference latency supports high-frequency control loops critical for complex manipulation tasks
• →Model specialization for specific layers of the control hierarchy may offer better scalability than scaling monolithic foundation models