Cognition to Control - Multi-Agent Learning for Human-Humanoid Collaborative Transport

Researchers developed a new three-layer hierarchy called cognition-to-control (C2C) for human-robot collaboration that combines vision-language models with multi-agent reinforcement learning. The system enables sustained deliberation and planning while maintaining real-time control for collaborative manipulation tasks between humans and humanoid robots.

• →C2C framework addresses the gap between high-level AI reasoning and low-latency physical control in human-robot collaboration.
• →The system uses a three-layer hierarchy combining VLM-based grounding, deliberative coordination, and whole-body control layers.
• →Multi-agent reinforcement learning is cast as a Markov potential game to optimize long-horizon coordination between human and robot partners.
• →Experiments show higher success rates and robustness compared to single-agent and end-to-end baseline approaches.
• →The system demonstrates emergent leader-follower behaviors without explicit role assignment in collaborative tasks.