In-Context Reinforcement Learning via Communicative World Models

Researchers introduce CORAL, a framework that enables reinforcement learning agents to adapt to new tasks without retraining by separating world modeling from control through emergent communication between two agents. The approach demonstrates improved sample efficiency and zero-shot adaptation across diverse environments, advancing in-context reinforcement learning capabilities.

This research addresses a fundamental challenge in reinforcement learning: agents trained on specific tasks struggle to generalize when deployed in new contexts without parameter updates. CORAL tackles this by reimagining the problem as emergent communication between specialized agents rather than traditional end-to-end policy learning.

The framework's innovation lies in its functional separation. An Information Agent pre-trained as a world model learns to compress task understanding into communicative messages shaped by a novel Causal Influence Loss—measuring how messages affect subsequent actions. This contrasts with conventional approaches where agents try to simultaneously learn representations and control strategies, often leading to overfitting. By fixing the Information Agent during deployment, the Control Agent can focus solely on interpreting context and executing actions.

For the AI research community, this work validates that emergent communication protocols can encode transferable knowledge across diverse task distributions. The demonstrated zero-shot adaptation in both online and offline settings suggests practical applications in robotics, autonomous systems, and game-playing agents where rapid task switching is valuable. The improved sample efficiency addresses deployment constraints in real-world scenarios where data collection is expensive.

The significance extends beyond academic benchmarks. If CORAL-like approaches scale effectively, they could accelerate the development of more adaptable AI systems that require fewer task-specific training runs. This has implications for reducing computational costs in AI development and enabling systems to operate effectively in unpredictable environments. Future work should examine how these communicative representations perform across truly novel task distributions and whether the approach scales to more complex domains.

• →CORAL separates world modeling from control through two-agent communication, enabling better generalization without retraining.
• →A novel Causal Influence Loss shapes communication by measuring message impact on agent actions.
• →The framework achieves zero-shot adaptation across diverse online and offline environments with improved sample efficiency.
• →Information Agent remains fixed during deployment, allowing Control Agents to quickly adapt to new tasks.
• →Results suggest emergent communication protocols can encode transferable knowledge across varied task distributions.