Dreaming Of Others: Latent Teammate Modeling In World Models For Multi-Agent Reinforcement Learning

Researchers propose a novel architecture for multi-agent reinforcement learning that models teammates as learnable components within a world model, using a Theory-of-Mind head to infer partner behavior and enable zero-shot coordination. This approach extends Dreamer-style models beyond single-agent settings by factorizing latent states into environment and teammate representations, potentially advancing cooperative AI systems.

This research addresses a fundamental challenge in multi-agent reinforcement learning: how agents can effectively coordinate with partners whose internal policies remain unobservable. The proposed solution treats teammates not as external entities but as structured, learnable components within an agent's world model, representing a meaningful conceptual shift in how AI systems approach collaboration.

The work builds on Dreamer, a successful world model architecture proven in single-agent environments, and extends it to handle the complexity of multi-agent coordination. By factorizing the latent state into environment and teammate components, the system creates interpretable representations of partner behavior including character, intent, and predicted actions. This architectural choice mirrors human social reasoning, where we mentally model others' intentions to predict their behavior.

The introduction of a Theory-of-Mind head marks the research's most significant contribution, enabling agents to infer teammate characteristics from limited trajectory data. This capability directly supports zero-shot and few-shot coordination scenarios where agents must quickly adapt to new partners—a practical requirement for real-world deployment. The ability to generalize across diverse collaborators without extensive retraining could substantially reduce sample complexity in multi-agent systems.

While this is foundational research without immediate market applications, it positions world models as simulators of social behavior rather than purely environmental predictors. For AI development communities, this opens directions toward more generalizable and human-compatible systems. The proposed benchmarks and evaluation protocols provide measurement frameworks that future research can build upon, potentially establishing standards for assessing multi-agent coordination capabilities.

• →World models can be extended to MARL by factorizing latent states into environment and teammate components with auxiliary Theory-of-Mind heads.
• →The approach enables zero-shot and few-shot coordination by inferring teammate behavior characteristics from partial trajectories.
• →Treating teammates as learnable model components mirrors human social reasoning and could improve generalization across diverse partners.
• →This research reframes world models as social behavior simulators, expanding their application beyond environmental dynamics prediction.
• →Proposed benchmarks establish evaluation protocols for assessing multi-agent coordination in partially observable settings.