COMAP: Co-Evolving World Models and Agent Policies for LLM Agents

Researchers introduce COMAP, a framework that enables language model agents to improve through co-evolution of world models and policies via closed-loop interaction, eliminating the need for external rewards. The approach achieves significant performance gains across multiple benchmarks, demonstrating that self-improving AI agents can adapt their internal representations to match their evolving behavior patterns.

COMAP represents a meaningful advancement in autonomous AI agent development by addressing a fundamental limitation in current language model systems: the inability of fixed world models to adapt as agents evolve. Traditional approaches train world models once and freeze them, creating a mismatch between the model's assumptions and the agent's actual decision-making patterns. This research demonstrates that allowing these components to co-evolve through real interaction creates a reinforcing cycle where better world models enable better decisions, which in turn provide better training data for improved models.

The framework's significance lies in its elimination of external reward signals or verifier systems, which have been necessary crutches for agent improvement but limit practical deployment in complex environments. By relying on self-distillation and the agent's own reflection on prediction reliability, COMAP creates a more autonomous learning loop. The 16.75% performance improvement on smaller models like Qwen3-4B is particularly noteworthy, suggesting the approach scales effectively across different model sizes.

For the AI development community, this work implies that future agent systems may require fundamentally different architectures—ones designed for continuous adaptation rather than static deployment. The consistent improvements across embodied task planning, web navigation, and tool-use domains indicate this isn't a narrow solution but a broadly applicable methodology.

The accessibility of published code accelerates adoption and validation by other researchers. Future developments will likely focus on scaling these principles to larger models, extending the framework to multi-agent scenarios, and understanding how world model accuracy directly correlates with long-horizon task success.

• →COMAP enables language agents to improve without external rewards by co-evolving world models and policies through closed-loop interaction
• →The framework achieves 16.75% relative performance improvement on Qwen3-4B and shows consistent gains across embodied, web navigation, and tool-use tasks
• →Self-distillation allows world models to adapt to on-policy agent distributions, eliminating the fixed-model limitation of existing systems
• →The approach demonstrates that agents performing future-aware reflection on prediction reliability outperform baselines in long-horizon decision-making
• →Published code availability enables rapid community validation and potential integration into autonomous AI systems