PatchWorld: Gradient-Free Optimization of Executable World Models

Researchers introduce PatchWorld, a gradient-free framework that converts offline trajectories into executable Python world models for AI agents operating in partially observable environments. The method achieves 76.4% success on planning tasks without requiring LLM calls during prediction, while revealing a fundamental tradeoff between observation accuracy and decision-making utility in executable world models.

PatchWorld addresses a critical challenge in agent-based AI systems: enabling agents to build and use interpretable world models for planning under uncertainty. Rather than relying on opaque neural network predictions, the framework generates symbolic, executable Python code that models environment dynamics. This approach leverages counterexample-guided code repair to incrementally improve the world model from offline trajectory data, producing programs that are human-inspectable and locally patchable—a significant advantage over black-box neural models in safety-critical applications.

The research builds on longstanding work in symbolic AI and program synthesis while incorporating insights from recent advances in code generation and planning. Previous approaches either sacrificed interpretability for prediction accuracy or used LLMs extensively within the planning loop, increasing computational costs and latency. PatchWorld's gradient-free design removes dependence on differentiable simulators, broadening applicability across diverse environment types.

The framework's 76.4% macro success rate on AgentGym benchmarks demonstrates competitive performance without continuous LLM invocation during world-model prediction. However, the discovery of a fundamental tradeoff between observation fidelity and action-discriminative dynamics reveals important constraints: optimizing for accurate state reconstruction can degrade the model's ability to distinguish between decisions. This tension matters for real-world deployment, where planning agents may prioritize decision quality over perfect observation modeling.

The work has implications for autonomous systems, robotics, and interactive agents where interpretability and robustness matter as much as raw performance. The open-source release enables broader evaluation and potential industrial adoption, particularly in domains where explainable AI and symbolic reasoning carry regulatory or safety requirements.

• →PatchWorld generates human-inspectable executable world models without gradient-based optimization or extensive LLM calls during prediction
• →The framework achieves 76.4% planning success on AgentGym environments, competitive with gradient-based methods while maintaining interpretability
• →A fundamental tradeoff exists between improving observation accuracy and maintaining decision-discriminative action dynamics in world models
• →Symbolic, locally-patchable code offers advantages over black-box neural models for safety-critical agent applications
• →The gradient-free approach enables world model construction from offline trajectories without requiring differentiable simulators