WorldCoder-Bench: Benchmarking Physically Grounded 3D World Synthesis

Researchers introduce WorldCoder-Bench, a comprehensive benchmark for evaluating how well AI language models can generate interactive 3D web environments built with Three.js. The benchmark reveals that current frontier models achieve only 19.9-27.8% verification coverage, with failures primarily stemming from state management issues rather than missing visual elements.

WorldCoder-Bench addresses a critical gap in AI evaluation methodology by moving beyond pixel-level and DOM-node analysis to assess the functional correctness of generated 3D interactive environments. As large language models expand from code generation into complex runtime systems, traditional benchmarks prove insufficient—they cannot evaluate whether generated programs maintain proper state synchronization, enforce physical constraints, or preserve user interaction chains across a Three.js canvas execution environment.

The benchmark's design reflects the growing sophistication of LLM applications. Rather than simply measuring whether models can produce syntactically correct code, WorldCoder-Bench employs StateProbe, an execution-based protocol that sandboxes generated programs and verifies hidden behavioral contracts. This approach mirrors broader industry trends toward behavioral verification rather than structural compliance. The inclusion of 2,026 expert-curated tasks across simulation, rendering, and application scenarios creates a rigorous testing framework that accounts for physical realism and interactive coherence.

The results carry significant implications for AI development. The gap between best-in-class model performance (27.8%) and theoretical success suggests that current approaches struggle with state-schema management—a systematic weakness that impacts practical deployment. This finding suggests developers cannot yet rely on LLMs for autonomous 3D world generation without substantial human oversight. For AI researchers and tool developers, the benchmark identifies specific failure modes worth optimizing around, while the introduction of Return on Automation and Time Efficiency Multiplier metrics acknowledges that imperfect solutions may still provide value for simpler use cases. The work establishes evaluation standards that will likely influence how future models train on code generation tasks.

• →WorldCoder-Bench introduces the first specialized benchmark for evaluating AI-generated 3D interactive environments with hidden state verification.
• →Current frontier LLMs achieve only 19.9-27.8% verification coverage, with state management failures dominating over missing visual assets.
• →StateProbe execution protocol enables sandboxed testing of runtime behavior, addressing limitations of pixel-only evaluation methods.
• →Benchmark reveals cost-adjusted utility metrics show cheaper models can provide value on easier domains despite lower absolute performance.
• →Research identifies state-schema drift and broken interaction chains as primary failure modes rather than scene generation deficiencies.