One Image is All You Need: Agentic One-Shot Image Generation via Text-Based World Models for Long-Tail Spatial Perception

Researchers introduce WMGen-v1, an AI framework combining vision-language models with diffusion techniques to generate synthetic training data for autonomous systems. The system addresses the critical challenge of rare, safety-critical scenarios in spatial perception by creating physically plausible synthetic data from single reference images, demonstrating that models trained purely on generated data can approach real-world performance levels.

WMGen-v1 represents a meaningful advancement in synthetic data generation for spatial AI applications, tackling a fundamental problem in deploying autonomous systems at scale. Real-world sensor data exhibits extreme long-tail distributions where safety-critical edge cases—like unusual weather conditions or rare traffic scenarios—occur infrequently, making them expensive and dangerous to collect. Existing generative approaches like diffusion models and GANs struggle with spatial consistency, producing physically implausible scenes that fail to train robust detectors.

This work builds on the convergence of large language models and vision systems. By leveraging an LVLM to parse spatial relationships from a single image and an LLM to reason about physical constraints and commonsense scene dynamics, WMGen-v1 grounds synthetic generation in structured semantic understanding rather than pattern matching. The subsequent diffusion model then produces diverse variations while maintaining physical plausibility, creating a principled pipeline from semantic reasoning to pixel-level generation.

The implications extend across autonomous driving, robotics, and surveillance sectors where data scarcity and safety-criticality create deployment bottlenecks. Achieving near-parity between synthetic-only and real-data training suggests organizations can reduce expensive data collection campaigns while improving coverage of dangerous edge cases. This directly impacts development timelines and safety validation for autonomous systems.

Future work should examine generalization across domains, the quality of rare-case coverage, and whether improvements persist as synthetic data scale increases. The framework's reliance on high-quality reference images and structured reasoning may limit application to domains with limited visual data, warranting investigation into few-shot or zero-shot expansion capabilities.

• →WMGen-v1 uses LVLMs and LLMs to create spatially grounded synthetic training data from single reference images for autonomous systems.
• →Synthetic-only trained detectors approached real-world performance on aggregate metrics, addressing long-tail data scarcity challenges.
• →The framework combines semantic reasoning with diffusion models to ensure physical plausibility and reduce spatial inconsistencies in generated scenes.
• →Application spans autonomous driving, maritime surveillance, and robotics where safety-critical edge cases are rare and expensive to collect.
• →Results on ROADWork and LaRS benchmarks validate the approach's effectiveness against baseline generative methods.