WOMBET: World Model-based Experience Transfer for Robust and Sample-efficient Reinforcement Learning

Researchers introduce WOMBET, a framework that improves reinforcement learning efficiency in robotics by generating synthetic training data from a world model in source tasks and selectively transferring it to target tasks. The approach combines offline-to-online learning with uncertainty-aware planning to reduce data collection costs while maintaining robustness.

WOMBET addresses a fundamental challenge in robotics: the prohibitive cost and safety risks of collecting real-world training data. Traditional offline-to-online reinforcement learning relies on static datasets without optimizing how that data is generated, leaving potential performance gains on the table. This research bridges that gap by treating data generation as an integral part of the transfer learning process rather than a prerequisite.

The framework operates through three coordinated stages. First, it learns a world model—a learned representation of environment dynamics—from source task experience. Second, it generates synthetic trajectories through uncertainty-penalized planning, which balances exploration with confidence in predictions, then filters these trajectories by return quality and epistemic certainty. Third, it performs adaptive sampling during target task training, intelligently blending offline synthetic data with fresh online experience to ensure smooth knowledge transfer.

For the robotics and autonomous systems industries, this work carries practical significance. Sample efficiency directly translates to reduced development timelines and lower operational costs. The mathematical contribution—proving that the uncertainty-penalized objective provides a lower bound on true returns—gives practitioners theoretical grounding for their approach. The finite-sample error analysis quantifies how distribution mismatch and model approximation errors compound during transfer.

The experimental validation on continuous control benchmarks demonstrates measurable improvements over existing methods. This positions WOMBET as a potentially useful tool for companies developing robotic systems where data collection remains a bottleneck. Future applications could extend to autonomous vehicles, industrial manipulation, and other domains where safe, efficient learning from limited real-world data determines commercial viability.

• →WOMBET jointly optimizes data generation and transfer learning rather than treating them as separate pipeline stages.
• →Uncertainty-penalized planning filters synthetic data by confidence and return, improving downstream transfer reliability.
• →Theoretical analysis provides finite-sample error bounds capturing both distribution mismatch and model approximation effects.
• →Adaptive sampling between offline and online data enables stable transitions during target task fine-tuning.
• →Empirical results show sample efficiency and performance gains over strong baselines on continuous control tasks.