Offline Reinforcement Learning for Plasma Control in Nuclear Fusion: Codebase and Benchmark

Researchers introduce RL4F, an open-source benchmark for applying offline reinforcement learning to plasma control in nuclear fusion reactors. Using historical data from the DIII-D tokamak, the framework enables safe algorithm development without costly real-device experimentation, with model-based RL methods showing superior performance across multiple plasma control objectives.

This research addresses a critical bottleneck in fusion energy development: the inability to safely test control algorithms on expensive, dangerous tokamak hardware. By creating RL4F, researchers provide the fusion and AI communities with a standardized evaluation framework that mimics real tokamak dynamics while eliminating the risks and costs of live experimentation. This represents meaningful progress toward autonomous plasma management, a prerequisite for commercial fusion viability.

Offline reinforcement learning has emerged as a practical alternative to online learning in high-stakes environments. The benchmark's use of historical DIII-D discharge data grounds the evaluation in real-world physics rather than simplified simulations, increasing the relevance of algorithm comparisons. The finding that offline model-based RL methods outperform imitation learning on most tasks underscores how dynamics modeling becomes crucial when dealing with complex, long-horizon control problems that tokamak operation demands.

The open-source release carries significant implications for both fusion research and AI algorithm development. For fusion, it democratizes access to sophisticated control optimization tools beyond institutions with tokamak facilities. For the broader offline RL community, it provides a challenging benchmark with real physical constraints, potentially accelerating algorithmic improvements applicable across robotics, autonomous systems, and other domains requiring learning from offline data.

The work's significance lies not in immediate commercial applications but in establishing infrastructure for sustained progress. As fusion projects race toward demonstration reactors, having standardized offline RL benchmarks reduces development timelines and enables collaborative research. The lack of a dominant method across all tasks suggests substantial room for algorithmic innovation and specialization.

• →RL4F provides the first standardized offline RL benchmark for realistic multi-actuator plasma control derived from real tokamak data.
• →Model-based offline RL methods demonstrated superior average performance over imitation learning baselines across four full-profile tracking objectives.
• →Open-sourcing the codebase, datasets, and framework enables broader AI research while advancing fusion reactor control capabilities.
• →The benchmark addresses the critical safety and cost barriers that prevent direct algorithm testing on experimental tokamaks.
• →No single algorithm dominates all tasks, indicating significant opportunities for specialized offline RL algorithm development in fusion domains.