Building Better Environments for Autonomous Cyber Defence

Workshop participants from academia, industry, and government convened in November 2025 to establish best practices for designing reinforcement learning environments in autonomous cyber defence. The resulting framework and guidelines address a critical gap in documented knowledge about RL environment development for network security applications, including critical infrastructure protection.

The cybersecurity landscape increasingly relies on autonomous systems to defend networks against sophisticated threats, yet the field lacks comprehensive documentation on how to properly build and evaluate the reinforcement learning environments that train these systems. This workshop represents a significant effort to codify tacit knowledge from practitioners with hands-on experience, converting tradecraft wisdom into actionable frameworks for the broader community.

Autonomous cyber defence has emerged as a priority area due to the scale and speed of modern attacks, which often exceed human response capabilities. While academic literature on RL for cybersecurity exists, it remains fragmented and fails to address practical implementation challenges, edge cases, and evaluation pitfalls that experienced teams encounter repeatedly. By bringing together stakeholders from government, industry, and academia, the workshop created a rare opportunity to synthesize diverse perspectives into unified guidance.

The framework's focus on decomposing the interface between RL environments and real systems addresses a fundamental challenge: bridging simulation and deployment. Organizations developing autonomous defence capabilities face substantial risks when deploying agents trained in unrealistic environments, making environment fidelity and proper evaluation methodologies critical. The published guidelines will reduce development cycles and improve agent reliability across sectors handling sensitive infrastructure.

Looking ahead, adoption of these best practices could accelerate autonomous cyber defence deployment in critical sectors including power grids, healthcare, and financial systems. However, standardized evaluation metrics and open benchmarks will be necessary for broader industry adoption. Future work should focus on establishing publicly available environment templates and validation standards that allow organizations to confidently deploy autonomous systems in production networks.

• →Workshop participants established a comprehensive framework for decomposing RL cyber environment interfaces with real systems.
• →Current best practices for ACD environment development address critical gaps in academic and grey literature.
• →Proper environment design directly impacts autonomous agent reliability in critical infrastructure protection scenarios.
• →Framework standardization could accelerate adoption of autonomous cyber defence across government and industry sectors.
• →The guidelines codify tradecraft knowledge previously scattered across disconnected teams and organizations.