ScenePilot: Controllable Boundary-Driven Critical Scenario Generation for Autonomous Driving

ScenePilot is a new framework for generating safety-critical scenarios to test autonomous driving systems by targeting the boundary between physically feasible and infeasible situations. Using constrained reinforcement learning combined with physical feasibility constraints, the method achieves 6.2 percentage points higher collision rates while maintaining physical validity, enabling more effective stress testing of AV safety systems.

ScenePilot addresses a fundamental gap in autonomous vehicle safety testing: the ability to generate realistic, challenging scenarios that don't resort to physically impossible crash conditions. Traditional scenario generation approaches face a critical trade-off—either they create visually extreme failures that violate vehicle physics, or they enforce feasibility constraints so strictly that they fail to stress-test the autonomy stack effectively. This research bridges that divide through a sophisticated framework that explicitly models the boundary band: scenarios physically achievable in principle but still capable of causing failures in deployed systems.

The technical approach combines RSS-derived physical feasibility scoring with an online-learned risk predictor, using constrained multi-objective reinforcement learning to keep exploration near the feasibility frontier. The step-level shielding mechanism prevents the system from wandering into physically impossible territory while still discovering high-risk scenarios. This represents a meaningful evolution in simulation-based testing methodology for autonomous systems.

From an industry perspective, this work has substantial implications for AV developers and safety validation teams. Higher-fidelity stress testing directly translates to more robust systems before real-world deployment. The demonstrated ability to reduce downstream crash rates through adversarial fine-tuning on these boundary-band scenarios provides a practical pathway for improving safety performance. As autonomous vehicles move toward broader deployment, the quality of simulation-based testing becomes increasingly critical to public safety and regulatory approval.

The open-source availability of ScenePilot code may accelerate adoption across the AV industry, potentially establishing this boundary-driven approach as a standard evaluation methodology. Future work likely involves scaling this approach to more complex driving scenarios and multi-vehicle interactions.

• →ScenePilot generates physically feasible yet failure-inducing scenarios, achieving 6.2 percentage points higher collision rates than prior methods.
• →The framework combines RSS physical feasibility constraints with learned risk prediction to navigate the boundary between solvable and unsolvable driving scenarios.
• →Adversarial fine-tuning on these boundary-band scenarios consistently reduces downstream crash rates in deployed autonomy stacks.
• →The approach addresses a critical limitation in AV safety testing by avoiding physically impossible crashes while maximizing scenario challenge.
• →Open-source release positions ScenePilot as a potential industry standard for simulation-based autonomous vehicle stress testing.