ScenicRules: An Autonomous Driving Benchmark with Multi-Objective Specifications and Abstract Scenarios

Researchers introduce ScenicRules, a new benchmark for evaluating autonomous driving systems that combines multi-objective prioritized specifications with formal environment models. The framework uses a Hierarchical Rulebook to encode driving objectives and their priority relations, enabling more realistic assessment of autonomous vehicle performance against human driving standards.

ScenicRules addresses a critical gap in autonomous driving evaluation methodology. Existing benchmarks fail to capture the inherent trade-offs between competing objectives—collision avoidance, traffic rule compliance, and efficient progress—that characterize real-world driving. This new framework formalizes these multi-objective constraints with explicit priority relations, reflecting how human drivers naturally resolve conflicting goals in complex traffic scenarios.

The benchmark's innovation lies in its combination of three components: quantitative evaluation metrics for diverse driving objectives, a Hierarchical Rulebook architecture that maintains interpretability while encoding priority relations, and formally modeled scenarios in the Scenic language covering varied driving contexts and near-accident situations. This structured approach moves beyond simplistic pass-fail metrics toward nuanced performance assessment that mirrors human judgment.

For the autonomous vehicle industry, this work has substantial implications. Current AV development relies on metrics that often fail to capture safety-critical edge cases or reveal systematic weaknesses in decision-making logic. ScenicRules provides developers with diagnostic tools to identify failure modes and validate that their systems handle competing objectives appropriately. The open-source availability through GitHub democratizes access to rigorous evaluation standards.

Looking forward, adoption of formalized, priority-aware benchmarks could accelerate AV safety validation and build confidence in deployment-ready systems. This work also signals growing academic focus on the formal verification of autonomous systems, potentially influencing regulatory frameworks that will govern future AV approvals and liability standards.

• →ScenicRules benchmark enables evaluation of autonomous vehicles under competing, prioritized objectives that reflect real-world driving complexity.
• →The Hierarchical Rulebook framework formally encodes driving rules and their priority relations in an interpretable, adaptable manner.
• →Experimental validation shows the benchmark's metrics align well with human driving judgments and effectively expose agent failures.
• →Open-source release provides the AV development community with standardized evaluation tools for safety-critical decision-making.
• →Formalized scenario modeling in the Scenic language enables systematic testing across diverse traffic contexts and near-accident situations.