Uncertainty-Aware and Temporally Regulated Expert Advice in Reinforcement Learning for Autonomous Driving

Researchers propose an uncertainty-aware reinforcement learning framework for autonomous driving that uses expert guidance to enable safer exploration while avoiding over-dependence on advice. The method combines epistemic and aleatoric uncertainty thresholds with a regulated commitment-cooldown strategy, demonstrating 5-7% improvements in success rates and reduced failures in CARLA simulations for unsignalized intersection navigation.

This research addresses a fundamental challenge in autonomous driving development: how to train reinforcement learning agents through exploration without creating safety hazards. Traditional RL requires agents to experiment with novel behaviors to improve, but in driving contexts, poor exploration decisions can result in collisions or off-road incidents. The proposed solution elegantly separates uncertainty into two types—epistemic uncertainty (knowledge gaps) and aleatoric uncertainty (irreducible randomness)—using adaptive thresholds that evolve as the agent gains confidence. This prevents unnecessary expert interventions when the agent already understands its environment well.

The framework's innovation lies in its regulated guidance mechanism. Rather than providing continuous expert advice, which would prevent genuine learning, or allowing unrestricted exploration, which invites danger, the approach uses a commitment-cooldown strategy with stochastic early-stopping. This ensures agents experience coherent expert-guided maneuvers long enough to learn meaningful patterns while remaining exposed to exploration pressure. By integrating expert and agent experiences in a shared replay buffer within an implicit quantile network, the method efficiently leverages expert demonstrations without architectural complexity.

For the autonomous driving industry, this represents progress toward deployable training pipelines. The 5-7% improvement in success rates may seem modest, but intersection navigation remains one of the most challenging real-world driving tasks. The results suggest uncertainty-aware methods provide better safety-learning tradeoffs than standard reinforcement learning approaches. However, CARLA simulation results don't guarantee real-world performance; weather, sensor degradation, and adversarial human drivers introduce complexities absent from virtual environments. The work signals that managed exploration through uncertainty quantification deserves greater attention in safety-critical AI applications.

• →Uncertainty-aware expert guidance improves RL safety in autonomous driving by triggering interventions only when epistemic or aleatoric uncertainty exceeds adaptive thresholds.
• →The commitment-cooldown strategy prevents over-reliance on expert advice while ensuring agents absorb coherent maneuvers before returning to exploration.
• →CARLA experiments show 5-7% success improvements and reduced failures compared to baseline implicit quantile network approaches.
• →Shared replay buffer integration enables efficient reuse of expert trajectories without requiring architectural modifications to the RL backbone.
• →Regulatory guidance mechanisms addressing uncertainty may provide templates for other safety-critical RL applications beyond autonomous driving.