Delay-Aware Reinforcement Learning for Highway On-Ramp Merging under Stochastic Communication Latency

Researchers introduce DAROM, a reinforcement learning framework designed to handle stochastic communication delays in autonomous vehicle highway merging scenarios. The system uses a delay-aware encoder to maintain decision-making performance despite V2I transmission latencies up to 2.0 seconds, achieving over 99% success rates in high-density traffic conditions.

This research addresses a critical gap between theoretical RL applications and real-world autonomous driving deployment. Standard reinforcement learning algorithms assume instantaneous state observation, but connected vehicle systems inevitably experience communication delays from edge processing and wireless transmission. DAROM's innovation lies in explicitly modeling these delays as part of the decision problem rather than treating them as external noise, fundamentally shifting how RL agents handle imperfect information in safety-critical applications.

The technical approach using a Delay-Aware Encoder that conditions on historical observations, masked actions, and explicit delay magnitude demonstrates sophisticated state reconstruction under uncertainty. By integrating physics-based safety controllers alongside learned policies, the framework balances learning-driven optimization with guaranteed safety constraints—a crucial requirement for autonomous systems. The use of real-world NGSIM traffic data in SUMO simulations suggests practical relevance rather than toy problem optimization.

For the autonomous vehicle industry, this work validates the feasibility of RSU-assisted perception architectures despite infrastructure limitations. Vehicle-to-infrastructure communication is increasingly deployed in smart highway systems, but inconsistent latency has been a known barrier to reliable autonomous decision-making. Demonstrating robust performance at 2.0-second delays makes V2I assistance viable even with imperfect communication. The research also advances broader RL safety considerations, offering methodologies applicable to other delay-sensitive control domains beyond transportation.

Future development should focus on real-world validation beyond simulation and exploration of how these techniques scale to mixed traffic environments with varying communication capabilities across vehicle types.

• →DAROM achieves 99%+ highway merging success in high-density traffic despite random V2I delays up to 2.0 seconds
• →Delay-aware encoder explicitly models communication latency as part of the state representation rather than treating it as disturbance
• →Integration of physics-based safety controllers with learned RL policies ensures collision risk reduction during autonomous merging
• →Framework validates RSU-assisted perception viability for connected autonomous vehicles despite inevitable infrastructure latencies
• →Approach demonstrates general methodology applicable to other safety-critical control domains facing stochastic observation delays