OracleTSC: Oracle-Informed Reward Hurdle and Uncertainty Regularization for Traffic Signal Control

Researchers introduce OracleTSC, an LLM-based traffic signal control system that combines reward hurdle mechanisms and uncertainty regularization to stabilize reinforcement learning training. The approach achieves 75% reduction in travel time while maintaining interpretability through natural language explanations, with strong cross-intersection generalization capabilities.

OracleTSC addresses a fundamental challenge in applying large language models to real-world control systems: the instability that arises when training signals are sparse and marginal. Traditional reinforcement learning approaches to traffic signal control have struggled with weak feedback loops where most actions produce minimal observable changes in congestion metrics. This research demonstrates that filtering noisy reward signals through a calibrated threshold mechanism, combined with encouraging consistency across model outputs, creates a more stable learning environment suitable for LLM fine-tuning.

The core innovation lies in recognizing that not all environmental feedback deserves equal weight in the training process. By implementing a reward hurdle that subtracts a threshold from observed rewards, the system effectively ignores marginal changes that would otherwise confuse the learning process. Uncertainty regularization further stabilizes training by maximizing probability of selected responses, reducing the variance typically associated with sampling-based language model outputs.

The practical implications are significant for urban infrastructure optimization. A compact 8B parameter model achieving 75% travel time reduction represents substantial efficiency gains with lower computational requirements than larger alternatives. More importantly, the cross-intersection generalization—transferring policies between structurally different intersections without additional training—suggests the approach captures fundamental traffic control principles rather than memorizing specific intersection configurations.

This work bridges the interpretability-performance gap that has hindered LLM adoption in critical infrastructure. Natural language explanations for signal timing decisions could enhance public trust in autonomous systems and simplify regulatory compliance. Future deployment could verify whether these improvements translate to real-world traffic conditions beyond simulation environments, and whether the approach scales to complex, interconnected signal networks.

• →OracleTSC reduces travel time by 75% and queue length by 67% using a compact LLaMA3-8B model with interpretable natural language reasoning.
• →Reward hurdle filtering and uncertainty regularization stabilize LLM fine-tuning by addressing sparse feedback and marginal reward signals.
• →Policies trained on single intersections transfer to structurally different intersections with 17% lower travel time without additional fine-tuning.
• →The approach maintains transparency through natural language explanations, addressing public trust concerns in autonomous traffic control systems.
• →Research validates that uncertainty-aware reward shaping effectively improves reinforcement learning stability for complex real-world control tasks.