On the Limitations of Ray-Tracing for Learning-Based RF Tasks in Urban Environments

Researchers evaluated the realism of Sionna ray-tracing simulator for outdoor cellular networks in Rome using 1,664 real user equipment measurements across six base stations. The study found that while precise antenna geometry and positioning are critical for simulation accuracy, capturing urban environmental noise remains an unsolved challenge that limits the simulator's practical applicability for real-world RF learning tasks.

This technical research addresses a fundamental gap in computational electromagnetics and machine learning for wireless communications. The study systematically benchmarks Sionna v1.0.2, a GPU-accelerated ray-tracing simulator, against measured data from an actual urban cellular network. Researchers discovered that simulation hyperparameters like path depth and material properties have minimal impact on accuracy, while antenna placement and orientation prove decisive—improvements of 5-130% in signal correlation were achieved through greedy optimization alone.

The findings have significant implications for the machine learning community's reliance on synthetic data for RF applications. Fingerprinting-based localization using only simulated reference data achieved errors one-third better than the unoptimized baseline but remained twice worse than real-world-only methods. This persistent gap highlights that realistic geometry and antenna modeling, while necessary, cannot compensate for unmeasured urban scattering phenomena—multipath effects from irregular building surfaces, weather, vegetation, and human activity that elude deterministic simulation.

For developers building learning-based RF systems, the research suggests a hybrid validation approach: simulators enable cost-effective initial training and parameter exploration, but real-world fine-tuning or transfer learning remains essential for deployment. The open challenge of urban noise modeling presents both a research opportunity and a practical constraint for companies commercializing AI-driven cellular optimization, spectrum management, or localization services. Organizations must budget for substantial measurement campaigns to calibrate models rather than relying purely on physics-based simulation.

• →Antenna geometry and orientation prove far more critical than ray-tracing solver parameters for simulation fidelity
• →Optimized simulator achieved 5-130% improvement in signal correlation but still underperformed real data by 2x in localization tasks
• →Urban environmental noise and unmeasured scattering effects remain the primary barrier to high-fidelity outdoor RF simulation
• →Hybrid approaches combining simulation with real-world calibration are necessary for accurate machine learning on RF tasks
• →The study validates Sionna's technical soundness while exposing fundamental limits of deterministic electromagnetic modeling in complex environments