ChaosNetBench: Benchmarking Spatio-Temporal Graph Neural Networks on Chaotic Lattice Dynamics

Researchers introduce ChaosNetBench, a synthetic benchmark framework for evaluating spatio-temporal graph neural networks (STGNNs) on chaotic dynamical systems. The framework reveals that STGNNs outperform traditional baselines (TCN, N-BEATS, Transformers) in high-chaos regimes, while non-graph methods remain competitive in low-chaos conditions.

ChaosNetBench addresses a critical gap in machine learning evaluation methodology. Current STGNN benchmarks rely on fixed real-world datasets in single domains, limiting insights into how different architectures handle varying dynamical regimes. This synthetic framework introduces controlled chaos parameters—local chaos (K), coupling strength (ε), and system size (N)—across 96 instances and 9,600 trajectories, enabling reproducible, multidimensional analysis.

The research reflects broader maturation in AI evaluation practices. As neural networks increasingly model physical systems for weather prediction, traffic forecasting, and scientific computing, understanding performance boundaries becomes essential. Graph-based architectures like Graph WaveNet and STAEformer show resilience to chaos, while simpler temporal models excel under ordered conditions. This regime-dependent transition has practical implications for deployment decisions.

The framework's importance extends beyond academic rigor. Organizations deploying AI for dynamic system prediction—energy grids, climate modeling, transportation networks—benefit from systematic guidance on architecture selection based on underlying chaos levels. The work provides quantitative chaos indicators and evaluation protocols that standardize comparison methodology across different system conditions.

The synthetic nature of ChaosNetBench enables controlled experimentation impossible with real datasets. Researchers can isolate architectural strengths without domain-specific confounds. Future work likely involves testing emerging architectures, scaling to larger systems, and bridging synthetic benchmarks with real-world performance validation across diverse physical domains.

• →STGNNs demonstrate superior performance in chaotic regimes, while traditional baselines remain competitive under low-chaos conditions
• →ChaosNetBench provides the first standardized synthetic benchmark with tunable chaos parameters for systematic STGNN evaluation
• →The framework enables controlled multidimensional testing across 96 system instances, overcoming limitations of fixed real-world datasets
• →Graph-based architectures show greater resilience to both local and global chaos compared to non-graph temporal models
• →Regime-dependent performance transitions suggest architecture selection should depend on underlying system chaos levels