Unifying Dynamical Systems and Graph Theory to Mechanistically Understand Computation in Neural Networks

Researchers demonstrate that recurrent neural networks implement computation through multi-hop pathways across graph structures rather than direct connections alone. They introduce resolvent-RNNs (R-RNNs) that constrain these pathways to achieve better temporal sparsity and robustness than traditional L1 regularization, revealing fundamental principles about how neural networks process information.

This research advances fundamental understanding of how neural networks translate structure into computation, bridging neuroscience and machine learning. The key insight—that multi-hop pathways matter more than individual weights—challenges conventional approaches to regularization and network optimization. The authors show that standard regularization techniques like L1 penalties operate at the wrong level of abstraction, constraining single connections rather than the functional communication patterns that actually drive computation.

The development of resolvent-RNNs represents a meaningful methodological shift. By targeting multi-hop pathways instead of individual parameters, R-RNNs induce temporal sparsity aligned with task structure while maintaining superior robustness under strong regularization. This finding suggests that neural network efficiency gains come not from randomly removing weights, but from systematically constraining information flow pathways. The work builds on growing recognition that structural and functional connectivity diverge in both biological and artificial systems, necessitating more sophisticated analysis frameworks.

For the broader machine learning community, this research offers practical implications for network design and training. Better understanding of how networks route information temporally could inform architecture choices and regularization strategies across domains from neuroscience to AI systems. The R-RNN approach may enable more efficient models with fewer parameters without sacrificing performance, particularly valuable for resource-constrained applications. The theoretical framework—connecting dynamical systems, graph theory, and information routing—provides tools for mechanistic interpretation of network behavior, moving beyond black-box analysis toward interpretable computation.

• →Neural networks implement computation through multi-hop information pathways across graph structures, not just direct connections.
• →Resolvent-RNNs constrain multi-hop pathways rather than individual weights, achieving better temporal sparsity than L1 regularization.
• →Standard regularization techniques operate at an incorrect level of abstraction for optimizing network computation.
• →The framework connects dynamical systems and graph theory to mechanistically understand how networks process hierarchically structured tasks.
• →Better sparsity-function alignment improves robustness and enables more efficient neural network design.