Topological Neural Dynamics: A Neuron-wise Framework for Sequence Modeling

Researchers introduce Topological Neural Dynamics (TND), a novel sequence modeling framework that replaces traditional layer-wise neural computation with neuron-wise dynamics where individual neurons evolve independently through explicit graph topology. In a Pong behavior cloning benchmark, TND outperforms RNNs, LSTMs, continuous-time networks, and Transformers with a catch rate more than three times higher than the strongest baseline, suggesting this architectural approach offers a more effective inductive bias for sequence modeling.

The research presents a fundamental shift in how neural networks can be architected for sequence modeling tasks. Rather than forcing all neurons in a layer to share computational dynamics through a single parameterized operator, TND allows each neuron to evolve according to its own local dynamics function while maintaining structured interactions through an explicit directed graph topology. This mirrors biological neural systems and complex dynamical systems where global intelligence emerges from locally-operating units.

The innovation builds on a longstanding observation in dynamical systems research: that rich, complex behavior often arises not from top-down, globally-coordinated operations but from interactions between independently-evolving local components. Current dominant architectures—from RNNs to Transformers—all enforce layer-wise synchronization, which may unnecessarily constrain how information flows and transforms through the network.

The empirical validation on a Pong behavior cloning task demonstrates substantial practical advantages. Achieving a mean of 17.47 consecutive catches versus the next-best baseline's ~5 catches represents a dramatic performance improvement on a task requiring sequential decision-making and temporal reasoning. This suggests neuron-wise dynamics may be particularly valuable for embodied AI and control tasks.

For the broader AI community, this work challenges architectural assumptions baked into decades of deep learning practice. If validated across more diverse benchmarks and datasets, TND could inspire a new class of architectures that decouple neurons from layer-level synchronization constraints. The framework remains preliminary—limited to discrete-time formulations and single benchmark evaluation—but signals an underexplored design space in neural architecture research.

• →TND replaces layer-wise neural computation with independent neuron-wise dynamics connected through explicit graph topology, achieving 3x performance gains on Pong behavior cloning.
• →The approach mirrors biological and complex dynamical systems where global behavior emerges from locally-evolving units with structured interactions.
• →Current standard architectures (RNNs, LSTMs, Transformers) enforce layer-wise synchronization that may unnecessarily constrain information flow and transformation.
• →The framework demonstrates particular promise for sequential decision-making and control tasks requiring temporal reasoning.
• →Results remain preliminary and limited to single benchmark, requiring broader validation across diverse tasks before architectural paradigm shift occurs.