Advancing Direct Training for Spiking Neural Networks with Circulate-Firing Neurons and Learnable Gradients

Researchers propose a novel direct training algorithm for Spiking Neural Networks that addresses performance gaps with traditional ANNs through circulate-firing neurons, learnable surrogate gradients, and balanced loss functions. The method demonstrates competitive results across datasets and extends effectively to Transformer architectures, potentially advancing energy-efficient neural network applications.

This research tackles a fundamental challenge in neuromorphic computing: closing the performance gap between biologically-inspired Spiking Neural Networks and conventional Artificial Neural Networks. SNNs promise significant energy efficiency benefits, critical for edge computing and specialized hardware, but have historically underperformed on complex tasks. The proposed innovations address core technical limitations that have constrained SNN development.

The circulate-firing neuron model represents a meaningful architectural advance by better leveraging membrane potential dynamics—the electrostatic behavior that forms the computational basis of SNNs. Traditional spiking neurons waste information by treating membrane potentials simplistically. The introduction of time-step-wise learnable surrogate gradients addresses a training bottleneck: previous methods used fixed gradient approximations during backpropagation, creating systematic estimation errors. Learnable gradients enable the network to optimize its own gradient computation pathway, analogous to how modern deep learning frameworks adapt their optimization strategies.

The balanced loss function demonstrates awareness of SNN-specific training dynamics, where asymmetric potential states can lead to suboptimal convergence. Compatibility with Transformer architectures signals broader applicability across modern deep learning paradigms.

For the neuromorphic and edge AI sectors, this work validates that SNN performance limitations stem from training methodology rather than fundamental computational constraints. Successful application to Transformers opens pathways for deploying efficient large models on resource-constrained devices. The research strengthens the commercial case for neuromorphic hardware manufacturers and enables new applications in robotics, autonomous systems, and IoT.

• →Circulate-firing neurons enhance information representation by better utilizing membrane potential dynamics in SNNs
• →Learnable surrogate gradients enable more precise backpropagation compared to fixed gradient functions across time steps
• →The proposed method achieves competitive performance on multiple datasets and generalizes to Transformer architectures
• →Research validates that SNN performance gaps are addressable through improved training algorithms rather than fundamental architectural constraints
• →Results support commercial viability of neuromorphic hardware for energy-efficient AI applications