Gradient-Free Training of Spiking Neural Networks via Low-Rank Evolution Strategies

Researchers introduce EGGROLL, a low-rank factorization technique that enables gradient-free training of Spiking Neural Networks (SNNs) using Evolution Strategies, reducing computational overhead by 2.23x while maintaining 79.21% accuracy on N-MNIST. This breakthrough addresses the long-standing challenge of training SNNs on neuromorphic hardware without requiring backpropagation infrastructure.

The development of EGGROLL represents a meaningful advance in neuromorphic computing by solving a fundamental efficiency problem in Spiking Neural Network training. SNNs promise significant energy savings on specialized hardware, but their discrete spike threshold creates a non-differentiable barrier that traditional gradient-based methods cannot directly optimize. While surrogate-gradient approaches have dominated recent work, they require backpropagation infrastructure incompatible with true on-chip learning scenarios where hardware constraints are severe.

Evolution Strategies offer an elegant alternative for hardware-constrained environments, but their computational demands scale linearly with parameter count, rendering them impractical for modern neural networks. EGGROLL's low-rank factorization converts the memory requirement from O(mn) to O(r(m+n)), enabling ES to scale to larger models. The 2.23x speedup with maintained accuracy demonstrates the practical viability of this approach.

This work opens pathways for neuromorphic accelerators to train models locally without external computation, reducing latency and power consumption in edge AI applications. For developers working with neuromorphic hardware like Intel's Loihi or emerging analog processors, this provides a concrete alternative to surrogate-gradient methods. The accuracy-speed tradeoff EGGROLL introduces is tunable based on rank selection, allowing practitioners to optimize for their specific hardware constraints.

Future research should explore EGGROLL's performance on larger datasets and more complex architectures to establish scalability bounds. Integration with commercial neuromorphic platforms would validate real-world applicability and potentially accelerate adoption of SNNs in energy-constrained computing scenarios.

• →EGGROLL reduces Evolution Strategies memory footprint from O(mn) to O(r(m+n)), enabling practical SNN training on neuromorphic hardware
• →Gradient-free training achieves 79.21% accuracy on N-MNIST while improving computational speed by 2.23x over full-rank ES
• →This approach eliminates dependency on backpropagation infrastructure, enabling true on-chip learning without surrogate gradients
• →The method introduces tunable accuracy-speed tradeoffs through low-rank factorization rank selection for hardware-specific optimization
• →Results demonstrate viability for edge AI and neuromorphic computing applications with severe power and latency constraints