STARS: Spike Tail-Aware Relational Synthesis for ANN-to-SNN Data-Free Knowledge Distillation

Researchers introduce STARS, a data-free knowledge distillation method that improves the transfer of learning from artificial neural networks (ANNs) to spiking neural networks (SNNs) without access to original training data. The technique combines batch normalization matching with relational consistency and threshold-aware regularization, achieving significant accuracy improvements across standard benchmarks.

This research addresses a critical challenge in neuromorphic computing: enabling SNNs to match ANN performance while maintaining their energy efficiency advantages. SNNs represent a paradigm shift toward brain-inspired computing with substantially lower power consumption, but their adoption requires overcoming a performance gap. The paper's contribution lies in recognizing that existing data-free knowledge distillation approaches—which rely on batch normalization statistics—inadequately capture the spike-based dynamics that define SNN behavior.

The STARS method introduces two key innovations. Relational Consistency Alignment preserves cross-sample relationships between teacher and student networks, ensuring that the synthetic training data maintains meaningful structure. Tail-Aware Regularization directly addresses SNN-specific requirements by focusing on threshold-crossing probabilities, the mechanism underlying spike generation. This represents a fundamental insight: SNN students require different guidance than conventional neural networks because their decision boundaries depend on temporal threshold dynamics rather than continuous activations.

The experimental validation spans multiple datasets (CIFAR-10, CIFAR-100, Tiny-ImageNet) and demonstrates substantial improvements—up to 6.7% accuracy gains on CIFAR-100. These results suggest the method's robustness across different problem scales. For the broader neuromorphic computing community, this work validates that SNN-specific constraints can meaningfully improve knowledge transfer without access to original training data, a practical constraint in many deployment scenarios.

Looking forward, this research strengthens the viability of SNNs for energy-constrained applications where training data is proprietary or unavailable. The methodology could enable wider adoption of neuromorphic hardware by reducing deployment friction, particularly in edge computing and IoT contexts where both power efficiency and data privacy matter.

• →STARS improves ANN-to-SNN knowledge distillation without original training data using threshold-aware constraints
• →Relational consistency and tail-aware regularization directly address SNN-specific spike dynamics beyond standard batch normalization
• →Experiments show up to 6.7% accuracy improvements on CIFAR-100 over conventional data-free distillation baselines
• →The method works as a plug-and-play enhancement compatible with existing neuromorphic training pipelines
• →Results strengthen neuromorphic computing viability for edge deployments where training data unavailability is common