CLANE: Continual Learning of Actions on Neuromorphic Hardware from Event Cameras

Researchers have developed CLANE, a neuromorphic hardware system deployed on Intel Loihi 2 that enables continuous learning of human actions from event cameras without forgetting previously learned classes. The system achieves 70.4% accuracy on a 50-class action recognition dataset while consuming 100x less energy and delivering 16x lower latency than conventional GPU-based approaches, advancing on-device AI for AR/VR and robotics applications.

CLANE represents a significant advancement in edge AI by successfully combining neuromorphic computing with continual learning—a notoriously difficult problem where systems must learn new information without degrading performance on previously learned tasks. The deployment on Intel Loihi 2 neuromorphic hardware is particularly notable because it demonstrates that spiking neural networks can handle complex, real-world action recognition tasks while maintaining the energy efficiency advantages that make them attractive for embedded systems.

The underlying innovation addresses a critical gap in robotics and AR/VR applications, where privacy-sensitive visual processing and rapid adaptation to new scenarios are essential requirements. Event cameras, which capture visual information asynchronously and sparsely, align naturally with neuromorphic processing paradigms, yet integrating them into practical continual learning pipelines has proven challenging. CLANE's architecture—combining 2D spiking CNNs with custom modules like the Temporal Aggregation Layer and fixed-point Normalization Layer—shows how specialized hardware can be optimized for specific computational patterns rather than forcing neuromorphic systems into conventional deep learning frameworks.

The performance metrics indicate substantial practical benefits for deployment scenarios. A 100x energy reduction relative to GPU baselines and 16x latency improvements make on-device learning feasible for battery-powered robots and AR headsets. The 70.4% accuracy on THU E-ACT-50 under real-world conditions, rather than controlled laboratory settings, suggests the approach generalizes beyond synthetic benchmarks. This work establishes neuromorphic hardware as increasingly viable for sophisticated AI tasks previously dominated by conventional processors, potentially reshaping hardware requirements for next-generation robotics and spatial computing applications.

• →CLANE successfully demonstrates end-to-end continual learning on neuromorphic hardware (Intel Loihi 2), solving a previously undeployed application
• →System achieves 100x energy reduction and 16x lower latency compared to conventional GPU-based CNN+GRU approaches for the same task
• →Event cameras paired with spiking neural networks provide natural computational alignment for efficient, asynchronous visual processing
• →Real-world performance on 50-class action recognition validates neuromorphic approaches beyond controlled laboratory benchmarks
• →Specialized hardware modules for temporal aggregation and fixed-point normalization enable practical neuromorphic solutions rather than generic implementations