AINeutralarXiv – CS AI · 7h ago6/10
🧠
Spectral Collapse Drives Loss of Plasticity in Deep Continual Learning
Researchers identify that deep neural networks lose plasticity during continual learning due to Hessian spectral collapse, where curvature information vanishes and prevents gradient-based optimization. The study proposes regularization techniques combining high effective feature rank maintenance and L2 penalties to preserve learning capacity across sequential tasks.