8 articles tagged with #plasticity. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers have identified the mathematical mechanisms behind 'loss of plasticity' (LoP), explaining why deep learning models struggle to continue learning in changing environments. The study reveals that properties promoting generalization in static settings actually hinder continual learning by creating parameter space traps.
Researchers identify a critical problem in LLM post-training where excessive Supervised Fine-Tuning (SFT) reduces model plasticity, limiting subsequent Reinforcement Learning (RL) effectiveness. They propose 'Rejuvenation,' a method combining base-anchored model fusion and targeted neuron reset to restore plasticity while preserving SFT knowledge, demonstrating improved RL performance on reasoning and agentic tasks.
Researchers identify dynamical isometry—maintaining consistent layer-wise Jacobian singular values—as a mechanism for preserving neural network plasticity during continual learning under non-stationary conditions. They propose AdamO, an adaptive optimizer combining isometry regularization with gradient updates, demonstrating improved performance across supervised and reinforcement-learning benchmarks where traditional networks suffer progressive learning degradation.
Researchers identify that deep neural networks lose plasticity during continual learning due to Hessian spectral collapse, where curvature information vanishes and prevents gradient-based optimization. The study proposes regularization techniques combining high effective feature rank maintenance and L2 penalties to preserve learning capacity across sequential tasks.
Researchers from arXiv demonstrate that activation function design is crucial for maintaining neural network plasticity in continual learning scenarios. They introduce two new activation functions (Smooth-Leaky and Randomized Smooth-Leaky) that help prevent models from losing their ability to adapt to new tasks over time.
Researchers have introduced Spark, a new modular framework for spiking neural networks that aims to improve energy efficiency and data processing compared to traditional neural networks. The framework demonstrates its capabilities by solving complex problems like the sparse-reward cartpole using simple plasticity mechanisms, potentially advancing continuous learning approaches similar to biological systems.
Researchers developed a minimal AI architecture where a 'perspective latent' creates history-dependent perception in artificial agents. The system allows identical observations to be processed differently based on accumulated experience, demonstrating measurable plasticity that persists even after conditions return to normal.
Researchers propose DASP (Decoupling Adaptation for Stability and Plasticity), a novel framework for adapting multi-modal AI models to changing test environments. The method addresses key challenges of negative transfer and catastrophic forgetting by using asymmetric adaptation strategies that treat biased and unbiased modalities differently.
