AINeutralarXiv – CS AI · Jun 117/10
🧠A comprehensive survey examines Federated Continual Learning (FCL), which combines federated learning's privacy-preserving distributed training with continual learning's ability to adapt to evolving data. The research addresses a critical gap in current FL systems that assume static data, proposing frameworks for real-world applications like healthcare and IoT where data streams continuously shift, causing performance degradation and catastrophic forgetting.
AIBullisharXiv – CS AI · Jun 97/10
🧠Researchers demonstrate that artificial neural networks can mitigate catastrophic forgetting—the tendency to lose previously learned information when training on new tasks—by applying unsupervised replay mechanisms after sequential learning periods, mimicking biological sleep-based memory consolidation. This approach defers interference correction until after multiple new tasks are learned, suggesting a more efficient pathway for developing continual learning AI systems.
AIBullisharXiv – CS AI · Jun 27/10
🧠Researchers introduce RAFT, a framework addressing the problem of catastrophic forgetting in domain-specific fine-tuning of language models. By combining data refinement with answer-conditioned distillation, RAFT achieves 23.2% improvement in domain accuracy while recovering 10-18% of general capability losses typically incurred during fine-tuning.
AIBullisharXiv – CS AI · May 297/10
🧠Researchers demonstrate that Evolution Strategies (ES) can effectively fine-tune large language models without catastrophic forgetting of prior tasks, contrary to recent concerns. By introducing Anchored Weight Decay (AWD), a regularization technique that constrains optimization toward initial parameters, the work shows ES-based continual learning is viable and computationally efficient compared to reinforcement learning approaches.
AIBullisharXiv – CS AI · May 97/10
🧠Researchers present a statistical-physics framework explaining how large language models develop multi-step reasoning through reinforcement learning with verifiable rewards (RLVR), modeling the process as inverse tree freezing in a concept network. They propose Annealed-RLVR, a timing-optimized training method that outperforms standard RLVR by applying supervised fine-tuning at peak frustration rather than after convergence, preventing policy collapse.
AIBullisharXiv – CS AI · May 77/10
🧠Researchers propose a novel framework that models language model memory as a Markov transition matrix, enabling efficient incorporation of new knowledge without catastrophic forgetting. The approach requires only linear sample complexity in the number of existing tokens and achieves zero forgetting through minimal parameter updates via an embedding-tuning algorithm.
AIBullisharXiv – CS AI · May 77/10
🧠Researchers propose Anchored Learning, a new fine-tuning method that prevents catastrophic forgetting in large language models by controlling distributional drift through a dynamically evolving reference anchor. The technique achieves near-optimal performance gains while reducing degradation from over 53% to under 5% on benchmark tasks.
AIBullisharXiv – CS AI · May 77/10
🧠Researchers propose skill neologisms—soft tokens added to LLM vocabularies—as a scalable approach to continual learning that enables models to acquire new capabilities without catastrophic forgetting or weight updates. The method demonstrates that independently trained skill tokens can compose zero-shot and work with out-of-distribution tasks, offering a practical alternative to fine-tuning.
AIBullisharXiv – CS AI · Apr 147/10
🧠Researchers propose VaCoAl, a hyperdimensional computing architecture that combines sparse distributed memory with Galois-field algebra to address limitations in modern AI systems like catastrophic forgetting and the binding problem. The deterministic system demonstrates emergent properties equivalent to spike-timing-dependent plasticity and achieves multi-hop reasoning across 25.5M paths in knowledge graphs, positioning it as a complementary third paradigm to large language models.
AIBullisharXiv – CS AI · Apr 147/10
🧠Researchers introduce soul.py, an open-source architecture addressing catastrophic forgetting in AI agents by distributing identity across multiple memory systems rather than centralizing it. The framework implements persistent identity through separable components and a hybrid RAG+RLM retrieval system, drawing inspiration from how human memory survives neurological damage.
AINeutralarXiv – CS AI · Apr 107/10
🧠Researchers introduce the Informational Buildup Framework (IBF), a new approach to continual learning that eliminates catastrophic forgetting by treating information as structural alignment rather than stored parameters. The framework demonstrates superior performance across multiple domains including chess and image classification, achieving near-zero forgetting without requiring raw data replay.
AIBullisharXiv – CS AI · Mar 177/10
🧠Researchers introduce SCAN, a new framework for editing Large Language Models that prevents catastrophic forgetting during sequential knowledge updates. The method uses sparse circuit manipulation instead of dense parameter changes, maintaining model performance even after 3,000 sequential edits across major models like Gemma2, Qwen3, and Llama3.1.
🧠 Llama
AIBullisharXiv – CS AI · Mar 56/10
🧠Researchers discovered that pretrained Vision-Language-Action (VLA) models demonstrate remarkable resistance to catastrophic forgetting in continual learning scenarios, unlike smaller models trained from scratch. Simple Experience Replay techniques achieve near-zero forgetting with minimal replay data, suggesting large-scale pretraining fundamentally changes continual learning dynamics for robotics applications.
AIBullisharXiv – CS AI · Mar 47/103
🧠Researchers have identified a critical flaw in reinforcement learning fine-tuning of large language models that causes degradation in multi-attempt performance despite improvements in single attempts. Their proposed solution, Diversity-Preserving Hybrid RL (DPH-RL), uses mass-covering f-divergences to maintain model diversity and prevent catastrophic forgetting while improving training efficiency.
AIBullisharXiv – CS AI · Mar 46/103
🧠Researchers developed cPNN (Continuous Progressive Neural Networks), a new AI architecture that handles evolving data streams with temporal dependencies while avoiding catastrophic forgetting. The system addresses concept drift in time series data by combining recurrent neural networks with progressive learning techniques, showing quick adaptation to new concepts.
AIBullisharXiv – CS AI · Mar 37/103
🧠Researchers introduce Dream2Learn (D2L), a continual learning framework that enables AI models to generate synthetic training data from their own internal representations, mimicking human dreaming for knowledge consolidation. The system creates novel 'dreamed classes' using diffusion models to improve forward knowledge transfer and prevent catastrophic forgetting in neural networks.
AIBullisharXiv – CS AI · Feb 277/106
🧠Researchers introduce GraftLLM, a new method for transferring knowledge between large language models using 'SkillPack' format that preserves capabilities while avoiding catastrophic forgetting. The approach enables efficient model fusion and continual learning for heterogeneous models through modular knowledge storage.
AIBullisharXiv – CS AI · Jun 236/10
🧠Researchers propose Attention-Spectrum Regularization (ASR), a new continual learning framework for multimodal large language models that prevents catastrophic forgetting when adapting to new visual domains and tasks without replaying past data. ASR preserves cross-modal attention patterns by storing compact spectral statistics rather than actual training examples, demonstrating improved performance on vision-language benchmarks.
AINeutralarXiv – CS AI · Jun 236/10
🧠Researchers identify 'scientific amnesia' as a critical failure mode in continual DPO (Direct Preference Optimization) training pipelines where LLMs preserve learned behaviors but fail to accumulate reusable methodological knowledge across sequential training campaigns. Testing five strategy proposers on a 30-campaign benchmark reveals that most approaches degrade performance, with only conservative rule-based scheduling showing consistent improvement.
AINeutralarXiv – CS AI · Jun 236/10
🧠Researchers introduce TASER, a continual learning framework designed to handle highly heterogeneous tasks by dynamically expanding atomic skills and routing them based on task requirements. The work addresses catastrophic forgetting in AI systems learning sequential tasks with diverse reasoning patterns, validated on a new benchmark called HeteroCLBench comprising 19 tasks across 9 cognitive dimensions.
AINeutralarXiv – CS AI · Jun 236/10
🧠Researchers propose RECALL, an active learning framework for Vision-Language-Action (VLA) models that uses uncertainty-guided data collection to improve robot learning efficiency. While targeted recovery demonstrations outperform passive imitation learning, the approach reveals critical challenges with catastrophic forgetting when new data isn't balanced with retention mechanisms.
AINeutralarXiv – CS AI · Jun 236/10
🧠Researchers present CADRE, a parameter-efficient adaptation framework for medical vision-language models that addresses catastrophic forgetting and model drift when updating deployed systems. By combining low-rank adaptation with elastic weight consolidation and prior-anchoring penalties, CADRE reduces forgetting sevenfold while training only 0.23% of parameters, demonstrating improved stability across different medical imaging modalities.
AINeutralarXiv – CS AI · Jun 236/10
🧠HERMAN introduces a hierarchical representation matching framework for CLIP-based class-incremental learning, using LLM-generated textual descriptors to capture multi-level semantic relationships. The approach addresses limitations in existing vision-language models by leveraging hierarchical visual concepts rather than simplistic templates, demonstrating improved performance on multiple benchmarks.
AINeutralarXiv – CS AI · Jun 196/10
🧠LOKI is a new method for lifelong knowledge editing in language models that dynamically selects which layers to update and avoids catastrophic forgetting without requiring access to previous training data. The approach achieves up to 14% improvement in accuracy over existing methods by using the Hilbert-Schmidt Independence Criterion and null-space projection techniques.
AINeutralarXiv – CS AI · Jun 86/10
🧠Researchers introduce SETA, a machine learning framework that addresses catastrophic forgetting in large language models through sparse expert decomposition. The method separates task-specific and shared knowledge into distinct expert modules, enabling models to retain previous capabilities while learning new ones—a fundamental challenge in continual AI development.
