Parameter-Efficient Fine-Tuning for Continual Learning: A Neural Tangent Kernel Perspective

Researchers introduce NTK-CL, a new framework for parameter-efficient fine-tuning in continual learning that uses Neural Tangent Kernel theory to address catastrophic forgetting. The approach achieves state-of-the-art performance by tripling feature representation and implementing adaptive mechanisms to maintain task-specific knowledge while learning new tasks.

• →NTK-CL framework eliminates task-specific parameter storage while adaptively generating task-relevant features for continual learning.
• →The research identifies three key factors affecting continual learning performance: training sample size, task-level feature orthogonality, and regularization.
• →The framework triples feature representation of each sample, reducing both task-interplay and task-specific generalization gaps.
• →NTK-CL achieves state-of-the-art performance on established parameter-efficient continual learning benchmarks.
• →The work provides theoretical foundation using Neural Tangent Kernel theory to understand and improve continual learning systems.