Task-Differentiated Atomic Skill Expansion and Routing for Continual Learning Across Highly Heterogeneous Tasks

Researchers introduce TASER, a continual learning framework designed to handle highly heterogeneous tasks by dynamically expanding atomic skills and routing them based on task requirements. The work addresses catastrophic forgetting in AI systems learning sequential tasks with diverse reasoning patterns, validated on a new benchmark called HeteroCLBench comprising 19 tasks across 9 cognitive dimensions.

This research tackles a fundamental challenge in machine learning: enabling systems to learn multiple diverse tasks sequentially without forgetting previous knowledge or wasting computational capacity. Traditional continual learning assumes tasks are semantically related, but real-world applications often involve drastically different task types—reasoning, language, vision, and more. TASER's contribution lies in its three-part architecture: determining when to introduce new capabilities based on task divergence, maintaining semantic distinctness through orthogonality constraints, and dynamically composing relevant skills via task-conditioned gating.

The problem addresses an increasingly important frontier in AI development. As models scale and tackle broader domains, the ability to learn continuously without catastrophic forgetting becomes critical for efficient deployment. Current methods waste capacity by either over-sharing parameters across dissimilar tasks or maintaining entirely separate models. The introduction of HeteroCLBench standardizes evaluation across 19 diverse tasks, providing the research community with a rigorous testing ground previously lacking in continual learning literature.

For AI practitioners and organizations, this framework could reduce training overhead and improve model efficiency when deploying systems across multiple domains. The approach's emphasis on task-differentiated expansion suggests future AI systems could intelligently allocate resources proportional to task complexity rather than using fixed architectures. The research signals broader industry movement toward more adaptive, efficient learning paradigms that mimic how humans selectively activate knowledge based on context.

• →TASER dynamically determines skill expansion based on task divergence and model uncertainty rather than using fixed capacity allocation
• →Orthogonality-enhanced skill detection ensures learned skills remain semantically distinct and independently reusable across tasks
• →HeteroCLBench provides a standardized benchmark with 19 diverse tasks spanning 9 cognitive dimensions for evaluating continual learning systems
• →The framework outperforms existing baselines by improving plasticity while reducing catastrophic forgetting in highly heterogeneous task sequences
• →Task-conditioned gating mechanism enables lightweight composition of relevant skills without maintaining separate specialized models