Task diversity produces systematic transfer but inhibits continual reinforcement learning

Researchers introduce Banyan, a benchmark for studying continual reinforcement learning that reveals task diversity improves immediate transfer between tasks but fails to sustain learning across multiple distribution shifts. While agents trained on diverse tasks generalize well to new task distributions, they forget earlier tasks and struggle with longer-horizon objectives as training continues.

The research addresses a fundamental challenge in reinforcement learning: reconciling the benefits of multi-task training with the requirements of continual learning in dynamic environments. Banyan's three-axis task diversity framework—map layouts, object interactions, and goal hierarchies—provides researchers with fine-grained control to isolate which factors enable transfer. This methodological contribution matters because previous studies relied on frozen-weight evaluation, masking whether learned representations remain plastic enough for ongoing adaptation.

The findings reveal a critical gap between zero-shot generalization and continual learning capability. Task diversity successfully positions agents near previously achieved performance levels when encountering new distributions, suggesting the learned representations capture transferable features. However, this local transfer breaks down systemically over longer timescales. Agents progressively forget earlier task distributions while plateau on complex tasks, indicating that diversity alone cannot overcome catastrophic forgetting and the exploration-exploitation tradeoff inherent in sequential learning.

For AI developers and researchers, Banyan offers a valuable diagnostic tool to test hypotheses about why current continual learning approaches fail despite strong single-task performance. The benchmark enables systematic investigation of whether forgetting stems from representation collapse, replay buffer limitations, or fundamental learning dynamics. This work influences algorithm design priorities: practitioners cannot assume that diverse pretraining solves continual learning problems and must instead develop complementary mechanisms like selective replay or dynamic task weighting to achieve sustained performance across distribution shifts.

• →Task diversity enables zero-shot transfer to new task distributions but does not guarantee sustained continual learning over multiple shifts.
• →Agents begin training new tasks near previous performance levels when task diversity increases, even with structural policy changes.
• →Longer-horizon tasks plateau and catastrophic forgetting occurs as training progresses across distribution shifts.
• →Banyan's three controllable diversity axes allow systematic benchmarking of transfer learning and continual learning trade-offs.
• →Current multi-task learning approaches require supplementary mechanisms beyond diversity to achieve proper continual learning.