Pretrained Vision-Language-Action Models are Surprisingly Resistant to Forgetting in Continual Learning

Researchers discovered that pretrained Vision-Language-Action (VLA) models demonstrate remarkable resistance to catastrophic forgetting in continual learning scenarios, unlike smaller models trained from scratch. Simple Experience Replay techniques achieve near-zero forgetting with minimal replay data, suggesting large-scale pretraining fundamentally changes continual learning dynamics for robotics applications.

• →Pretrained VLA models show significantly better resistance to catastrophic forgetting compared to smaller policy models trained from scratch.
• →Simple Experience Replay achieves zero forgetting in VLAs even with small replay buffer sizes.
• →Large-scale pretraining enables models to maintain forward learning capabilities while mitigating forgetting.
• →VLAs can retain relevant knowledge from prior tasks and rapidly recover seemingly forgotten skills through finetuning.
• →The research suggests continual learning dynamics are fundamentally altered by large-scale pretraining in robotics applications.