Fly-CL: A Fly-Inspired Framework for Enhancing Efficient Decorrelation and Reduced Training Time in Pre-trained Model-based Continual Representation Learning

Researchers introduce Fly-CL, a bio-inspired framework for continual representation learning that significantly reduces training time while maintaining performance comparable to state-of-the-art methods. The approach, inspired by fly olfactory circuits, addresses multicollinearity issues in pre-trained models and enables more efficient similarity matching for real-time applications.

• →Fly-CL framework reduces training time while achieving performance equal to or better than current state-of-the-art continual learning methods.
• →The bio-inspired approach addresses multicollinearity problems in similarity-matching stages of pre-trained models.
• →Framework is compatible with a wide range of pre-trained backbones, making it broadly applicable.
• →Theoretical analysis demonstrates how Fly-CL progressively resolves multicollinearity with low time complexity.
• →Extensive experiments validate effectiveness across diverse network architectures and data regimes.