AIBullisharXiv – CS AI · 10h ago7/10
🧠
Scaling Linear Mode Connectivity and Merging to Billion Parameter Pretrained Transformers
Researchers propose a scalable framework for linear mode connectivity (LMC) that enables merging of billion-parameter pretrained transformers through dual bidirectional optimization. The method achieves near-zero loss barriers on language models and maintains strong performance on vision models, demonstrating that resolving parameter symmetries allows large AI models to be merged via simple linear interpolation paths.