#muon News & Analysis

2 articles tagged with #muon. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

2 articles

AINeutralarXiv – CS AI · Jun 235/10

🧠

Ky Fan Norms and Beyond: Dual Norms and Combinations for Matrix Optimization

Researchers introduce the Fanion family of optimization algorithms that extend beyond spectral norms used in the Muon optimizer, leveraging Ky Fan norm duals for matrix optimization in deep learning. Two variants, F-Muon and S-Muon, match or exceed Muon's performance across diverse tasks, with particular improvements on synthetic convex problems.

AINeutralarXiv – CS AI · May 126/10

🧠

Optimizer-Induced Mode Connectivity: From AdamW to Muon

Researchers demonstrate that neural network solutions trained with specific optimizers like AdamW and Muon form connected sets at large network widths, revealing optimizer-dependent structure in loss landscapes. The study shows that different optimizers converge to disconnected solutions with provable loss barriers in small networks, while empirically in GPT-2 pretraining, same-optimizer paths preserve model spectra differently than cross-optimizer paths.