AINeutralarXiv – CS AI · 3h ago6/10
🧠
Stochastic Gradient Descent with Momentum is Algorithmically Stable
Researchers have demonstrated that Stochastic Gradient Descent with Momentum (SGDM), a fundamental optimization algorithm in machine learning, maintains strong generalization properties through algorithmic stability analysis. The study resolves a longstanding conjecture that momentum, while accelerating training, might harm generalization performance, providing tight stability bounds applicable to both Polyak's and Nesterov's momentum schemes.