#implicit-bias2 articles

2 articles

AINeutralarXiv – CS AI · 5h ago

🧠

Implicit Bias of Per-sample Adam on Separable Data: Departure from the Full-batch Regime

New research reveals that per-sample Adam optimizer's implicit bias differs significantly from full-batch Adam in machine learning training. The study shows incremental Adam can converge to different solutions than expected, potentially impacting AI model optimization strategies.

AINeutralarXiv – CS AI · 5h ago

🧠

Implicit Bias of the JKO Scheme

Researchers analyzed the implicit bias of the Jordan-Kinderlehrer-Otto (JKO) scheme, a time-discretization method for Wasserstein gradient flow used in optimizing energy functionals over probability measures. They found that the JKO scheme adds a deceleration term at second order that corresponds to canonical implicit biases like Fisher information for entropy and kinetic energy for Riemannian gradient descent.