Implicit Bias of Per-sample Adam on Separable Data: Departure from the Full-batch Regime

New research reveals that per-sample Adam optimizer's implicit bias differs significantly from full-batch Adam in machine learning training. The study shows incremental Adam can converge to different solutions than expected, potentially impacting AI model optimization strategies.