Why Adam Can Beat SGD: Second-Moment Normalization Yields Sharper Tails

Research paper establishes the first theoretical separation between Adam and SGD optimization algorithms, proving Adam achieves better high-probability convergence guarantees. The study provides mathematical backing for Adam's superior empirical performance through second-moment normalization analysis.