AINeutralarXiv – CS AI · 8h ago6/10
🧠
A Geometric Characterization of the Stationary Plateau for Two-Layer Neural Networks
Researchers characterize the geometric structure of loss landscape plateaus in two-layer neural networks, focusing on how duplicating hidden neurons creates affine sets of stationary points. The study classifies whether these plateau points are local minima or saddles based on an 'inner Hessian' matrix, revealing that splitting a minimum can produce mixed or all-saddle plateaus, while splitting saddles always yields saddle plateaus.