AINeutralarXiv – CS AI · 14h ago5/10
🧠
Wolkowicz-Styan Upper Bound on the Hessian Eigenspectrum for Cross-Entropy Loss in Nonlinear Smooth Neural Networks
Researchers derive a closed-form upper bound for the Hessian eigenspectrum of cross-entropy loss in smooth nonlinear neural networks using the Wolkowicz-Styan bound. This analytical approach avoids numerical computation and expresses loss sharpness as a function of network parameters, training sample orthogonality, and layer dimensions—advancing theoretical understanding of the relationship between loss geometry and generalization.