#mathematical-analysis News & Analysis

5 articles tagged with #mathematical-analysis. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

5 articles

AINeutralarXiv – CS AI · May 126/10

🧠

Scaling Limits of Long-Context Transformers

Researchers present a theoretical analysis of how transformer attention mechanisms scale with context length, identifying a critical threshold where attention shifts from uniform averaging to focusing on individual keys. The findings establish that this transition point depends on local geometric properties of the key distribution rather than global features, with implications for understanding transformer behavior at extreme context lengths.

AINeutralarXiv – CS AI · Apr 145/10

🧠

Wolkowicz-Styan Upper Bound on the Hessian Eigenspectrum for Cross-Entropy Loss in Nonlinear Smooth Neural Networks

Researchers derive a closed-form upper bound for the Hessian eigenspectrum of cross-entropy loss in smooth nonlinear neural networks using the Wolkowicz-Styan bound. This analytical approach avoids numerical computation and expresses loss sharpness as a function of network parameters, training sample orthogonality, and layer dimensions—advancing theoretical understanding of the relationship between loss geometry and generalization.

AINeutralarXiv – CS AI · Mar 54/10

🧠

Implicit Bias of the JKO Scheme

Researchers analyzed the implicit bias of the Jordan-Kinderlehrer-Otto (JKO) scheme, a time-discretization method for Wasserstein gradient flow used in optimizing energy functionals over probability measures. They found that the JKO scheme adds a deceleration term at second order that corresponds to canonical implicit biases like Fisher information for entropy and kinetic energy for Riemannian gradient descent.

AINeutralarXiv – CS AI · Feb 274/104

🧠

A 1/R Law for Kurtosis Contrast in Balanced Mixtures

Researchers prove a mathematical law showing that kurtosis-based Independent Component Analysis (ICA) becomes less effective in wide, balanced mixtures due to contrast decay following a 1/R relationship. The study demonstrates that purification techniques can restore contrast performance and provides theoretical bounds for practical implementation.

AINeutralarXiv – CS AI · Mar 24/106

🧠

Geometric structure of shallow neural networks and constructive ${\mathcal L}^2$ cost minimization

Researchers developed a new approach to minimize cost functions in shallow ReLU neural networks through explicit construction rather than gradient descent. The study provides mathematical upper bounds for cost minimization and characterizes the geometric structure of network minimizers in classification tasks.