y0news
AnalyticsDigestsSourcesTopicsRSSAICrypto

#mathematical-analysis News & Analysis

8 articles tagged with #mathematical-analysis. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

8 articles
AINeutralarXiv – CS AI · Jun 85/10
🧠

A Temporal Spatial Minimax Rate for Smoothly-Varying Distributions in Wasserstein Space

A new mathematical framework establishes minimax rates for predicting future probability distributions in Wasserstein space based on noisy observations of smoothly-varying curves. The research provides both lower bounds and conditional upper bounds for distribution estimation, revealing how prediction accuracy degrades with dimensionality and unobserved future time horizons.

AINeutralarXiv – CS AI · Jun 56/10
🧠

Deciphering Two Training Clocks in Grokking via Deep Linear Network Theory with Conditional ReLU Reduction

Researchers formalize the grokking phenomenon—where neural networks fit training data quickly but learn generalizable rules slowly—by analyzing deep linear networks and ReLU MLPs. The study identifies two distinct training timescales: fast classification loss decay and slower representation simplification, with implications for understanding how neural networks generalize.

AINeutralarXiv – CS AI · Jun 46/10
🧠

Why Muon Outperforms Adam: A Curvature Perspective

Researchers demonstrate that Muon, an optimizer for large language model training, outperforms Adam by approximately 2x efficiency through lower Normalized Directional Sharpness (NDS) rather than smaller update scales. Using curvature analysis and stylized quadratic problems, the work reveals that Muon's advantage stems from better balancing of update energy across heterogeneous curvature regions, with benefits amplified in data-imbalanced scenarios.

AINeutralarXiv – CS AI · May 126/10
🧠

Scaling Limits of Long-Context Transformers

Researchers present a theoretical analysis of how transformer attention mechanisms scale with context length, identifying a critical threshold where attention shifts from uniform averaging to focusing on individual keys. The findings establish that this transition point depends on local geometric properties of the key distribution rather than global features, with implications for understanding transformer behavior at extreme context lengths.

AINeutralarXiv – CS AI · Apr 145/10
🧠

Wolkowicz-Styan Upper Bound on the Hessian Eigenspectrum for Cross-Entropy Loss in Nonlinear Smooth Neural Networks

Researchers derive a closed-form upper bound for the Hessian eigenspectrum of cross-entropy loss in smooth nonlinear neural networks using the Wolkowicz-Styan bound. This analytical approach avoids numerical computation and expresses loss sharpness as a function of network parameters, training sample orthogonality, and layer dimensions—advancing theoretical understanding of the relationship between loss geometry and generalization.

AINeutralarXiv – CS AI · Mar 54/10
🧠

Implicit Bias of the JKO Scheme

Researchers analyzed the implicit bias of the Jordan-Kinderlehrer-Otto (JKO) scheme, a time-discretization method for Wasserstein gradient flow used in optimizing energy functionals over probability measures. They found that the JKO scheme adds a deceleration term at second order that corresponds to canonical implicit biases like Fisher information for entropy and kinetic energy for Riemannian gradient descent.

AINeutralarXiv – CS AI · Feb 274/104
🧠

A 1/R Law for Kurtosis Contrast in Balanced Mixtures

Researchers prove a mathematical law showing that kurtosis-based Independent Component Analysis (ICA) becomes less effective in wide, balanced mixtures due to contrast decay following a 1/R relationship. The study demonstrates that purification techniques can restore contrast performance and provides theoretical bounds for practical implementation.