#approximation-theory News & Analysis

3 articles tagged with #approximation-theory. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

3 articles

AIBullisharXiv – CS AI · Apr 107/10

🧠

Path Regularization: A Near-Complete and Optimal Nonasymptotic Generalization Theory for Multilayer Neural Networks and Double Descent Phenomenon

Researchers propose a new nonasymptotic generalization theory for multilayer neural networks using path regularization, proving near-minimax optimal error bounds without requiring unbounded loss functions or infinite network dimensions. The theory notably explains the double descent phenomenon and solves an open problem in approximation theory for neural networks.

AIBullisharXiv – CS AI · Mar 46/103

🧠

On the Expressive Power of Transformers for Maxout Networks and Continuous Piecewise Linear Functions

Researchers establish theoretical foundations for Transformer networks' expressive power by connecting them to maxout networks and continuous piecewise linear functions. The study proves Transformers inherit universal approximation capabilities of ReLU networks while revealing that self-attention layers implement max-type operations and feedforward layers perform token-wise affine transformations.

AINeutralarXiv – CS AI · Mar 47/102

🧠

Covering Numbers for Deep ReLU Networks with Applications to Function Approximation and Nonparametric Regression

Researchers have derived tight bounds on covering numbers for deep ReLU neural networks, providing fundamental insights into network capacity and approximation capabilities. The work removes a log^6(n) factor from the best known sample complexity rate for estimating Lipschitz functions via deep networks, establishing optimality in nonparametric regression.