#approximation-theory News & Analysis

5 articles tagged with #approximation-theory. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

5 articles

AIBullisharXiv – CS AI · Apr 207/10

🧠

Closing the Theory-Practice Gap in Spiking Transformers via Effective Dimension

Researchers establish the first comprehensive theoretical framework for spiking transformers, proving their universal approximation capabilities and deriving tight spike-count lower bounds. Using effective dimension analysis, they explain why spiking transformers achieve 38-57× energy efficiency on neuromorphic hardware and provide concrete design rules validated across vision and language benchmarks with 97% prediction accuracy.

AIBullisharXiv – CS AI · Apr 107/10

🧠

Path Regularization: A Near-Complete and Optimal Nonasymptotic Generalization Theory for Multilayer Neural Networks and Double Descent Phenomenon

Researchers propose a new nonasymptotic generalization theory for multilayer neural networks using path regularization, proving near-minimax optimal error bounds without requiring unbounded loss functions or infinite network dimensions. The theory notably explains the double descent phenomenon and solves an open problem in approximation theory for neural networks.

AIBullisharXiv – CS AI · Mar 46/103

🧠

On the Expressive Power of Transformers for Maxout Networks and Continuous Piecewise Linear Functions

Researchers establish theoretical foundations for Transformer networks' expressive power by connecting them to maxout networks and continuous piecewise linear functions. The study proves Transformers inherit universal approximation capabilities of ReLU networks while revealing that self-attention layers implement max-type operations and feedforward layers perform token-wise affine transformations.

AINeutralarXiv – CS AI · Mar 47/102

🧠

Covering Numbers for Deep ReLU Networks with Applications to Function Approximation and Nonparametric Regression

Researchers have derived tight bounds on covering numbers for deep ReLU neural networks, providing fundamental insights into network capacity and approximation capabilities. The work removes a log^6(n) factor from the best known sample complexity rate for estimating Lipschitz functions via deep networks, establishing optimality in nonparametric regression.

AINeutralarXiv – CS AI · May 126/10

🧠

Efficient Ensemble Selection from Binary and Pairwise Feedback

Researchers present new algorithms for efficiently selecting small, high-performing ensembles of AI systems using minimal model evaluations. The work addresses both binary feedback (correct/incorrect outcomes) and pairwise feedback (preference comparisons), providing theoretical guarantees and practical query-saving methods validated through LLM experiments.

$ETH