#deep-learning-optimization News & Analysis

5 articles tagged with #deep-learning-optimization. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

5 articles

AIBullisharXiv – CS AI · Jun 97/10

🧠

RAPID: Layer-Wise Redundancy-Aware Pruning and Importance-Driven Token Merging for Efficient ViT

Researchers introduce RAPID, a depth-aware token reduction framework for Vision Transformers that uses different pruning and merging strategies across network layers to reduce computational costs while maintaining accuracy. The method achieves superior performance compared to existing approaches like ToMe, with up to 4.29% higher accuracy in aggressive compression scenarios.

AIBullisharXiv – CS AI · Jun 27/10

🧠

Quantum Tunneling-Aware Machine Learning: Physics-Derived Noise Models for Robust Deployment

Researchers introduce Quantum Tunneling-Aware Machine Learning (QTAML), a physics-based approach to model electron leakage errors in AI chips as transistors scale toward quantum limits. The method achieves 95% accuracy while reducing error-correction overhead by 3.4x to 33.6x compared to conventional approaches, with no retraining or inference-time costs.

AINeutralarXiv – CS AI · Jun 96/10

🧠

Structured Neuron Pruning in Deep Neural Networks Using Multi-Armed Bandits

Researchers present a novel structured pruning framework that uses multi-armed bandit algorithms to remove redundant neurons from deep neural networks. The approach treats each neuron as a bandit arm, testing its importance through temporary masking and loss measurement, then applies various MAB policies (UCB1, Thompson Sampling, etc.) to identify which neurons to prune. Experiments across tabular and deep learning tasks show MAB-based pruning significantly outperforms traditional magnitude-based and greedy pruning methods.

AINeutralarXiv – CS AI · May 296/10

🧠

Singularity-aware Optimization via Randomized Geometric Probing: Towards Stable Non-smooth Optimization

Researchers introduce Singularity-aware Adam (S-Adam), a novel optimizer addressing instability in deep learning with non-smooth components like ReLU activations. The method uses a Local Geometric Instability metric to dynamically adjust step sizes, demonstrating up to 6% accuracy improvements on benchmark datasets while mitigating gradient oscillations.

AINeutralarXiv – CS AI · May 116/10

🧠

The Effect of Mini-Batch Noise on the Implicit Bias of Adam

Researchers present a theoretical framework showing how mini-batch noise in Adam optimizer training affects the implicit bias toward sharper or flatter loss landscape regions, finding that optimal momentum hyperparameters shift based on batch size—small batches favor the default (0.9, 0.999) settings while larger batches benefit from closer β₁ and β₂ values.