Structured Neuron Pruning in Deep Neural Networks Using Multi-Armed Bandits

Researchers present a novel structured pruning framework that uses multi-armed bandit algorithms to remove redundant neurons from deep neural networks. The approach treats each neuron as a bandit arm, testing its importance through temporary masking and loss measurement, then applies various MAB policies (UCB1, Thompson Sampling, etc.) to identify which neurons to prune. Experiments across tabular and deep learning tasks show MAB-based pruning significantly outperforms traditional magnitude-based and greedy pruning methods.

This research addresses a fundamental challenge in neural network optimization: reducing model size while maintaining performance. Deep neural networks commonly contain redundant computational units that consume memory and processing power without meaningfully contributing to predictions. The structured pruning framework converts neuron removal into a sequential decision-making problem, where multi-armed bandit algorithms systematically identify which neurons can be safely eliminated. This approach differs from prior work by treating pruning as an exploration-exploitation tradeoff rather than applying fixed heuristics based on weight magnitude or activation patterns.

The multi-armed bandit formulation is well-suited to this problem because it balances computational cost with information gain. Each candidate neuron evaluation requires only a forward-backward pass on a mini-batch, making the approach practical for large-scale models. The framework's flexibility to support multiple policies—from exploration-focused methods like Thompson Sampling to exploitation-focused approaches like UCB1—enables adaptation to different model architectures and computational constraints.

The experimental validation is rigorous, employing proper statistical testing methods (Friedman test with Nemenyi post-hoc analysis) across diverse domains including tabular classification, regression, and deep learning tasks. UCB1 and Thompson Sampling consistently achieve superior performance ranks, suggesting these confidence-bound strategies effectively navigate the pruning search space. The framework's demonstrated improvements over unpruned baselines and existing pruning methods indicate practical utility for model compression in resource-constrained environments.

For practitioners, this work provides a principled alternative to manual pruning thresholds or magnitude-based heuristics. Future research directions include adaptive hyperparameter selection and theoretical complexity analysis.

• →Multi-armed bandit algorithms effectively identify which neurons to prune by treating neuron removal as a sequential decision problem with measurable rewards.
• →UCB1 and Thompson Sampling outperform traditional magnitude-based and greedy pruning approaches across image, text, and tabular learning tasks.
• →Structured pruning via MAB policies reduces parameter count while maintaining or improving model performance with minimal computational overhead.
• →The framework's statistical validation using Friedman and Nemenyi tests demonstrates significant improvements over unpruned baselines and conventional pruning methods.
• →MAB-based pruning offers practical advantages for model compression in resource-constrained environments by removing entire neurons rather than individual weights.