Energy-Driven Adaptive Visual Token Pruning for Efficient Vision-Language Models

Researchers developed E-AdaPrune, an energy-driven adaptive pruning framework that optimizes Vision-Language Models by dynamically allocating visual tokens based on image information density. The method shows up to 0.6% average improvement across benchmarks, with a notable 5.1% boost on reasoning tasks, while adding only 8ms latency per image.

• →E-AdaPrune uses singular value spectrum analysis to determine optimal token budgets for different images based on information density.
• →The framework outperforms fixed-budget approaches across nine benchmarks and three VLM backbones including LLaVA models.
• →The method achieves significant performance gains without introducing additional learnable parameters.
• →Implementation adds minimal computational overhead with only 8ms additional latency per image.
• →The approach shows particularly strong results on complex reasoning tasks with up to 5.1% relative improvement.