CLASP: Class-Adaptive Layer Fusion and Dual-Stage Pruning for Multimodal Large Language Models

Researchers introduce CLASP, a token reduction framework that optimizes Multimodal Large Language Models by intelligently pruning visual tokens through class-adaptive layer fusion and dual-stage pruning. The approach addresses computational inefficiency in MLLMs while maintaining performance across diverse benchmarks and architectures.

CLASP represents a meaningful advancement in optimizing multimodal AI systems, addressing a critical bottleneck in current MLLM architectures. The framework tackles visual token redundancy—a fundamental efficiency problem that constrains deployment of these models in resource-limited environments. Unlike static pruning approaches that apply uniform compression rules, CLASP adapts its token reduction strategy based on input class characteristics, enabling more sophisticated feature extraction.

The technical innovation centers on dual-stage pruning that balances two competing objectives: preserving attention-salient tokens critical for relevance while maintaining coverage through redundancy-aware tokens. This approach mirrors how humans process visual information—focusing on salient details while maintaining contextual awareness. The multi-layer vision feature fusion moves beyond conventional single-layer ViT approaches, potentially capturing richer semantic information across different abstraction levels.

For the AI industry, this work directly impacts model efficiency and deployment feasibility. Reducing computational overhead enables running advanced MLLMs on consumer hardware and edge devices, expanding market accessibility. The plug-and-play nature of CLASP suggests compatibility with existing MLLM architectures, facilitating rapid adoption across different models and reducing fragmentation in optimization strategies.

The broad experimental validation across benchmarks and pruning ratios indicates robustness, though real-world deployment performance in production systems remains to be demonstrated. Future developments might focus on extending these techniques to other modalities and exploring whether similar class-adaptive approaches benefit pure language models, potentially reshaping efficiency standards across the AI stack.

• →CLASP achieves aggressive visual token reduction through intelligent dual-stage pruning that adapts to input characteristics rather than applying static compression rules.
• →The framework integrates multi-layer vision feature fusion to create category-specific representations, improving feature richness compared to single-layer approaches.
• →Class-adaptive token allocation balances relevance preservation and coverage maintenance, enabling both efficiency and performance across diverse tasks.
• →The plug-and-play design enables straightforward integration with existing MLLM architectures without architectural modifications.
• →Extensive benchmarking demonstrates consistent improvements across different pruning ratios and model architectures, suggesting reliable performance generalization.