Q-Zoom: Query-Aware Adaptive Perception for Efficient Multimodal Large Language Models

Q-Zoom is a new framework that improves the efficiency of multimodal large language models by intelligently processing high-resolution visual inputs. Using adaptive query-aware perception, the system achieves 2.5-4.4x faster inference speeds on document and high-resolution tasks while maintaining or exceeding baseline accuracy across multiple MLLM architectures.

Q-Zoom addresses a fundamental bottleneck in multimodal language models: the computational cost of processing high-resolution images through quadratic self-attention mechanisms. Current MLLMs indiscriminately increase image resolution to handle fine-grained tasks like document OCR and dense scene analysis, but this approach floods the model with redundant visual tokens, creating severe inference latency issues that limit practical deployment. The framework solves this through intelligent query routing rather than brute-force resolution scaling.

The technical innovation centers on two components working in tandem. A Dynamic Gating Network acts as an intelligent filter, determining when high-resolution processing is genuinely necessary versus when coarse features suffice. For queries requiring detailed perception, a Self-Distilled Region Proposal Network identifies task-relevant regions without explicit supervision, using self-supervised learning to avoid annotation overhead. This coarse-to-fine approach mimics human perception, where attention narrows to relevant details only when needed.

The performance results demonstrate substantial real-world value. Testing on Qwen2.5-VL-7B shows 2.52x speedup for document tasks and 4.39x for high-resolution scenarios while preserving accuracy metrics. Notably, the framework can be configured for higher accuracy rather than speed, surpassing baseline performance by 1.1-8.1% depending on the benchmark. Transfer learning validation across Qwen3-VL, LLaVA, and emerging reasoning models suggests broad applicability.

This development matters for enterprises deploying MLLMs in production environments where inference cost directly impacts scalability and margins. Faster processing enables more users per GPU, reducing operational expenses while maintaining quality. The technology establishes a new efficiency frontier in multimodal AI, influencing how future models balance visual understanding with computational constraints.

• →Q-Zoom achieves 2.5-4.4x inference speedup on high-resolution vision tasks through query-aware adaptive processing
• →Dynamic Gating Network intelligently bypasses unnecessary high-resolution computation while maintaining task accuracy
• →Self-Distilled Region Proposal Network uses self-supervised learning to identify task-relevant visual regions without manual annotation
• →Framework transfers effectively across multiple MLLM architectures including Qwen, LLaVA, and reasoning-based models
• →Configurable for either speed optimization (2.5-4.4x faster) or accuracy improvement (1.1-8.1% gain) depending on deployment needs