GazeVLM: Active Vision via Internal Attention Control for Multimodal Reasoning

Researchers introduce GazeVLM, a vision-language model that implements active attention control mechanisms mimicking human visual reasoning. The 4B-parameter model autonomously generates gaze tokens to dynamically focus on task-relevant visual details, achieving 4-5% performance improvements over comparable VLMs without increasing context window size.

GazeVLM addresses a fundamental limitation in current vision-language models: their passive, unfocused processing of visual information. Traditional VLMs accumulate massive token contexts that dilute spatial reasoning and produce hallucinations, whereas human vision operates through directed attention guided by task goals. This research demonstrates that implementing metacognitive control mechanisms—allowing models to self-direct their attention toward relevant visual regions—produces measurable performance gains.

The architecture represents a paradigm shift in multimodal AI design. Rather than relying on external tools like cropping or exponentially expanding context windows, GazeVLM internalizes attention control directly into its reasoning loop. The model generates special gaze tokens that trigger suppression biases on irrelevant visual features while maintaining peripheral awareness. This simulates human foveal fixation without architectural complexity, enabling fluid transitions between global scene understanding and localized reasoning.

The performance results validate the approach. On high-resolution benchmarks (HRBench-4k and HRBench-8k), GazeVLM surpasses parameter-equivalent VLMs by nearly 4% and outperforms more complex agentic multimodal pipelines by over 5%. The training methodology using Group Relative Policy Optimization (GRPO) with grounding rewards proves effective for developing this capability.

This advancement has implications for AI systems requiring spatial reasoning, visual question-answering, and high-resolution image analysis. The efficiency gains—achieving performance improvements without inflating model size or context length—make this approach commercially viable for deployment in resource-constrained environments. Future iterations may reveal how this active vision paradigm scales to larger models and more complex reasoning tasks.

• →GazeVLM implements autonomous attention control allowing models to dynamically focus on task-relevant visual regions rather than processing all visual information passively.
• →The 4B-parameter model achieves 4-5% performance improvements on high-resolution benchmarks without expanding context windows or adding external processing tools.
• →The architecture uses special gaze tokens to trigger suppression biases on irrelevant features while maintaining global scene awareness, simulating human foveal vision.
• →Training with Group Relative Policy Optimization (GRPO) specifically rewards valid grounding, enabling the model to learn effective attention strategies.
• →Efficiency gains and performance improvements make this approach viable for deployment scenarios where model size and computational resources are constrained.