Visually-Guided Policy Optimization for Multimodal Reasoning

Researchers propose Visually-Guided Policy Optimization (VGPO), a framework that enhances vision-language models' ability to focus on visual information during reasoning tasks. The method addresses a fundamental limitation where text-dominated VLMs suffer from weak visual attention and temporal visual forgetting, improving performance on multimodal reasoning and visual-dependent tasks.

This research tackles a critical bottleneck in vision-language model development: the tendency of these systems to rely heavily on textual information while neglecting visual cues. VGPO introduces mechanisms specifically designed to maintain and amplify visual attention throughout multi-step reasoning processes, addressing both sparse visual token activation and the degradation of visual focus over reasoning steps. The framework employs a Visual Attention Compensation mechanism paired with dual-grained advantage re-weighting that operates at both token and trajectory levels, creating a systematic approach to prioritize visual information during policy optimization.

The broader context reflects ongoing efforts to balance multimodal reasoning in large language models. While VLMs have advanced significantly through reinforcement learning with verifiable rewards (RLVR), they inherit a fundamental architectural bias toward text processing. This research identifies and quantifies temporal visual forgetting—a phenomenon where models progressively lose visual context as they reason—establishing this as a measurable problem requiring structural solutions rather than mere fine-tuning.

For the AI and machine learning community, VGPO represents meaningful progress toward more balanced multimodal systems. Improved visual reasoning has practical implications for applications requiring accurate visual understanding: autonomous systems, medical imaging analysis, document processing, and visual question answering. The method's demonstrated improvements suggest that deliberate architectural modifications can significantly enhance model capabilities without proportional computational costs.

Future development will likely explore whether similar compensation mechanisms apply to other modalities or whether this approach generalizes across different VLM architectures. The quantifiable visual activation improvements could influence how researchers design multimodal systems moving forward.

• →VGPO introduces Visual Attention Compensation to localize and amplify visual cues during multimodal reasoning
• →Temporal visual forgetting—degradation of visual focus across reasoning steps—is identified as a significant VLM limitation
• →Dual-grained advantage re-weighting operates at both token and trajectory levels to prioritize visual information
• →Framework demonstrates superior performance on mathematical multimodal reasoning and visual-dependent tasks
• →Research addresses fundamental text-bias in vision-language models through structured policy optimization modifications