Not All Tokens See Equally: Perception-Grounded Policy Optimization for Large Vision-Language Models

Researchers introduce Perception-Grounded Policy Optimization (PGPO), a novel fine-tuning framework that improves how large vision-language models learn from visual inputs by strategically allocating learning signals to vision-dependent tokens rather than treating all tokens equally. Testing on the Qwen2.5-VL series demonstrates an average 18.7% performance boost across multimodal reasoning benchmarks.

This research addresses a fundamental inefficiency in how large vision-language models optimize their reasoning processes. Traditional reinforcement learning approaches distribute equal learning signals across all generated tokens, creating a dilution effect where critical visual-reasoning steps receive no more emphasis than routine linguistic predictions. The proposed PGPO framework quantifies visual dependency using KL divergence metrics to identify which tokens genuinely depend on visual information, then dynamically amplifies learning signals for those tokens while suppressing noise from purely linguistic patterns.

The advancement builds on recent progress in reinforcement learning from verifiable rewards for multimodal systems, where the field recognized that not all generated content contributes equally to solving vision-language tasks. By implementing a threshold-gated mechanism that conserves total learning signal mass while reshaping its distribution, PGPO prevents training instability while focusing optimization efforts where they matter most. The method essentially treats multimodal reasoning as a hierarchical learning problem where perception-grounded steps deserve priority over standard language generation.

For the AI and machine learning community, this development represents a meaningful step toward more efficient fine-tuning of large multimodal models, which consume substantial computational resources. The 18.7% average improvement across seven challenging benchmarks suggests practical gains in reasoning accuracy without requiring larger models or additional training data. The framework's regularization properties also address a persistent challenge in vision-language model training: preventing optimization collapse while maintaining gradient stability.

Future applications likely extend to other multimodal tasks requiring precise visual grounding, from medical image analysis to autonomous systems. The theoretical framework could inspire similar perception-priority approaches in other domains combining multiple input modalities.

• →PGPO introduces fine-grained token-level credit assignment that amplifies learning for vision-dependent tokens while suppressing linguistic noise
• →Average 18.7% performance improvement across seven multimodal reasoning benchmarks on Qwen2.5-VL models demonstrates significant practical gains
• →The framework quantifies visual dependency through KL divergence to identify which tokens genuinely require visual information for correct reasoning
• →PGPO acts as a regularizer that reduces gradient variance and prevents training collapse in large vision-language model optimization
• →Threshold-gated mechanism maintains total learning signal mass while dynamically reshaping its distribution for perception-grounded tasks