ProcessThinker: Enhancing Multi-modal Large Language Models Reasoning via Rollout-based Process Reward

ProcessThinker introduces a novel post-training method for multimodal large language models that provides step-level process rewards without requiring explicit reward model training. By using rollout-based sampling to verify intermediate reasoning steps, the approach improves visual question answering across multiple benchmarks while reducing computational overhead compared to traditional process reward models.

ProcessThinker addresses a fundamental challenge in multimodal AI reasoning: distinguishing between early reasoning failures and late-stage errors. Traditional reinforcement learning approaches in visual question answering rely on sparse outcome-based rewards that cannot pinpoint where reasoning chains break down. This limitation becomes critical for complex tasks requiring multi-step logical inference, where a single error compounds across subsequent steps. The proposed method elegantly sidesteps the computational burden of training dedicated process reward models by leveraging rollout-based verification—sampling multiple continuations from each intermediate step and measuring their success rates. This approach distributes credit assignment throughout the reasoning process, encouraging models to develop more coherent logical chains rather than stumbling toward correct answers through inconsistent reasoning paths. The method's practical value lies in its efficiency; it requires only rewriting existing reasoning traces into step-tagged formats for cold-start fine-tuning before applying standard group relative policy optimization. Results across four demanding video understanding benchmarks—Video-MMMU, MMVU, VideoMathQA, and LongVideoBench—demonstrate consistent improvements over the baseline Qwen3-VL-8B-Instruct model. This advancement matters for developers building multimodal AI systems where reasoning transparency and reliability increasingly matter. As video understanding tasks grow more complex, techniques that improve intermediate reasoning quality directly enhance overall system trustworthiness, benefiting applications from autonomous systems to educational AI.

• →ProcessThinker eliminates the need to train explicit process reward models while providing step-level supervision through rollout-based verification.
• →The method improves dense credit assignment during reasoning, reducing inconsistent or self-contradictory logic chains in multimodal models.
• →Implementation requires only reformatting reasoning traces and applying standard GRPO training, making adoption practical for existing architectures.
• →Consistent performance gains across four challenging video understanding benchmarks indicate broad applicability to complex reasoning tasks.
• →The approach reduces training costs compared to traditional process reward model training while maintaining or exceeding performance improvements.