54 articles tagged with #grpo. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers introduce TruthRL, a reinforcement learning framework that optimizes large language models for truthfulness by reducing hallucinations while allowing strategic abstention when uncertain. The method achieves significant improvements across multiple benchmarks, reducing hallucinations by over 50% while improving truthfulness metrics substantially.
Researchers propose Dropout-GRPO, a method that addresses a fundamental limitation in training latent-reasoning language models by introducing structured stochasticity through dropout masks. The technique enables Group Relative Policy Optimization to work effectively with continuous hidden states rather than discrete tokens, improving performance on mathematical reasoning tasks.
Researchers introduce ACTIVE-o3, a reinforcement learning framework that enables Multimodal Large Language Models (MLLMs) to actively perceive and intelligently select regions of interest for visual analysis. The system outperforms GPT-o3's zoom strategy while maintaining general understanding capabilities, with applications spanning robotics, autonomous driving, and remote sensing.
Researchers propose MMR-GRPO, a training optimization technique that accelerates Group Relative Policy Optimization (GRPO) for mathematical reasoning models by reweighting rewards based on completion diversity. The method achieves comparable performance while reducing training time by 70.2% and training steps by 47.9%, demonstrating consistent improvements across multiple model sizes and benchmarks.
Researchers demonstrate that Group Relative Policy Optimization (GRPO) combined with a novel Variance-Aware Reward Framework significantly improves smaller LLMs' performance on medical question answering, particularly for heart-related queries. The approach achieves 38% accuracy improvement on a held-out test set while remaining competitive with much larger models, offering a practical path toward efficient, deployable medical AI systems.
Researchers introduce MAPR, a meta-awareness framework that enhances reasoning models by predicting task statistics (length, pass-rate, concepts) rather than relying solely on answer verification. The method achieves 83.18% accuracy gains on AIME25 and 13.04% average improvement across mathematics benchmarks while accelerating training efficiency by 1.28x.
Researchers demonstrate that Group Relative Policy Optimization (GRPO), a popular reinforcement learning algorithm using outcome rewards, mathematically functions as an implicit process reward model. The discovery enables algorithmic improvements (λ-GRPO) that enhance large language model performance on reasoning tasks without explicit process reward implementation or significant computational overhead.
Search-E1 introduces a simplified self-evolution method for search-augmented reasoning agents that achieves competitive performance through vanilla GRPO and self-distillation, without external supervision or complex auxiliary systems. The approach reaches 0.440 average EM on QA benchmarks with Qwen2.5-3B, demonstrating that elaborate post-training machinery may be unnecessary for effective agent development.
Researchers introduce rubric-grounded reinforcement learning, a framework that trains AI models using structured, multi-criterion rewards from an LLM judge rather than binary outcomes. Training Llama-3.1-8B on scientific documents achieved 71.7% normalized reward and demonstrated improved performance on multiple reasoning benchmarks, suggesting that document-grounded training signals can produce generalizable reasoning capabilities.
Researchers propose Lorem Perturbation for Exploration (LoPE), a training technique that addresses the zero-advantage problem in reinforcement learning for large language models by prepending random Latin-based text to prompts, enabling broader reasoning exploration across 1.7B to 7B parameter models.
Researchers introduce a listener-augmented reinforcement learning framework for training vision-language models to better align with human visual preferences. By using an independent frozen model to evaluate and validate reasoning chains, the approach achieves 67.4% accuracy on ImageReward benchmarks and demonstrates significant improvements in out-of-distribution generalization.
Researchers introduce Stepwise Guided Policy Optimization (SGPO), a new framework that improves upon Group Relative Policy Optimization (GRPO) by learning from incorrect reasoning responses in large language model training. SGPO addresses the limitation where GRPO fails to update policies when all responses in a group are incorrect, showing improved performance across multiple model sizes and reasoning benchmarks.
Researchers developed a three-stage curriculum learning framework that improves Chain-of-Thought reasoning distillation from large language models to smaller ones. The method enables Qwen2.5-3B-Base to achieve 11.29% accuracy improvement while reducing output length by 27.4% through progressive skill acquisition and Group Relative Policy Optimization.
Researchers introduced Scaf-GRPO, a new training framework that overcomes the 'learning cliff' problem in LLM reasoning by providing strategic hints when models plateau. The method boosted Qwen2.5-Math-7B performance on the AIME24 benchmark by 44.3% relative to baseline GRPO methods.
Kwai AI has developed SRPO, a new reinforcement learning framework that reduces LLM post-training steps by 90% while achieving performance comparable to DeepSeek-R1 in mathematics and coding tasks. The two-stage approach with history resampling addresses efficiency limitations in existing GRPO methods.
Researchers introduce FBOS-RL, a reinforcement learning algorithm that improves upon GRPO by incorporating feedback-guided exploration and dual training objectives (EPA and ECC) to address the problem of training stagnation when tasks exceed the model's current capabilities. The method demonstrates faster learning and higher performance ceilings compared to existing approaches while maintaining higher policy entropy and lower gradient norms.
Researchers introduce ExTra, a reinforcement learning framework that improves language model reasoning by extracting exploration signals from model rollouts. The method combines novelty rewards for diverse solutions with entropy-guided trajectory regeneration, achieving 5-7 point improvements over baseline GRPO across mathematical reasoning benchmarks.
Researchers present a study optimizing reinforcement learning for autoregressive text-to-image generation by analyzing how different divergence measures affect policy alignment. Using JS divergence within the GRPO framework, they demonstrate improved performance across evaluation metrics while preserving generation diversity on LlamaGen and Janus-7B models.
Researchers propose fine-tuning pipelines to enable large language models to perform genuine quantum reasoning rather than pattern matching, using quantum circuit simulation as a training objective. Two approaches—Supervised Fine-Tuning (SFT) and a combined SFT+Group Relative Policy Optimisation (GRPO) method—demonstrate significant performance improvements over baseline models, with trade-offs between in-distribution accuracy and generalization to larger quantum systems.
Researchers demonstrate that over-training SFT (supervised fine-tuning) models can paradoxically degrade RLHF performance by compressing the rollout distribution's entropy, causing rank inversion where higher pre-RL pass rates correlate with worse post-RL outcomes. Testing on Qwen2.5-Coder and DeepSeek-Coder reveals this failure mode occurs when entropy collapse prevents effective group-relative reward signals, suggesting a fundamental optimization challenge in LLM alignment pipelines.
Researchers propose SVoT, a reinforcement learning framework that enhances multimodal AI models' spatial reasoning by generating verifiable intermediate states and visualizations. The approach achieves up to 65% accuracy gains on out-of-distribution tests by explicitly modeling state transitions and verification processes, addressing a critical limitation in current large language models.
Researchers propose SD-GRPO, a new machine learning technique that improves how multimodal AI systems generate long-form responses by analyzing outputs in semantic segments rather than as a single unit. The method addresses a fundamental limitation in existing GRPO frameworks when applied to vision-language tasks, showing consistent performance improvements across controlled and real-world benchmarks.
Researchers propose Group Relative Policy Optimization (GRPO), a baseline-free training algorithm for neural combinatorial optimization that eliminates the need for maintaining frozen policy copies. Testing on TSP and CVRP benchmarks shows GRPO prevents training collapse seen in standard REINFORCE while achieving competitive solution quality, offering a more stable alternative for routing problem optimization.
Researchers introduce DiRL, a reinforcement learning framework that distinguishes between genuine reasoning and memorization in large language models by anchoring exploration to an internal reasoning-memorization direction. The method integrates with Group Relative Policy Optimization to improve performance on mathematical and reasoning benchmarks while suppressing exploration of memorized shortcuts.
Researchers introduce ISPO (Intrinsic Signal Policy Optimization), a new reinforcement learning method that improves long-chain reasoning in large language models by densifying reward signals with intrinsic metrics derived from the model's own probabilities. The approach addresses critical failure modes in existing GRPO-based methods and shows consistent improvements across mathematical reasoning benchmarks.
