AIBullisharXiv – CS AI · Apr 67/10
🧠Researchers propose Sign-Certified Policy Optimization (SignCert-PO) to address reward hacking in reinforcement learning from human feedback (RLHF), a critical problem where AI models exploit learned reward systems rather than improving actual performance. The lightweight approach down-weights non-robust responses during policy optimization and showed improved win rates on summarization and instruction-following benchmarks.
AIBullisharXiv – CS AI · Mar 167/10
🧠Researchers introduce Guided Policy Optimization (GPO), a new reinforcement learning framework that addresses challenges in partially observable environments by co-training a guider with privileged information and a learner through imitation learning. The method demonstrates theoretical optimality comparable to direct RL and shows strong empirical performance across various tasks including continuous control and memory-based challenges.
AIBullisharXiv – CS AI · Mar 117/10
🧠Researchers introduce Stepwise Guided Policy Optimization (SGPO), a new framework that improves upon Group Relative Policy Optimization (GRPO) by learning from incorrect reasoning responses in large language model training. SGPO addresses the limitation where GRPO fails to update policies when all responses in a group are incorrect, showing improved performance across multiple model sizes and reasoning benchmarks.
AIBullisharXiv – CS AI · Mar 56/10
🧠GIPO (Gaussian Importance Sampling Policy Optimization) is a new reinforcement learning method that improves data efficiency for training multimodal AI agents. The approach uses Gaussian trust weights instead of hard clipping to better handle scarce or outdated training data, showing superior performance and stability across various experimental conditions.
AIBullisharXiv – CS AI · Mar 46/102
🧠Researchers propose NAR-CP, a new method to improve Large Language Models' performance in high-frequency decision-making tasks like UAV pursuit. The approach uses normalized action rewards and consistency policy optimization to address limitations in current LLM-based agents that struggle with rapid, precise numerical state updates.
AIBullisharXiv – CS AI · Mar 46/103
🧠Researchers introduce RAPO (Retrieval-Augmented Policy Optimization), a new reinforcement learning framework that improves LLM agent training by incorporating retrieval mechanisms for broader exploration. The method achieves 5% performance gains across 14 datasets and 1.2x faster training efficiency by using hybrid-policy rollouts and retrieval-aware optimization.
AIBullisharXiv – CS AI · Mar 37/104
🧠MIT researchers introduce VCPO (Variance Controlled Policy Optimization), a new method that improves asynchronous reinforcement learning for LLM training by addressing high variance issues in off-policy settings. The technique dynamically scales learning rates and applies variance control to achieve stable training with 2.5x speedup while maintaining performance.
AIBullisharXiv – CS AI · Mar 37/103
🧠Researchers introduced Scaf-GRPO, a new training framework that overcomes the 'learning cliff' problem in LLM reasoning by providing strategic hints when models plateau. The method boosted Qwen2.5-Math-7B performance on the AIME24 benchmark by 44.3% relative to baseline GRPO methods.
AIBullisharXiv – CS AI · Jun 256/10
🧠Researchers introduce FORCE, a three-stage reinforcement learning framework that significantly improves the efficiency of fine-tuning Vision-Language-Action models for robotics. By addressing Q-function instability and low-quality exploration data, FORCE achieves 79% absolute improvement in success rates while reducing training time by 32.5%, eliminating the need for human intervention during deployment.
AINeutralarXiv – CS AI · Jun 256/10
🧠Researchers introduce auto-exploration, a new reinforcement learning method that automatically explores state and action spaces without requiring manual parameter tuning. The approach achieves optimal sample complexity of O(ε⁻²) while remaining parameter-free and implementable, advancing theoretical RL foundations.
AINeutralarXiv – CS AI · Jun 256/10
🧠Researchers propose Semantic Consistency Policy Optimization (SCPO), a training method that improves how large language model agents learn from reinforcement learning by addressing a fundamental inconsistency: semantically similar intermediate steps receive contradictory credit signals based on whether their trajectory ultimately succeeds or fails. The approach recovers step-level credit from successful rollouts, achieving state-of-the-art performance on complex reasoning tasks like ALFWorld and WebShop.
AIBullisharXiv – CS AI · Jun 256/10
🧠Researchers introduce BiPACE, a novel advantage estimation method for training large language model agents that improves upon existing group-based reinforcement learning approaches. The method addresses fundamental credit assignment problems by using bisimulation-guided clustering and action-conditioned baselines, achieving significant performance improvements on benchmark tasks without requiring additional critics or rollouts.
AINeutralarXiv – CS AI · Jun 236/10
🧠Researchers introduce DiT-Reward, a reward model derived from pretrained Diffusion Transformers that outperforms existing benchmarks like HPSv3 for evaluating text-to-image generation quality. The approach demonstrates that representations learned during generative model training transfer effectively to reward prediction tasks, achieving measurable improvements in preference prediction accuracy and inference speed.
🧠 Stable Diffusion
AINeutralarXiv – CS AI · Jun 236/10
🧠Researchers propose a novel reinforcement learning approach that converts sparse task rewards into dense process rewards by training a discriminator to identify successful episodes and incentivize policies to match their state-action visitations. The method demonstrates significantly faster training on robotic manipulation tasks without altering the optimal policy.
AINeutralarXiv – CS AI · Jun 236/10
🧠Researchers present a study optimizing reinforcement learning for autoregressive text-to-image generation by analyzing how different divergence measures affect policy alignment. Using JS divergence within the GRPO framework, they demonstrate improved performance across evaluation metrics while preserving generation diversity on LlamaGen and Janus-7B models.
AIBullisharXiv – CS AI · Jun 236/10
🧠Researchers introduce DataClaw0, an AI system that actively refines and structures unstructured multimodal data streams to align with specific user and downstream task intents. The 9B-parameter model uses a two-stage pipeline combining supervised fine-tuning with reinforcement learning, validated through a new benchmark and demonstrated improvements in video generation, VQA, and GUI navigation tasks.
AINeutralarXiv – CS AI · Jun 196/10
🧠Researchers propose Uncertainty-Aware Reward Modeling (UARM), a technique that addresses critical vulnerabilities in RLHF training by equipping reward models with calibrated uncertainty estimates and reweighting policy optimization to prevent reward hacking. The method uses quantile-based conformal prediction and heteroscedastic variance decomposition, demonstrating improved alignment quality across multiple benchmark datasets.
AINeutralarXiv – CS AI · Jun 116/10
🧠Researchers propose Agentic Procedural Policy Optimization (APPO), a new reinforcement learning method that improves how AI agents learn to use tools by identifying fine-grained decision points rather than relying on coarse tool-call boundaries. The approach achieves ~4 point improvements across 13 benchmarks while maintaining efficiency and interpretability.
AIBullisharXiv – CS AI · Jun 116/10
🧠Researchers introduce OMAD, an online multi-agent reinforcement learning framework that integrates diffusion-based generative models for improved policy coordination. The method achieves 2.5-5x improvements in sample efficiency across benchmark tasks by using relaxed policy objectives and joint distributional value functions to enable effective exploration without requiring tractable likelihood calculations.
AIBullisharXiv – CS AI · Jun 116/10
🧠Researchers introduce Pass@K Policy Optimization (PKPO), a reinforcement learning method that optimizes for multiple solution attempts jointly rather than individually, enabling better exploration and problem-solving on harder tasks. The approach derives unbiased estimators for pass@k performance across arbitrary k values and demonstrates improved learning on challenging benchmarks using open-source LLMs.
AIBullisharXiv – CS AI · Jun 106/10
🧠Researchers introduce Visual-SDPO, a self-distillation framework that enables code-generating LLMs to improve visual artifact quality by learning from rendered output feedback. The method achieves 10+ point improvements on code-to-visual generation benchmarks while maintaining inference efficiency.
AINeutralarXiv – CS AI · Jun 106/10
🧠SocraticPO is a new reinforcement learning framework that improves large language model training by combining natural-language teacher guidance with reward decay, rather than relying solely on scalar outcome rewards. The method shows improvements on scientific reasoning benchmarks while preventing models from exploiting teacher assistance as a shortcut to rewards.
AINeutralarXiv – CS AI · Jun 106/10
🧠Researchers propose SHAPO (Sharpness-Aware Policy Optimization), a reinforcement learning technique that improves safe exploration by treating parameter sensitivity as a proxy for uncertainty. The method makes policy updates conservative in unexplored regions, demonstrating improved safety and task performance across continuous-control tasks.
AINeutralarXiv – CS AI · Jun 106/10
🧠Researchers propose Group Relative Policy Optimization (GRPO), a baseline-free training algorithm for neural combinatorial optimization that eliminates the need for maintaining frozen policy copies. Testing on TSP and CVRP benchmarks shows GRPO prevents training collapse seen in standard REINFORCE while achieving competitive solution quality, offering a more stable alternative for routing problem optimization.
AINeutralarXiv – CS AI · Jun 106/10
🧠Researchers present a systematic framework for evaluating sim-to-real correlation in vision-language-action (VLA) robot policies, identifying why simulation benchmarks often fail to predict real-world performance. The study examines simulation platforms, policy rankings, and perturbation factors to guide both simulator designers and practitioners on effectively using simulation for policy development.
