AINeutralarXiv – CS AI · Jun 25/10
🧠Researchers compare dynamic entropy tuning in stochastic reinforcement learning policies versus deterministic policies for quadcopter control, finding that dynamic entropy adjustment in the Soft Actor-Critic algorithm prevents catastrophic forgetting and improves exploration efficiency compared to static entropy or purely deterministic approaches using TD3.
AINeutralarXiv – CS AI · Jun 16/10
🧠Researchers introduce Prompted Policy Optimization (PromptPO), a method using large language models as black-box policy optimizers for reinforcement learning tasks. The approach demonstrates competitive or superior performance to traditional RL algorithms in exploration-heavy and robotics domains while requiring fewer environment interactions, though it underperforms in continuous control tasks like MuJoCo.
AIBullisharXiv – CS AI · Jun 16/10
🧠Researchers propose S2L-PO, a framework that uses smaller language models as natural policy explorers to train larger models more efficiently. By leveraging the inherent policy-level diversity of smaller models rather than token-level randomness, the approach achieves significant accuracy improvements on mathematical reasoning tasks while reducing computational costs.
AINeutralarXiv – CS AI · Jun 16/10
🧠Researchers propose Safe Equilibrium Policy Optimization (SEPO), a training method that prevents language model agents from exploiting weaker opponents, colluding on harmful outcomes, or externalizing costs during multi-agent interactions. The technique augments standard reward optimization with penalties for exploitability and collusion risk, demonstrated across strategic domains including Prisoner's Dilemma, auctions, and poker.
AINeutralarXiv – CS AI · Jun 16/10
🧠This paper analyzes why reinforcement learning methods that update policies based on reward signals without explicitly tracking uncertainty can still be effective. Researchers prove that annealed softmax policies achieve near-optimal regret rates in many-armed Bayesian bandit settings when many near-optimal actions exist, providing theoretical justification for uncertainty-agnostic approaches used in modern language model training.
AIBullisharXiv – CS AI · Jun 16/10
🧠Researchers propose Boundary-Guided Policy Optimization (BGPO), a memory-efficient reinforcement learning algorithm for diffusion large language models that addresses a critical bottleneck in likelihood function approximation. By constructing a specially designed lower bound that enables gradient accumulation across samples while maintaining mathematical equivalence to traditional objectives, BGPO achieves superior performance on math, coding, and planning tasks with significantly reduced memory overhead.
AINeutralarXiv – CS AI · Jun 16/10
🧠Researchers propose Bottom-up Policy Optimization (BuPO), a novel reinforcement learning approach that optimizes internal layers of language models rather than treating them as unified policies. The study reveals that LLMs contain distinct internal policy structures with different entropy patterns across layers, offering new insights into how transformer-based models process reasoning tasks.
🧠 Llama
AINeutralarXiv – CS AI · May 296/10
🧠Researchers introduce Q-ALIGN DT, a machine learning framework that improves return-conditioned supervised learning by aligning return-to-go signals with actual policy performance using Q-value guidance. The method demonstrates superior controllability and generalization across reinforcement learning benchmarks, potentially advancing AI decision-making systems.
AIBullisharXiv – CS AI · May 296/10
🧠Researchers propose Hysteretic Policy Optimization (HPO), a refinement to GRPO reinforcement learning that addresses training instability in sparse-reward environments by downweighting negative-advantage updates and normalizing by mean length rather than per-response length. The adaptive variant (A-HPO) achieves 15% reward improvement over GRPO on benchmark tasks.
AIBullisharXiv – CS AI · May 296/10
🧠Researchers present Graph-Enhanced Policy Optimization (GEPO), a new training framework for multi-step LLM agents that improves credit assignment by analyzing state-transition graphs and task relevance. The method achieves 1.1-3.8% performance gains across multiple benchmarks by differentiating the importance of individual steps and trajectories based on their structural and semantic roles.
AINeutralarXiv – CS AI · May 296/10
🧠Researchers introduce Graph-Distance Contribution Reward (GDCR), a novel step-level credit assignment method for agentic search that evaluates individual agent actions by measuring progress toward answer nodes in knowledge graphs. Combined with Step Advantage Policy Optimization (SAPO), this approach improves upon trajectory-level reward systems that cannot assess the quality of intermediate steps, showing strong results across multiple benchmarks.
AINeutralarXiv – CS AI · May 296/10
🧠Researchers present Nested Causal Thompson Sampling (NCTS), a machine learning framework for sequential decision-making where strategic choices causally influence subsequent tactical decisions across multiple timescales. The work introduces PAC-Bayesian risk bounds that enable off-policy certification of deployment policies from historical data alone, enabling safer handover from legacy systems to learned agents.
AINeutralarXiv – CS AI · May 286/10
🧠Researchers introduce Frost Training, a novel method that applies gradient-based optimization from embedding space to improve LLM policy training on Cross-Entropy Games. The technique leverages signals previously used only in adversarial jailbreaking to accelerate model performance, achieving higher quality outputs faster in Monte Carlo-based optimization tasks.
AINeutralarXiv – CS AI · May 286/10
🧠Researchers propose EAPO, an entropy-driven adaptive method for training large reasoning models on open-ended question answering tasks. The approach dynamically adjusts the weighting of positive and negative samples during reinforcement learning training, demonstrating improved performance on medical QA datasets by balancing response diversity with stability.
AINeutralarXiv – CS AI · May 286/10
🧠Researchers introduce Global PSRO, an improved algorithm for computing Nash equilibria in two-player zero-sum games by using Population Exploitability metrics to guide strategy expansion more efficiently than existing methods. The approach reduces computational requirements while achieving better approximations of equilibrium solutions, advancing game-theoretic AI applications.
AINeutralarXiv – CS AI · May 286/10
🧠Researchers have developed Diffusion-Augmented Markov Decision Processes (DA-MDPs), a framework that integrates diffusion models into maximum entropy reinforcement learning to sample from optimal policy trajectory distributions. The approach is tested on three RL algorithms (PPO, WPO, REPPO) and demonstrates competitive or superior performance on continuous-control tasks while excelling at modeling multimodal action distributions.
AINeutralarXiv – CS AI · May 286/10
🧠Researchers introduce ECHO, a novel test-time reinforcement learning algorithm that addresses rollout collapse and noisy pseudo-labels through entropy-confidence hybrid optimization. The method improves sampling efficiency and training robustness across mathematical and visual reasoning benchmarks while performing better under limited computational budgets.
AIBullisharXiv – CS AI · May 276/10
🧠Researchers introduce Pilot-Commit, a new framework for optimizing reinforcement learning post-training of large language models by intelligently allocating computational budget to high-value prompts. The method achieves training speedups of 1.9x to 4.0x by identifying prompts with high reward variance where group-based updates are most effective, rather than uniformly distributing rollouts across all prompts.
AIBullisharXiv – CS AI · May 276/10
🧠Researchers introduce R²VPO, a new reinforcement learning method that replaces hard clipping mechanisms with ratio-variance regularization to improve policy optimization. Tested across large language models and robotic control tasks, the approach achieves better performance on mathematical reasoning and sample efficiency while maintaining stable learning.
$VPO
AIBullisharXiv – CS AI · May 276/10
🧠Researchers propose UCPO (Uncertainty-Aware Policy Optimization), a new reinforcement learning framework designed to improve large language model reliability by addressing advantage bias and reward hacking in uncertainty-based training. The method uses ternary advantage decoupling and dynamic reward adjustment to better calibrate model confidence levels in high-stakes applications.
AIBullisharXiv – CS AI · May 276/10
🧠Researchers introduce VeRPO, a reinforcement learning framework that converts partial test-case successes into dense, verifiable reward signals for code generation tasks. The method achieves up to 8.83% improvement in pass@1 metrics while eliminating the sparse reward problem that plagues traditional test-suite evaluation, offering a practical alternative to computationally expensive reward models.
AIBullisharXiv – CS AI · May 126/10
🧠Researchers propose Pair-GRPO, a unified theoretical framework for LLM alignment that addresses instability and interpretability issues in reinforcement learning from human preferences. The method introduces Soft-Pair-GRPO and Hard-Pair-GRPO variants with proven gradient equivalence, monotonic policy improvement, and superior performance on standard benchmarks.
AINeutralarXiv – CS AI · May 126/10
🧠Researchers propose DPA-GRPO, a novel training method for large language models that improves structured decision-making by using a generator-verifier framework where one model produces outputs and another validates them through safety assurance cases. The method demonstrates improved accuracy on tax calculation benchmarks and addresses the challenge of ensuring LLM outputs are locally correct, globally consistent, and auditable.
AIBullisharXiv – CS AI · May 126/10
🧠Researchers propose VIGOR, a verifier-free reinforcement learning method for large language models that eliminates dependency on gold labels or domain-specific verifiers by using gradient-norm measurements as intrinsic reward signals. The approach demonstrates measurable improvements over existing baselines on mathematical reasoning and exhibits cross-domain transfer to code tasks, addressing a major scalability constraint in current RL-based LLM training.
AINeutralarXiv – CS AI · May 116/10
🧠Researchers introduce Structured Role-Aware Policy Optimization (SRPO), a reinforcement learning method that improves multimodal AI reasoning by assigning credit to different token types based on their functional roles. The approach enhances vision-language models' ability to ground answers in visual evidence without requiring external reward models, advancing more reliable multimodal reasoning systems.
