AIBullisharXiv – CS AI · 4d ago7/10
🧠Researchers introduce 3SPO (State-Score-Supervised Policy Optimization), a reinforcement learning algorithm that optimizes LLM agent policies at each step rather than after complete episodes, addressing credit assignment challenges in sparse-reward environments. Experiments demonstrate 22.6% improvement over existing methods on ALFWorld benchmarks with 2.4x more state exploration and 1.8x faster convergence.
AIBullisharXiv – CS AI · Jun 47/10
🧠Researchers introduce DistIL, a distributional variant of the DAgger imitation learning algorithm that leverages rich feedback signals beyond binary correctness labels to improve AI reasoning models. The approach uses forward cross-entropy objectives to enable better credit assignment and demonstrates monotonic policy improvement guarantees, outperforming standard reinforcement learning methods across scientific reasoning, coding, and mathematical problem-solving tasks.
AIBullisharXiv – CS AI · Jun 17/10
🧠Researchers introduce HiPER, a hierarchical reinforcement learning framework that separates high-level planning from low-level execution for training LLM agents. The approach uses hierarchical advantage estimation to improve credit assignment in sparse-reward environments, achieving state-of-the-art results on interactive benchmarks with significant gains on long-horizon tasks.
AIBullisharXiv – CS AI · May 277/10
🧠Researchers propose GraphGPO, a novel reinforcement learning method that improves credit assignment in agentic tasks by aggregating trajectories into a state-transition graph rather than relying on coarse-grained outcome-based attribution. This approach enables step-level credit recognition and achieves state-of-the-art performance on challenging benchmarks while significantly improving training efficiency.
AIBullisharXiv – CS AI · May 277/10
🧠Researchers propose SRPO (Self-Reset Policy Optimization), a novel method that improves how language models learn from reasoning tasks by identifying and isolating problematic reasoning steps rather than treating entire solution trajectories uniformly. The technique uses the model itself to self-localize errors and reset to those points for resampling, outperforming standard approaches like GRPO without requiring external supervision.
AIBearisharXiv – CS AI · May 127/10
🧠Researchers propose TRACE, a credit assignment framework that improves multi-turn jailbreak attacks on large language models by identifying which dialogue turns actually contribute to harmful outcomes. The method achieves 25% higher attack success rates than existing approaches and can be repurposed to strengthen AI safety defenses.
AIBullisharXiv – CS AI · May 97/10
🧠Researchers propose a novel reinforcement learning framework that automatically generates process-level supervision from outcome-only feedback, eliminating the need for costly external process supervision. This approach enables fine-grained credit assignment in reasoning tasks by having models identify and learn from their own failed trajectories.
AIBullisharXiv – CS AI · May 97/10
🧠Researchers propose Selective Eligibility Traces (S-trace), a new method for reinforcement learning that improves credit assignment in large language models by selectively identifying critical reasoning steps rather than uniformly crediting entire trajectories. The approach demonstrates performance gains of 0.49-3.16% across Qwen models while improving sample and token efficiency compared to existing critic-free algorithms.
AIBullisharXiv – CS AI · May 97/10
🧠Researchers introduce BEACON, a milestone-guided policy learning framework that significantly improves training efficiency for long-horizon language agents by solving credit misattribution and sample inefficiency problems. The approach achieves 92.9% success rates on complex tasks—nearly double previous benchmarks—while improving sample utilization from 23.7% to 82.0%.
AIBullisharXiv – CS AI · May 47/10
🧠Researchers present AEM (Adaptive Entropy Modulation), a new credit assignment method for reinforcement learning that improves how language model agents learn from sparse rewards without requiring dense supervision. The technique adaptively modulates entropy during training to balance exploration and exploitation, achieving a 1.4% improvement on the challenging SWE-bench-Verified benchmark across models ranging from 1.5B to 32B parameters.
AIBullisharXiv – CS AI · Apr 147/10
🧠Researchers propose Generative Actor-Critic (GenAC), a new approach to value modeling in large language model reinforcement learning that uses chain-of-thought reasoning instead of one-shot scalar predictions. The method addresses a longstanding challenge in credit assignment by improving value approximation and downstream RL performance compared to existing value-based and value-free baselines.
AIBullisharXiv – CS AI · Apr 107/10
🧠Researchers introduce Perception-Grounded Policy Optimization (PGPO), a novel fine-tuning framework that improves how large vision-language models learn from visual inputs by strategically allocating learning signals to vision-dependent tokens rather than treating all tokens equally. Testing on the Qwen2.5-VL series demonstrates an average 18.7% performance boost across multimodal reasoning benchmarks.
AIBullisharXiv – CS AI · Mar 117/10
🧠Researchers introduced HCAPO, a new framework that uses hindsight credit assignment to improve Large Language Model agents' performance in long-horizon tasks. The system leverages LLMs as post-hoc critics to refine decision-making, achieving 7.7% and 13.8% improvements over existing methods on WebShop and ALFWorld benchmarks respectively.
AINeutralarXiv – CS AI · 3d ago6/10
🧠Researchers propose Agentic Procedural Policy Optimization (APPO), a new reinforcement learning method that improves how AI agents learn to use tools by identifying fine-grained decision points rather than relying on coarse tool-call boundaries. The approach achieves ~4 point improvements across 13 benchmarks while maintaining efficiency and interpretability.
AINeutralarXiv – CS AI · 3d ago6/10
🧠Researchers introduce HERO, a self-distillation framework for reinforcement learning agents that uses environment observations as feedback to improve multi-turn decision-making. The method addresses credit assignment problems in sequential tasks by converting observations into actionable diagnoses, outperforming existing approaches on benchmark tasks with limited training data.
AINeutralarXiv – CS AI · 4d ago6/10
🧠Researchers propose SD-GRPO, a new machine learning technique that improves how multimodal AI systems generate long-form responses by analyzing outputs in semantic segments rather than as a single unit. The method addresses a fundamental limitation in existing GRPO frameworks when applied to vision-language tasks, showing consistent performance improvements across controlled and real-world benchmarks.
AIBullisharXiv – CS AI · 4d ago6/10
🧠Researchers propose DAC (Divide and Cooperate), a multi-agent training framework that separates evidence retrieval and answer generation into two specialized agents with cross-agent learning signals. This approach addresses credit assignment problems in language models performing multi-step reasoning and achieves competitive performance using parameter-efficient LoRA modules, outperforming full fine-tuning baselines on QA benchmarks.
AIBullisharXiv – CS AI · 5d ago6/10
🧠LEAF (Low-rank Exploration with Adaptive Forking) introduces a novel tree-based reinforcement learning method for training speech-aware large language models that improves credit assignment by identifying shared response prefixes and assigning rewards at the span level rather than uniformly across tokens. The approach achieves superior performance compared to existing GRPO-style methods without requiring additional computational overhead, enabling smaller models to match or exceed larger baselines.
AINeutralarXiv – CS AI · Jun 56/10
🧠Researchers introduce RREDCoT, a novel method for improving reasoning language models by redistributing rewards at the segment level during reinforcement learning training. The approach addresses the high variance problem inherent in current Chain-of-Thought optimization methods by using the model itself to estimate which parts of reasoning traces deserve higher rewards, without requiring expensive additional computation.
AINeutralarXiv – CS AI · Jun 56/10
🧠Researchers propose TAPO (Tool-Aware Policy Optimization), a method that fixes credit misassignment problems in reinforcement learning for multimodal search agents. The technique improves training efficiency for AI systems that use tools, delivering consistent improvements across multiple benchmarks without requiring additional annotations or computational overhead.
AINeutralarXiv – CS AI · Jun 46/10
🧠Researchers identify Trace-Mediated Peak Bias (TMPB), a systematic failure in deep reinforcement learning where agents irrationally prioritize high-magnitude reward spikes over trajectories with greater cumulative returns. This phenomenon mirrors the human Peak-End Rule cognitive bias and reveals how mathematical constraints in credit assignment systems naturally produce human-like value distortions, with adaptive optimizers offering a potential solution.
AINeutralarXiv – CS AI · Jun 26/10
🧠Researchers propose ARCA, a new token-level credit assignment method for language model reinforcement learning that addresses degradation issues in parameter-efficient fine-tuning approaches like LoRA. By measuring where adapters actually modify hidden states rather than tracking output distribution shifts, ARCA provides non-degenerate credit signals competitive with existing baselines while requiring no additional learned components.
AINeutralarXiv – CS AI · Jun 26/10
🧠Researchers introduce SPADER, a reinforcement learning framework that enables large language models to discover multiple valid answers to complex questions through tool-augmented search. The system combines step-wise credit assignment with diversity-aware rewards to improve recall and F1 scores across multiple QA benchmarks.
AIBullisharXiv – CS AI · Jun 16/10
🧠Researchers introduce DecomposeR, a framework that trains language models to conduct deep research by explicitly representing plans as directed acyclic graphs rather than flat trajectories. The approach separates planning and execution into two distinct reinforcement learning stages, improving long-form answer generation by 5.1-8.0 points over comparable baselines on benchmark datasets.
AINeutralarXiv – CS AI · Jun 16/10
🧠Researchers introduce Score Broadcast and Decorrelation (SBD), a theoretical framework that generalizes biologically plausible credit assignment mechanisms across diverse loss functions beyond MSE. The framework unifies error broadcast—an alternative to backpropagation that avoids weight transport—under a single orthogonality principle, with experimental validation showing improvements over existing broadcast approaches on image classification tasks.
