AIBullisharXiv – CS AI · Jun 197/10
🧠Researchers demonstrate that the Lean proof assistant can provide fine-grained, process-level feedback during reinforcement learning training for theorem proving, beyond simple binary verification signals. By parsing proof attempts into tactic sequences and leveraging Lean's elaboration system, the approach delivers dense, verified credit signals grounded in type theory, showing improvements over outcome-only baselines on benchmarks like MiniF2F and ProofNet.
AIBullisharXiv – CS AI · Jun 47/10
🧠Researchers introduce SoLoPO, a framework that improves how large language models handle long-context information by decoupling preference optimization into short-context training and short-to-long reward alignment. The approach addresses fundamental limitations in LLM long-context capabilities while improving training efficiency and computational requirements.
AIBullisharXiv – CS AI · Jun 27/10
🧠Researchers introduce Expected Value Alignment (EVA), a novel reward-modeling technique that enables Large Language Models to provide continuous numerical scores while maintaining human-readable text output for formal mathematics verification in Lean 4. The method bridges a critical gap between discrete generative outputs and continuous value assessment needed for reinforcement learning in theorem proving systems.
AIBullisharXiv – CS AI · Jun 27/10
🧠Researchers introduce MAPR, a meta-awareness framework that enhances reasoning models by predicting task statistics (length, pass-rate, concepts) rather than relying solely on answer verification. The method achieves 83.18% accuracy gains on AIME25 and 13.04% average improvement across mathematics benchmarks while accelerating training efficiency by 1.28x.
AIBullisharXiv – CS AI · May 127/10
🧠Researchers introduce Auto-Rubric as Reward (ARR), a framework that replaces opaque scalar reward signals in multimodal AI alignment with explicit, structured criteria-based evaluation. By externalizing a model's implicit preferences into interpretable rubrics before comparison, ARR reduces evaluation bias and enables more reliable human-preference alignment in generative models.
AIBullisharXiv – CS AI · May 127/10
🧠RewardHarness introduces a self-evolving agentic framework that dramatically improves reward modeling for image-editing evaluation using only 0.05% of typical training data. By iteratively refining tools and skills from minimal examples rather than large-scale annotations, the system achieves 47.4% accuracy on benchmarks, outperforming GPT-5 and enabling more efficient AI alignment.
🧠 GPT-5
AIBullisharXiv – CS AI · May 117/10
🧠Researchers introduce rubric-grounded reinforcement learning, a framework that trains AI models using structured, multi-criterion rewards from an LLM judge rather than binary outcomes. Training Llama-3.1-8B on scientific documents achieved 71.7% normalized reward and demonstrated improved performance on multiple reasoning benchmarks, suggesting that document-grounded training signals can produce generalizable reasoning capabilities.
🧠 Llama
AIBullisharXiv – CS AI · May 97/10
🧠Researchers propose CAMEL, a new reward modeling framework that combines efficient single-token preference decisions with selective reflection for low-confidence cases, achieving 82.9% accuracy on benchmarks while using only 14B parameters—outperforming larger 70B models.
AIBullisharXiv – CS AI · Apr 207/10
🧠Researchers introduce AgentV-RL, an agentic verifier framework that enhances reward modeling for large language models by combining bidirectional reasoning agents with tool-use capabilities. The system addresses critical limitations in LLM verification by enabling forward and backward tracing of solutions, achieving 25.2% performance gains over existing methods and positioning agentic reward modeling as a promising new paradigm.
AIBullisharXiv – CS AI · Apr 157/10
🧠Researchers propose a label-free self-supervised reinforcement learning framework that enables language models to follow complex multi-constraint instructions without external supervision. The approach derives reward signals directly from instructions and uses constraint decomposition strategies to address sparse reward challenges, demonstrating strong performance across both in-domain and out-of-domain instruction-following tasks.
AIBullisharXiv – CS AI · Apr 137/10
🧠Researchers introduce a listener-augmented reinforcement learning framework for training vision-language models to better align with human visual preferences. By using an independent frozen model to evaluate and validate reasoning chains, the approach achieves 67.4% accuracy on ImageReward benchmarks and demonstrates significant improvements in out-of-distribution generalization.
🏢 Hugging Face
AIBullisharXiv – CS AI · Mar 97/10
🧠Researchers introduce RM-R1, a new class of Reasoning Reward Models (ReasRMs) that integrate chain-of-thought reasoning into reward modeling for large language models. The models outperform much larger competitors including GPT-4o by up to 4.9% across reward model benchmarks by using a chain-of-rubrics mechanism and two-stage training process.
🧠 GPT-4🧠 Llama
AIBullisharXiv – CS AI · Mar 56/10
🧠Researchers propose a new framework called Critic Rubrics to bridge the gap between academic coding agent benchmarks and real-world applications. The system learns from sparse, noisy human interaction data using 24 behavioral features and shows significant improvements in code generation tasks including 15.9% better reranking performance on SWE-bench.
AIBullisharXiv – CS AI · Mar 47/104
🧠Researchers present a new mathematical framework for training AI reward models using Likert scale preferences instead of simple binary comparisons. The approach uses ordinal regression to better capture nuanced human feedback, outperforming existing methods across chat, reasoning, and safety benchmarks.
AIBullisharXiv – CS AI · Mar 37/103
🧠Researchers introduce Robometer, a new framework for training robot reward models that combines progress tracking with trajectory comparisons to better learn from failed attempts. The system is trained on RBM-1M, a dataset of over one million robot trajectories including failures, and shows improved performance across diverse robotics applications.
AINeutralarXiv – CS AI · Jun 236/10
🧠ARCO introduces an adaptive rubric framework that enables large language model agents to receive step-level interpretable rewards during multi-step reasoning tasks. By jointly evolving the reward rubric and policy through co-training, the method achieves stronger performance on question-answering benchmarks while providing explainable feedback that clarifies why each step in a trajectory succeeds or fails.
AIBullisharXiv – CS AI · Jun 236/10
🧠EvoRubrics introduces a co-evolutionary reinforcement learning framework where a Policy LLM and Rubric Generator jointly improve through adversarial interaction, addressing the limitation of static reward criteria that lose discriminative power as models improve. The approach enables real-time evaluation adaptation and generates transferable reward models, with experiments showing consistent improvements over static and dynamic baselines.
AINeutralarXiv – CS AI · Jun 236/10
🧠Researchers introduce PrivacyAlign, a dataset and training methodology that improves how large language model agents handle privacy decisions by grounding them in human judgment. The work demonstrates that conditioning LLM judges on human annotations and using annotation-based reward modeling produces agents better aligned with actual user privacy expectations across diverse scenarios.
AINeutralarXiv – CS AI · Jun 236/10
🧠Researchers introduce DiT-Reward, a reward model derived from pretrained Diffusion Transformers that outperforms existing benchmarks like HPSv3 for evaluating text-to-image generation quality. The approach demonstrates that representations learned during generative model training transfer effectively to reward prediction tasks, achieving measurable improvements in preference prediction accuracy and inference speed.
🧠 Stable Diffusion
AINeutralarXiv – CS AI · Jun 236/10
🧠Researchers introduce RARM (Reference-Anchored Reward Model), a visual AI system that solves a major bottleneck in robot learning by converting single successful demonstrations into dense reward signals without task-specific engineering. The approach uses confidence-gated progress matching to avoid false-positive rewards, achieving superior performance across simulated and real-world manipulation tasks.
AINeutralarXiv – CS AI · Jun 196/10
🧠Researchers propose Uncertainty-Aware Reward Modeling (UARM), a technique that addresses critical vulnerabilities in RLHF training by equipping reward models with calibrated uncertainty estimates and reweighting policy optimization to prevent reward hacking. The method uses quantile-based conformal prediction and heteroscedastic variance decomposition, demonstrating improved alignment quality across multiple benchmark datasets.
AIBullisharXiv – CS AI · Jun 116/10
🧠Researchers introduce PRInTS, a generative process reward model designed to improve AI agents' ability to perform multi-step information-seeking tasks over long horizons. By combining dense scoring across multiple quality dimensions with trajectory summarization, PRInTS enables smaller language models to match or exceed frontier model performance on complex reasoning benchmarks.
AINeutralarXiv – CS AI · Jun 106/10
🧠Researchers introduce TRACE, a rollout budget allocation framework that improves reinforcement learning for large language models by optimizing reward signals across multi-turn agentic tasks. The method allocates computational resources to both initial prompts and intermediate decision points within conversations, demonstrating 2.8-point accuracy improvements on benchmarks at equivalent sampling costs.
AINeutralarXiv – CS AI · Jun 96/10
🧠Researchers propose PAFO, a Pareto fairness optimization framework that addresses bias in personalized reward models for large language models by improving performance for under-served user preference groups without degrading majority groups. The method uses group-specialized models and conditional margin-level supervision to create fairer LLM alignment across diverse user populations.
AINeutralarXiv – CS AI · Jun 96/10
🧠Researchers propose T²-GRPO, a reinforcement learning framework that optimizes large language models for dementia caregiver agents by balancing immediate patient feedback with long-term care outcomes. The method uses environment-grounded rewards and safety constraints to improve emotional intelligence in AI caregiving scenarios.
