AINeutralarXiv – CS AI · Jun 26/10
🧠Researchers propose CAST, a new self-distillation method for reinforcement learning in large language models that improves upon existing approaches by using answer-free teacher scoring and bidirectional advantage flipping. The method addresses limitations in Group Relative Policy Optimization (GRPO) by providing denser token-level guidance while maintaining alignment with trajectory correctness, demonstrating improvements in mathematical reasoning tasks.
AIBullisharXiv – CS AI · Jun 26/10
🧠Researchers propose Chunk-Level Guided Generation, a training-free method using off-the-shelf large language models to score intermediate reasoning steps during small-model inference for mathematical problem-solving. The approach matches or outperforms specialized reward model-based systems on benchmarks like MATH and GSM8K without requiring expensive step-level training data.
🧠 Llama
AINeutralarXiv – CS AI · Jun 26/10
🧠Researchers introduce MulFeRL, a reinforcement learning framework that uses multi-turn verbal feedback to improve AI reasoning on failed tasks. By converting qualitative feedback into trainable signals and assigning credit for incremental progress, the approach outperforms traditional reward-based methods on math problems and generalizes well to unseen domains.
AIBullisharXiv – CS AI · Jun 16/10
🧠Researchers propose S2L-PO, a framework that uses smaller language models as natural policy explorers to train larger models more efficiently. By leveraging the inherent policy-level diversity of smaller models rather than token-level randomness, the approach achieves significant accuracy improvements on mathematical reasoning tasks while reducing computational costs.
AINeutralarXiv – CS AI · May 296/10
🧠Researchers compared how large language models rate the interestingness of math problems against human judgments from college students and International Math Olympiad competitors. While LLMs show broad agreement with humans, they fail to match the distribution of human preferences and poorly explain why problems are interesting, though they can generate novel engaging problems after validity filtering.
AIBullisharXiv – CS AI · May 286/10
🧠Researchers propose LaneRoPE, a novel technique that enables multiple parallel language model sequences to coordinate and share information during generation, improving reasoning accuracy without significant architectural changes or inference overhead.
AINeutralarXiv – CS AI · May 286/10
🧠Researchers propose Sequential Bayesian Belief Tracking (SBBT), a framework for estimating the reliability of long reasoning chains in large language models before final answers are known. The study finds that probability calibration and ranking performance respond differently to various evidence types: scalar scores improve calibration metrics, while structural observations are needed for ranking tasks.
AINeutralarXiv – CS AI · May 286/10
🧠Researchers demonstrate that Lean formal proof verification produces unreliable signals for validating natural-language mathematical reasoning, with accuracy varying from 96% at high coverage to 20% at low coverage. They introduce COVCAL, a risk-control method that certifies when partial formal signals can be trusted, showing that feasibility depends critically on autoformalization quality and coverage rates.
AIBullisharXiv – CS AI · May 286/10
🧠Researchers propose Skill-Conditioned Gated Self-Distillation (SGSD), a novel method for improving large language model reasoning by leveraging an experience-derived skill bank rather than trusted reference answers. The approach validates skills through a multi-teacher framework and demonstrates consistent improvements over existing methods on mathematical reasoning benchmarks.
AINeutralarXiv – CS AI · May 286/10
🧠Researchers identify a critical failure mode in test-time reinforcement learning (TTRL) where majority voting locks onto incorrect answers, permanently suppressing correct signals in low-ability problems. They introduce TTRL-Guard, a framework using flip-rate monitoring and selective updating to prevent this 'Correct-Answer Extinction Window,' achieving 54% relative improvement on AIME 2025 benchmarks.
AINeutralarXiv – CS AI · May 276/10
🧠A new study comparing three LLM approaches to mathematical reasoning found that pure chain-of-thought prompting outperforms code execution methods in robustness across problem variations. When math problems were modified with simple changes like different names or numbers, code-based approaches showed greater accuracy drops, challenging the assumption that code execution improves reasoning reliability.
🧠 Claude🧠 Haiku
AINeutralarXiv – CS AI · May 276/10
🧠Researchers propose Token-to-Mask (T2M) remasking as an improved alternative to Token-to-Token editing in discrete diffusion language models, addressing fundamental limitations in error detection and context corruption. The method resets suspected erroneous tokens to mask state for re-prediction, demonstrating 5.92% improvement on mathematical benchmarks and fixing 59.4% of final-answer corruption cases.
AIBullisharXiv – CS AI · May 276/10
🧠Researchers propose PTA-GRPO, a two-stage framework that enhances LLM reasoning by combining high-level planning with reinforcement learning. The method first guides models to summarize reasoning into compact guidance, then uses this guidance to optimize both final outputs and reasoning quality, demonstrating consistent improvements across ten benchmarks.
AINeutralarXiv – CS AI · May 276/10
🧠Researchers introduce ORLoopBench, a benchmark suite that evaluates large language models on Operations Research tasks through an iterative solver-in-the-loop process rather than one-shot code generation. The framework enables models to debug infeasible mathematical models by inspecting constraint conflicts and repairing formulations, with an 8B model achieving 95.3% success on LP repair tasks—outperforming frontier APIs at 92.4%.
AIBullisharXiv – CS AI · May 126/10
🧠Researchers propose VIGOR, a verifier-free reinforcement learning method for large language models that eliminates dependency on gold labels or domain-specific verifiers by using gradient-norm measurements as intrinsic reward signals. The approach demonstrates measurable improvements over existing baselines on mathematical reasoning and exhibits cross-domain transfer to code tasks, addressing a major scalability constraint in current RL-based LLM training.
AINeutralarXiv – CS AI · May 126/10
🧠Researchers introduce Re²Math, a new benchmark for evaluating large language models' ability to retrieve relevant mathematical theorems and lemmas from academic literature during proof construction. The benchmark reveals significant gaps in current AI systems, with the best model achieving only 7.0% accuracy despite retrieving valid statements, indicating AI struggles to verify applicability to specific proof contexts.
AINeutralarXiv – CS AI · May 126/10
🧠Researchers introduce a strategy-level evaluation framework for large language models on mathematical reasoning tasks, revealing a significant gap between high answer accuracy and actual reasoning flexibility. While frontier models achieve 95-100% accuracy on single-solution prompts, they recover substantially fewer problem-solving strategies than human references when asked to generate multiple approaches, with only 39-71% coverage depending on the model and iteration count.
🧠 Claude🧠 Gemini
AINeutralarXiv – CS AI · May 126/10
🧠NoisyCoconut is an inference-time method that improves LLM reliability by injecting controlled noise into internal representations to generate diverse reasoning paths, enabling models to abstain when uncertain without requiring retraining. The technique reduces error rates from 40-70% to below 15% on mathematical reasoning tasks through unanimous agreement among noise-perturbed paths, offering practical reliability improvements compatible with existing models.
AINeutralarXiv – CS AI · May 126/10
🧠Researchers introduce AIPO, a reinforcement learning framework that enhances large language model reasoning by enabling active consultation with collaborative agents during training. The method addresses exploration limitations in current RL approaches and demonstrates consistent performance improvements across multiple mathematical and coding benchmarks.
AIBullisharXiv – CS AI · May 116/10
🧠Researchers propose GXPO, a new policy optimization technique for reinforcement learning that approximates multi-step lookahead using only three backward passes instead of many, improving large language model reasoning performance by 1.65-5.00 points over standard GRPO while achieving up to 4x step speedup.
🧠 Llama
AINeutralarXiv – CS AI · May 116/10
🧠Researchers propose using conditional optimal transport to improve calibration of Process Reward Models (PRMs) used in AI inference-time scaling, addressing the problem of overestimated success probabilities. The method enables better confidence bounds for mathematical reasoning tasks and improves downstream performance in Best-of-N selection frameworks.
AIBullisharXiv – CS AI · May 116/10
🧠Researchers propose CTPO (Cumulative Token Policy Optimization), a new approach to reinforcement learning for large language models that addresses the bias-variance tradeoff in importance sampling ratios. By using cumulative token-level ratios with position-adaptive clipping, CTPO achieves superior performance on mathematical reasoning benchmarks compared to existing methods like PPO and GRPO.
AINeutralarXiv – CS AI · May 96/10
🧠Researchers demonstrate that On-Policy Self-Distillation (OPSD) functions primarily as a compression mechanism rather than a correction tool for thinking-enabled mathematical reasoning models. They propose a revised training pipeline (SFT → RLVR → OPSD) that leverages OPSD's strengths in shortening responses while preserving accuracy on correct outputs.
AIBullisharXiv – CS AI · May 96/10
🧠Researchers introduce VHG, a verifier-enhanced framework that improves how large language models generate valid and challenging mathematical problems through three-party self-play involving a setter, solver, and independent verifier. The approach addresses critical limitations in existing problem generation methods by constraining reward signals to ensure both problem validity and difficulty, demonstrating substantial improvements over baseline approaches.
AINeutralarXiv – CS AI · Apr 206/10
🧠Researchers introduce SAI-DPO, a dynamic data sampling framework that adapts training data selection based on a model's evolving capabilities during training, rather than using static metrics. Tested on mathematical reasoning benchmarks including AIME24 and AMC23, SAI-DPO achieves state-of-the-art performance with significantly less training data, outperforming baselines by nearly 6 points.
