AIBullisharXiv – CS AI · Jun 96/10
🧠Researchers propose Thinking-Based Non-Thinking (TNT), a novel approach to train hybrid reasoning models that dynamically choose between fast responses and extended reasoning without the reward hacking problems that plague existing reinforcement learning methods. The technique achieves approximately 50% token efficiency gains while maintaining or improving accuracy across mathematical benchmarks, addressing a critical bottleneck in deploying large reasoning models.
AIBullisharXiv – CS AI · Jun 86/10
🧠Researchers introduce PTD-PO, a novel framework that improves how large vision-language models learn through reinforcement learning by providing dense guidance without exposing correct answers. The method uses spatial attention hints and reasoning steps to supervise token-level learning, achieving better performance than existing approaches while avoiding shortcuts in model training.
AINeutralarXiv – CS AI · Jun 46/10
🧠Researchers introduce a reinforcement learning framework called Modality-Aware Credit Assignment (MoCA) that improves Vision-Language Models by separately identifying whether failures stem from perception errors or reasoning flaws. The approach uses Perception Verification and Structured Verbal Verification to enable targeted supervision and scalable training across diverse vision-language tasks.
AINeutralarXiv – CS AI · Jun 46/10
🧠This research examines how the Scale-Invariant Signal-to-Distortion Ratio (SI-SDR) metric used to train and evaluate speech separation models performs poorly when training data contains noise, revealing fundamental limitations in the current benchmark approach. The authors propose reference enhancement techniques to mitigate this issue, though results indicate that processing introduces artifacts that limit overall quality improvements.
AINeutralarXiv – CS AI · Jun 26/10
🧠Researchers propose S-SPPO, an improved framework for aligning large language models with human preferences that addresses instability issues in Self-Play Preference Optimization. The method uses semantic calibration techniques to prevent policy degradation when the model generates semantically similar responses, achieving competitive performance on AlpacaEval 2.0 without additional human annotations.
🧠 Llama
AINeutralarXiv – CS AI · Jun 26/10
🧠Researchers propose Trajectory-aware On-Policy Distillation (TOPD), a method that improves large language model reasoning by using near-future trajectory information to identify genuine reasoning divergences rather than surface-level token mismatches. The technique achieves significant performance gains on mathematical reasoning benchmarks, improving AIME24 scores from 60.0% to 63.3%.
AINeutralarXiv – CS AI · Jun 26/10
🧠A comprehensive academic primer synthesizes over 150 studies on post-training reasoning data for large language models, organizing the field around four core questions: what data objects exist, what makes them useful, how they are constructed, and how they scale. This foundational work provides an attribution framework for future reasoning-data releases and post-training approaches in AI development.
AIBullisharXiv – CS AI · Jun 16/10
🧠Researchers propose S2L-PO, a framework that uses smaller language models as natural policy explorers to train larger models more efficiently. By leveraging the inherent policy-level diversity of smaller models rather than token-level randomness, the approach achieves significant accuracy improvements on mathematical reasoning tasks while reducing computational costs.
AINeutralarXiv – CS AI · Jun 16/10
🧠Researchers propose DareU, a novel LLM unlearning framework that uses data attribution rewards and reinforcement learning to remove training data influence from large language models. Unlike existing approaches that maximize loss on forget sets, this method reduces attribution scores to forgotten data owners, addressing critical issues of over-forgetting and model utility degradation.
AINeutralarXiv – CS AI · Jun 16/10
🧠A new study finds that language models can improve by learning from their own generated text, but only when the synthetic data is compatible with the student model's existing capabilities. The research reveals that synthetic data utility is relational rather than intrinsic, and surprisingly, this self-training approach can reduce verbatim memorization by 95% without explicit unlearning objectives.
AINeutralarXiv – CS AI · Jun 15/10
🧠Researchers propose Trust-Region behavior Blending (TRB), a warmup technique that improves on-policy distillation by having student models learn from a teacher-aligned policy during early training stages rather than weak student rollouts. The method anneals the constraint over time until training returns to pure student policy, demonstrating stronger performance in math-reasoning tasks.
AINeutralarXiv – CS AI · Jun 16/10
🧠Researchers introduce PlanningBench, a framework for generating scalable and verifiable planning datasets to evaluate and train large language models on complex task coordination. The system uses a constraint-driven synthesis pipeline with adaptive difficulty control and finds that current frontier LLMs struggle with coupled constraints, though reinforcement learning on verified data improves performance across planning and instruction-following tasks.
AINeutralarXiv – CS AI · Jun 16/10
🧠Researchers introduce REAL, a reinforcement learning framework that optimizes LLMs used as automated evaluators by recognizing ordinal relationships in scoring tasks rather than treating outputs as binary outcomes. The method demonstrates significant performance improvements across model scales, achieving up to +8.40 Pearson correlation gains on Qwen3-32B compared to supervised fine-tuning baselines.
AINeutralarXiv – CS AI · May 296/10
🧠Researchers identify a critical failure mode in masked diffusion language models where confidence-based decoding strategies cause reasoning errors on complex tasks. The study demonstrates that confidence-aligned training amplifies these failures by an order of magnitude, while random masking preserves robust reasoning capabilities across five reasoning tasks.
AINeutralarXiv – CS AI · May 296/10
🧠Researchers propose EKSFT, a novel fine-tuning method that selectively masks high-entropy and high-KL divergence tokens during supervised fine-tuning of large language models. The approach aims to preserve pre-trained model distributions while efficiently activating task-relevant capabilities in low-data regimes, demonstrating improved performance on mathematical reasoning benchmarks.
AINeutralarXiv – CS AI · May 296/10
🧠Researchers compared two automatic label error detection methods—Confident Learning and Dataset Cartography—for filtering noisy training data in Russian text classification tasks. The study reveals that filtering effectiveness depends heavily on dataset characteristics, with significant improvements only on small, noisy datasets, while larger corpora with low noise show no benefit from filtering.
AINeutralarXiv – CS AI · May 296/10
🧠Researchers introduce GASP, a framework that enhances Vision-Language Models' 3D spatial reasoning by injecting geometric priors directly into transformer layers rather than relying on 3D VQA datasets. The approach uses contrastive learning on point correspondences and depth consistency supervision, achieving 70%+ correspondence accuracy and 18-29% improvements on spatial benchmarks without any 3D VQA training data.
AIBullisharXiv – CS AI · May 296/10
🧠Researchers propose Canonical-Context On-Policy Distillation (CCOPD), a training method that improves large language models' ability to solve problems when information is revealed incrementally across multiple conversation turns rather than all at once. By using a frozen teacher model with complete context to guide a student model receiving fragmented information, CCOPD achieves 32% relative performance improvement on multi-turn tasks while maintaining single-prompt performance.
AINeutralarXiv – CS AI · May 286/10
🧠IRDS introduces a new data selection method for reinforcement learning with verifiable rewards (RLVR) that uses sparse autoencoders to identify interpretable, high-value training instances. The approach achieves significant accuracy improvements on math reasoning benchmarks while reducing computational costs by an order of magnitude compared to existing methods.
🧠 Llama
AINeutralarXiv – CS AI · May 286/10
🧠Researchers present a novel framework analyzing how reinforcement learning (RL) and supervised fine-tuning (SFT) differently shape reasoning in large language models. The study reveals that RL compresses incorrect reasoning paths while SFT expands correct ones, explaining why the two-stage training approach produces superior reasoning capabilities across models of 1.5B to 14B parameters.
AINeutralarXiv – CS AI · May 276/10
🧠Researchers introduce GAC, a noise-aware adaptive controller that optimizes the mixing of supervised fine-tuning and reinforcement learning during AI model post-training. By dynamically adjusting mixing weights based on gradient variance and signal disagreement, GAC outperforms fixed schedules across math, code, science, and logic tasks with minimal computational overhead.
AINeutralarXiv – CS AI · May 126/10
🧠Researchers present a mathematical framework quantifying the value of brain imaging data for training machine learning models, deriving scaling laws that establish exchange rates between neural recordings and task samples. The work identifies specific conditions where brain data improves model performance and robustness, providing theoretical foundations for when neural data collection is economically justified.
AINeutralarXiv – CS AI · May 126/10
🧠AdaPreLoRA addresses a fundamental challenge in fine-tuning large language models by proposing a new optimization method that combines Adafactor preconditioning with Low-Rank Adaptation. The technique achieves competitive or superior performance across multiple benchmarks while maintaining memory efficiency comparable to standard LoRA optimizers.
AIBullisharXiv – CS AI · May 116/10
🧠Researchers demonstrate that ProteinJEPA, a latent-space prediction technique, can complement traditional masked language modeling (MLM) in protein language models, achieving better downstream task performance when combined strategically. The optimal approach—masked-position MLM+JEPA—wins 10 out of 16 evaluation tasks against MLM-only baselines while maintaining computational efficiency.
AINeutralarXiv – CS AI · May 116/10
🧠Researchers propose vOPD (On-Policy Distillation with control variate baseline), a stabilization technique for training large language models that reduces gradient variance without adding computational overhead. The method leverages reinforcement learning principles to make on-policy distillation more reliable and efficient, matching expensive full-vocabulary baselines while maintaining lightweight single-sample estimation.
