AIBullisharXiv – CS AI · Jun 37/10
🧠Researchers introduce EvoTrainer, an autonomous framework that co-evolves large language model policies and training harnesses through empirical feedback, matching or exceeding human-engineered reinforcement learning baselines across mathematical reasoning, code generation, and software engineering tasks. The approach moves beyond static recipe-based training to jointly optimize both policies and the training infrastructure that interprets them.
AIBullisharXiv – CS AI · Jun 27/10
🧠Researchers demonstrate that sparse neural networks can improve scaling efficiency in data-limited training scenarios, where models must train multiple epochs on repeated data. The study introduces a scaling law predicting performance across varying sparsity levels (up to 93.75%), finding that moderate sparsity around 50% optimizes loss while higher sparsity improves compute efficiency, challenging assumptions that sparsity is purely an efficiency tool.
AIBullisharXiv – CS AI · Jun 27/10
🧠Researchers introduce Subnetwork Data Parallelism (SDP), a distributed training framework that reduces memory consumption by 28-60% during neural network pre-training by partitioning models into structured subnetworks trained across workers without exchanging activations. The method supports both backward and forward masking regimes and maintains or improves performance across transformer and CNN architectures.
AINeutralarXiv – CS AI · Jun 27/10
🧠Researchers demonstrate that the slow power-law convergence observed during large language model training stems fundamentally from softmax and cross-entropy operations when learning peaked distributions. This universal 1/3 time scaling exponent represents an intrinsic optimization bottleneck that could explain neural scaling laws and potentially guide more efficient training methods.
AINeutralarXiv – CS AI · Jun 27/10
🧠Researchers propose On-Policy Critique Distillation (OPCD), a method enabling weak AI models to effectively supervise stronger ones by providing revision guidance rather than direct answers. The approach filters high-quality critiques and distills them into stronger models through adaptive learning, advancing scalable oversight for complex tasks.
AIBullisharXiv – CS AI · Jun 27/10
🧠Researchers propose Preference Delta Aggregation (PDA), a framework that combines weak preference signals from multiple smaller language model pairs into LoRA adapters, then merges them using Geometric Alignment Merging to improve larger models. The approach achieves 6.8-7.3 point improvements on knowledge reasoning and agentic search benchmarks by effectively composing complementary capabilities.
AIBullisharXiv – CS AI · Jun 17/10
🧠Researchers propose Feedback Distillation, a novel post-training method for language models that improves reasoning tasks by having models learn from their own feedback at the token level. Applied to Lean4 theorem-proving, the approach outperforms standard GRPO methods in trajectory diversity and scalability while complementing existing reinforcement learning approaches.
AIBullisharXiv – CS AI · Jun 17/10
🧠Researchers propose DeMix, a framework that uses model merging to efficiently determine optimal data mixtures for large language model pre-training without expensive repeated training cycles. The approach decouples the search process from training costs, enabling evaluation of multiple data combinations while also releasing a 22-token dataset to support open research.
AIBullisharXiv – CS AI · Jun 17/10
🧠Researchers propose DARTS, a novel approach to accelerate large language model reinforcement learning by reshaping the rollout distribution toward conciseness and certainty, reducing computational inefficiencies caused by long-tail response lengths. The method achieves up to 1.77x speedup through distribution-aware trajectory sampling without sacrificing model performance.
AIBullisharXiv – CS AI · May 297/10
🧠Researchers propose PEAR, a novel supervised fine-tuning (SFT) method that optimizes language models with downstream reinforcement learning in mind rather than in isolation. The approach uses importance sampling to reweight training data, addressing a critical distribution mismatch between offline SFT and online RL stages, achieving up to 14.6% performance gains on mathematical reasoning benchmarks.
AIBullisharXiv – CS AI · May 297/10
🧠Researchers propose ESPO, an optimization technique that improves large language model training by detecting and terminating failed reasoning trajectories early rather than forcing completion. The method reduces computational waste by over 20% while achieving superior performance on mathematical reasoning benchmarks compared to standard PPO training.
AIBullisharXiv – CS AI · May 297/10
🧠Researchers introduce PuzzleClone, a DSL-driven framework that automatically synthesizes large-scale, verifiable datasets for training LLMs on mathematical and logical reasoning tasks. The team generates PC-83K, a benchmark of 83,000+ diverse puzzles, and demonstrates that models fine-tuned on this dataset achieve substantial performance improvements across multiple logic and mathematical benchmarks.
AIBullisharXiv – CS AI · May 287/10
🧠SynthTools introduces an LLM-based pipeline for generating synthetic tool environments at scale, creating a dataset of 73,883 validated tools across 6,800 environments and 79,925 verifiable tasks. The framework demonstrates that agents trained on synthetic tool-use data can transfer capabilities to real APIs, addressing a critical bottleneck in agentic AI system development.
AIBullisharXiv – CS AI · May 287/10
🧠Researchers propose COSE, a self-evolution framework for large language models that uses confidence signals to filter noisy self-generated training feedback without external verifiers. The method demonstrates consistent improvements across 19 benchmarks and multiple model sizes (0.6B–4B parameters), achieving state-of-the-art performance in reasoning and mathematics tasks.
🧠 Llama
AIBullisharXiv – CS AI · May 277/10
🧠Researchers introduce SAERL, a data engineering framework that uses Sparse Autoencoders to extract intrinsic signals from LLM internals for improved reinforcement learning post-training. The method achieves 3% accuracy gains and 20% faster convergence on math reasoning tasks by modeling data diversity, difficulty, and quality—demonstrating that model internals provide practical signals beyond external training data metrics.
AIBullisharXiv – CS AI · May 277/10
🧠GraphDancer is a new post-training framework that enables large language models to reason over heterogeneous graph-structured data by combining natural-language reasoning with graph function execution. The two-stage curriculum approach uses structural complexity ordering to teach models to explore and reason over graphs, achieving strong cross-domain generalization with only a 3B parameter backbone.
AIBullisharXiv – CS AI · May 277/10
🧠Researchers propose GraphGPO, a novel reinforcement learning method that improves credit assignment in agentic tasks by aggregating trajectories into a state-transition graph rather than relying on coarse-grained outcome-based attribution. This approach enables step-level credit recognition and achieves state-of-the-art performance on challenging benchmarks while significantly improving training efficiency.
AIBullisharXiv – CS AI · May 277/10
🧠Researchers introduce MedGuideX, a medical language model trained on executable clinical decision logic extracted from practice guidelines, achieving 10.28% accuracy improvement over existing methods. The approach transforms procedural guideline structures into synthetic training data that teaches models both correct clinical decisions and counterfactual reasoning, with physician validation confirming improved explanation quality.
AIBullisharXiv – CS AI · May 277/10
🧠Researchers propose Divergence Proximal Policy Optimization (DPPO), a replacement for PPO's ratio clipping mechanism that better handles the large vocabularies in LLM fine-tuning. The new approach uses direct policy divergence estimates instead of noisy token probability ratios, offering improved training stability and efficiency.
AIBullisharXiv – CS AI · May 127/10
🧠Researchers introduce CauSim, a framework that enables large language models to improve causal reasoning by constructing increasingly complex executable causal simulators. The approach transforms causal reasoning from a scarce-data problem into a scalable supervised learning task, allowing LLMs to generate synthetic training data and demonstrate improved performance across different representations.
AIBullisharXiv – CS AI · May 127/10
🧠Researchers identify weight gradient (Wgrad) quantization as the primary cause of instability in FP4 training of large language models, while forward and activation gradient quantization prove relatively benign. Using deterministic Hadamard rotations on AMD MI355X GPUs, they demonstrate that structured micro-scaling errors—not insufficient randomness—drive training divergence, offering insights for efficient LLM pretraining.
🧠 Llama
AIBullisharXiv – CS AI · May 127/10
🧠Researchers introduce G-Zero, a verifier-free framework that enables large language models to improve autonomously through self-play without relying on external judges or proxy models. The approach uses an intrinsic reward mechanism called Hint-δ to identify and address the Generator model's blind spots, achieving scalable self-evolution across unverifiable domains.
AIBullisharXiv – CS AI · May 117/10
🧠Researchers introduce ROPD, a rubric-based on-policy distillation framework that replaces teacher logits with structured semantic rubrics for model alignment. The approach achieves up to 10x better sample efficiency than logit-based methods while enabling distillation from proprietary black-box LLMs, addressing a critical scalability limitation in current model training.
AIBullisharXiv – CS AI · May 117/10
🧠Researchers propose ESSAM, a novel training framework combining Evolution Strategies with Sharpness-Aware Maximization to fine-tune large language models for mathematical reasoning while dramatically reducing GPU memory requirements. The approach achieves comparable accuracy to reinforcement learning methods like PPO and GRPO while using 18-10× less memory, addressing a critical bottleneck in LLM development.
AIBullisharXiv – CS AI · May 117/10
🧠Researchers introduce rubric-grounded reinforcement learning, a framework that trains AI models using structured, multi-criterion rewards from an LLM judge rather than binary outcomes. Training Llama-3.1-8B on scientific documents achieved 71.7% normalized reward and demonstrated improved performance on multiple reasoning benchmarks, suggesting that document-grounded training signals can produce generalizable reasoning capabilities.
🧠 Llama
