114 articles tagged with #model-training. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers compared two automatic label error detection methods—Confident Learning and Dataset Cartography—for filtering noisy training data in Russian text classification tasks. The study reveals that filtering effectiveness depends heavily on dataset characteristics, with significant improvements only on small, noisy datasets, while larger corpora with low noise show no benefit from filtering.
Researchers propose EKSFT, a novel fine-tuning method that selectively masks high-entropy and high-KL divergence tokens during supervised fine-tuning of large language models. The approach aims to preserve pre-trained model distributions while efficiently activating task-relevant capabilities in low-data regimes, demonstrating improved performance on mathematical reasoning benchmarks.
Researchers propose Canonical-Context On-Policy Distillation (CCOPD), a training method that improves large language models' ability to solve problems when information is revealed incrementally across multiple conversation turns rather than all at once. By using a frozen teacher model with complete context to guide a student model receiving fragmented information, CCOPD achieves 32% relative performance improvement on multi-turn tasks while maintaining single-prompt performance.
Researchers introduce GASP, a framework that enhances Vision-Language Models' 3D spatial reasoning by injecting geometric priors directly into transformer layers rather than relying on 3D VQA datasets. The approach uses contrastive learning on point correspondences and depth consistency supervision, achieving 70%+ correspondence accuracy and 18-29% improvements on spatial benchmarks without any 3D VQA training data.
Researchers identify a critical failure mode in masked diffusion language models where confidence-based decoding strategies cause reasoning errors on complex tasks. The study demonstrates that confidence-aligned training amplifies these failures by an order of magnitude, while random masking preserves robust reasoning capabilities across five reasoning tasks.
Researchers present a novel framework analyzing how reinforcement learning (RL) and supervised fine-tuning (SFT) differently shape reasoning in large language models. The study reveals that RL compresses incorrect reasoning paths while SFT expands correct ones, explaining why the two-stage training approach produces superior reasoning capabilities across models of 1.5B to 14B parameters.
IRDS introduces a new data selection method for reinforcement learning with verifiable rewards (RLVR) that uses sparse autoencoders to identify interpretable, high-value training instances. The approach achieves significant accuracy improvements on math reasoning benchmarks while reducing computational costs by an order of magnitude compared to existing methods.
Researchers introduce GAC, a noise-aware adaptive controller that optimizes the mixing of supervised fine-tuning and reinforcement learning during AI model post-training. By dynamically adjusting mixing weights based on gradient variance and signal disagreement, GAC outperforms fixed schedules across math, code, science, and logic tasks with minimal computational overhead.
Researchers present a mathematical framework quantifying the value of brain imaging data for training machine learning models, deriving scaling laws that establish exchange rates between neural recordings and task samples. The work identifies specific conditions where brain data improves model performance and robustness, providing theoretical foundations for when neural data collection is economically justified.
AdaPreLoRA addresses a fundamental challenge in fine-tuning large language models by proposing a new optimization method that combines Adafactor preconditioning with Low-Rank Adaptation. The technique achieves competitive or superior performance across multiple benchmarks while maintaining memory efficiency comparable to standard LoRA optimizers.
Researchers propose Shadow Mask Distillation to address the memory bottleneck created by KV cache compression during reinforcement learning post-training of large language models. The technique tackles the critical off-policy bias that emerges when compressed contexts are used during rollout generation while full contexts are used for parameter updates, a problem that amplifies instability in RL optimization.
Researchers introduce Mutual Reinforcement Learning, a framework enabling heterogeneous language models to share training experiences while maintaining separate parameters and tokenizers. The system uses three mechanisms—Shared Experience Exchange, Multi-Worker Resource Allocation, and a Tokenizer Heterogeneity Layer—to coordinate reinforcement learning across incompatible model architectures, with outcome-level success transfer showing the best stability-support trade-off.
Researchers demonstrate that ProteinJEPA, a latent-space prediction technique, can complement traditional masked language modeling (MLM) in protein language models, achieving better downstream task performance when combined strategically. The optimal approach—masked-position MLM+JEPA—wins 10 out of 16 evaluation tasks against MLM-only baselines while maintaining computational efficiency.
Researchers propose vOPD (On-Policy Distillation with control variate baseline), a stabilization technique for training large language models that reduces gradient variance without adding computational overhead. The method leverages reinforcement learning principles to make on-policy distillation more reliable and efficient, matching expensive full-vocabulary baselines while maintaining lightweight single-sample estimation.
Researchers introduce MaPPO, a new preference optimization method for large language models that integrates prior reward knowledge into the training objective. Building on Direct Preference Optimization (DPO), MaPPO demonstrates consistent improvements across multiple benchmarks while maintaining computational efficiency and compatibility with existing DPO variants.
Researchers introduce UniSD, a unified self-distillation framework that systematically improves large language model adaptation without requiring external teacher models. The framework combines multiple complementary mechanisms and demonstrates consistent performance gains of +5.4 points over baseline models across six benchmarks, advancing efficient LLM training techniques.
Researchers demonstrate that using the same optimizer during both pretraining and finetuning of large language models reduces catastrophic forgetting while maintaining or improving task performance. This "optimizer-model consistency" effect suggests optimizers create regularization patterns that preserve learned knowledge, with implications for efficient model adaptation strategies.
Researchers prove that supervised fine-tuning (SFT) and reinforcement learning (RL) cannot be decoupled during large language model post-training, as each method degrades the performance gains of the other. The theoretical findings, verified experimentally, challenge the widespread industry practice of alternating these two training approaches and suggest optimal RL duration exists to balance competing objectives.
Researchers introduce TUR-DPO, an improved method for aligning large language models with human preferences that incorporates reasoning topology and uncertainty awareness. Unlike standard Direct Preference Optimization, this approach evaluates not just answer correctness but the quality of the reasoning process, showing improvements across mathematical reasoning, factual QA, and dialogue tasks while maintaining training simplicity.
Researchers introduce PRISM, a three-stage training pipeline that addresses distributional drift in large multimodal models by inserting a distribution-alignment stage between supervised fine-tuning and reinforcement learning. The method uses a Mixture-of-Experts discriminator to correct perception and reasoning errors, achieving 4.4-6.0 percentage point improvements on multimodal benchmarks compared to standard SFT-to-RLVR approaches.
Researchers discover that post-trained language models experience systematic output diversity collapse, where fine-tuning methods reduce the variety of generated responses compared to base models. This collapse is determined during training by data composition choices and cannot be fixed through inference-time adjustments, with implications for scaling methods and creative AI applications.
Researchers introduce a multi-agent framework to map data lineage in large language models, revealing how post-training datasets evolve and interconnect. The analysis uncovers structural redundancy, benchmark contamination propagation, and proposes lineage-aware dataset construction to improve LLM training diversity and quality.
Researchers fine-tuned Qwen2.5-VL-32B, a leading open-source vision-language model, to improve its ability to autonomously perform web interactions through visual input alone. Using a two-stage training approach that addresses cursor localization, instruction sensitivity, and overconfidence bias, the model's success rate on single-click web tasks improved from 86% to 94%.
Researchers have optimized the Bielik v3 language models (7B and 11B parameters) by replacing universal tokenizers with Polish-specific vocabulary, addressing inefficiencies in morphological representation. This optimization reduces token fertility, lowers inference costs, and expands effective context windows while maintaining multilingual capabilities through advanced training techniques including supervised fine-tuning and reinforcement learning.
Researchers introduce MEDS, a memory-enhanced reward shaping framework that addresses a critical reinforcement learning failure mode where language models repeatedly generate similar errors. By tracking historical behavioral patterns and penalizing recurring mistake clusters, the method achieves consistent performance improvements across multiple datasets and models while increasing sampling diversity.
