16 articles tagged with #training-stability. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers propose Optimal Token Baseline (OTB), a new variance reduction technique for reinforcement learning in large language models that addresses training instability in long-horizon tasks. The method reduces token consumption by over 65% while maintaining performance equivalent to models using 8x larger batch sizes, offering significant efficiency gains for LLM-RL training.
Researchers identify a fundamental flaw in current FP4 training approaches for large language models: E2M1 formats suffer from systematic "Shrinkage Bias" that degrades training stability. They propose UFP4, a uniform 4-bit recipe using E1M2/INT4 grids that outperforms existing E2M1 baselines across multiple model scales, suggesting future AI accelerators should prioritize uniform grid formats for training.
Researchers propose Predictive Routing Replay (PR2), a technique to stabilize reinforcement learning training on Mixture of Experts LLMs by predicting router evolution and reducing the mismatch between rollout and training phases. The method addresses router drift—a critical instability source in MoE-based models undergoing RL fine-tuning—through lightweight prediction mechanisms that anticipate expert activation changes.
Researchers propose GradientStabilizer, a new technique to address training instability in deep learning by replacing gradient magnitude with statistically stabilized estimates while preserving direction. The method outperforms gradient clipping across multiple AI training scenarios including LLM pre-training, reinforcement learning, and computer vision tasks.
Researchers have developed Curvature-Aware Policy Optimization (CAPO), a new algorithm that improves training stability and sample efficiency for Large Language Models by up to 30x. The method uses advanced mathematical optimization techniques to identify and filter problematic training samples, requiring intervention on fewer than 8% of tokens.
Researchers developed Residual Koopman Spectral Profiling (RKSP), a method that predicts transformer training instability from a single forward pass at initialization with 99.5% accuracy. The technique includes Koopman Spectral Shaping (KSS) which can prevent training divergence and enable 50-150% higher learning rates across various AI models including GPT-2 and LLaMA-2.
Researchers propose Affine-Scaled Attention, a new mechanism that improves Transformer model training stability by introducing flexible scaling and bias terms to attention weights. The approach shows consistent improvements in optimization behavior and downstream task performance compared to standard softmax attention across multiple language model sizes.
FedSteer is a novel federated learning method that addresses gradient staleness in decentralized training systems where clients participate inconsistently. By projecting stale gradients onto a dynamically-maintained subspace and applying corrective techniques, the approach prevents training instability and achieves up to 7% accuracy improvements over existing baselines.
Researchers propose Group Relative Policy Optimization (GRPO), a baseline-free training algorithm for neural combinatorial optimization that eliminates the need for maintaining frozen policy copies. Testing on TSP and CVRP benchmarks shows GRPO prevents training collapse seen in standard REINFORCE while achieving competitive solution quality, offering a more stable alternative for routing problem optimization.
Researchers propose CPPO (Cumulative Prefix-divergence Policy Optimization), a new reinforcement learning method that improves upon standard PPO approaches for LLM training by accounting for position-dependent effects and cumulative policy divergence. The method uses position-weighted thresholds and prefix budgets to better regulate token-level deviations during autoregressive generation, showing improved training stability and reasoning accuracy across model scales.
Researchers reveal that Sharpness-Aware Minimization (SAM), a popular deep learning training method, has convergence instability near saddle points and may actually escape saddle points more poorly than standard gradient descent. The study demonstrates that momentum and batch-size adjustments are critical for mitigating these instabilities and achieving strong generalization performance.
Researchers propose a Conflict-aware Penalty and Statistical Loss framework to address gradient norm conflicts in multimodal sentiment analysis, where dominant text modalities suppress weaker acoustic and visual streams. The approach achieves state-of-the-art results on CMU-MOSI benchmarks by balancing modality contributions and stabilizing training dynamics.
Researchers introduce R²VPO, a new reinforcement learning method that replaces hard clipping mechanisms with ratio-variance regularization to improve policy optimization. Tested across large language models and robotic control tasks, the approach achieves better performance on mathematical reasoning and sample efficiency while maintaining stable learning.
Researchers propose a simple technique for stabilizing reinforcement learning training in PPO algorithms by randomly dropping 25% of transitions during rollouts. The method removes gradient redundancy caused by causally-dependent state sequences, improving training consistency across multiple environments without algorithmic modifications.
Researchers propose Quantile Advantage Estimation (QAE) to stabilize Reinforcement Learning with Verifiable Rewards (RLVR) for large language model reasoning. The method replaces mean baselines with group-wise K-quantile baselines to prevent entropy collapse and explosion, showing sustained improvements on mathematical reasoning tasks.
Researchers introduced heterogeneous time steps (HTS) for equilibrium propagation, a biologically plausible alternative to backpropagation for training neural networks. The approach assigns neuron-specific time constants based on biological distributions, improving training stability while maintaining competitive performance and enhancing biological realism.
