3 articles tagged with #gradient-estimation. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers introduce Dynamic Pruning Policy Optimization (DPPO), a new framework that accelerates AI language model training by 2.37x while maintaining accuracy. The method addresses computational bottlenecks in Group Relative Policy Optimization through unbiased gradient estimation and improved data efficiency.
Researchers propose improved divergence measures for training Generative Flow Networks (GFlowNets), comparing Renyi-ฮฑ, Tsallis-ฮฑ, and KL divergences to enhance statistical efficiency. The work introduces control variates that reduce gradient variance and achieve faster convergence than existing methods, bridging GFlowNets training with generalized variational inference frameworks.
Researchers propose CAGE (Confidence-Adaptive Gradient Estimation) to solve the training-inference mismatch problem in neural networks that use soft mixtures during training but hard selection during inference. The method achieves over 98% accuracy on MNIST with zero selection gap, significantly outperforming existing approaches like Gumbel-ST which suffers accuracy collapse.
