5 articles tagged with #optimizer. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers propose GradientStabilizer, a new technique to address training instability in deep learning by replacing gradient magnitude with statistically stabilized estimates while preserving direction. The method outperforms gradient clipping across multiple AI training scenarios including LLM pre-training, reinforcement learning, and computer vision tasks.
A critical bug was discovered in Solidity's optimizer and ABIEncoderV2 that could affect smart contract security and functionality. The vulnerability impacts contracts using specific optimization settings and encoding features, potentially leading to unexpected behavior or security risks.
A bug in the Solidity optimizer was discovered and reported through the Ethereum Foundation Bounty program by Christoph Jentzsch. The vulnerability has been patched with the release of Solidity version 0.4.11 on May 3, 2017.
Researchers introduce Velocity-Regularized Adam (VRAdam), a physics-inspired optimizer that improves deep neural network training by adding velocity-based regularization to prevent oscillations and instability. VRAdam demonstrates superior performance compared to standard optimizers like AdamW across multiple benchmarks including image classification, language modeling, and generative modeling tasks.
Researchers introduce Pro-KLShampoo, an improved optimizer for LLM pre-training that combines Kronecker-factored preconditioning with gradient orthogonalization. By exploiting the observed spike-and-flat eigenvalue structure in KL-Shampoo's preconditioners, Pro-KLShampoo achieves better validation loss, reduced memory usage, and faster training across multiple model scales.
