y0news
AnalyticsDigestsSourcesTopicsRSSAICrypto

#low-precision-training News & Analysis

2 articles tagged with #low-precision-training. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

2 articles
AIBullisharXiv – CS AI · 6h ago7/10
🧠

Normalized Architectures are Natively 4-Bit

Researchers demonstrate that nGPT, a neural architecture that normalizes weights and hidden representations to a unit hypersphere, achieves stable 4-bit precision training without requiring additional quantization interventions. The approach leverages mathematical properties of dot products to maintain stronger signal-to-noise ratios, enabling efficient training of models up to 30B parameters.

AIBullisharXiv – CS AI · Mar 37/104
🧠

A Convergence Analysis of Adaptive Optimizers under Floating-point Quantization

Researchers introduce the first theoretical framework analyzing convergence of adaptive optimizers like Adam and Muon under floating-point quantization in low-precision training. The study shows these algorithms maintain near full-precision performance when mantissa length scales logarithmically with iterations, with Muon proving more robust than Adam to quantization errors.