#quantization-aware-training News & Analysis

3 articles tagged with #quantization-aware-training. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

3 articles

AIBullisharXiv – CS AI · Jun 87/10

🧠

Scale When Needed: Adaptive Neuron-level Mixed Precision Quantization Aware Training

Researchers propose Neuron-Level Mixed-Precision Quantization Aware Training (NMP-QAT), a neural network compression technique that independently optimizes precision for individual neurons rather than entire layers. The method achieves better compression-accuracy trade-offs than existing approaches, making it particularly valuable for deploying AI models on resource-constrained edge devices in 6G networks.

AIBullisharXiv – CS AI · May 277/10

🧠

Max-Window Scale Estimation for Near-Lossless HiF8 W8A8 Quantization-Aware Training

Researchers develop a systematic approach to quantization-aware training for large language models using 8-bit floating-point formats, identifying and solving two critical failure modes—amax saturation and catastrophic forgetting—that don't surface in standard training metrics. Their solution achieves near-lossless performance with only 0.43% degradation on benchmark tasks, advancing practical LLM deployment efficiency.

AINeutralarXiv – CS AI · May 296/10

🧠

Singularity-aware Optimization via Randomized Geometric Probing: Towards Stable Non-smooth Optimization

Researchers introduce Singularity-aware Adam (S-Adam), a novel optimizer addressing instability in deep learning with non-smooth components like ReLU activations. The method uses a Local Geometric Instability metric to dynamically adjust step sizes, demonstrating up to 6% accuracy improvements on benchmark datasets while mitigating gradient oscillations.