AINeutralarXiv – CS AI · 18h ago6/10
🧠
Understanding Quantization-Aware Training: Gradients at Quantized Weights Bias to the Low-Loss Basin
Researchers propose a geometric framework explaining why post-training quantization (PTQ) fails at aggressive bitwidths while quantization-aware training (QAT) succeeds in recovery. The study reveals that gradients in QAT acquire an inward bias toward low-loss regions, enabling quantized neural networks to maintain accuracy where simpler PTQ methods collapse.