Quantization Inflates Reasoning: Token Inflation as a Hidden Cost of Low-Bit Reasoning Models
Researchers demonstrate that low-bit quantization of reasoning models introduces a hidden cost: quantized models generate significantly longer chains of thought to maintain accuracy, offsetting per-token speedup gains. The study introduces metrics to measure this token inflation and finds quantization-aware training as the most effective mitigation strategy.

