Quantized Keys Steal Attention: Bias Correction for KV-Cache Compression in Video Diffusion

Researchers have developed a bias correction technique for quantizing KV-cache memory in video diffusion models, addressing a fundamental problem where quantization noise causes inflated attention to cached data. The method recovers near-full quality video generation while using 50% less memory than standard approaches, enabling longer video synthesis without sacrificing output quality.

Video diffusion models face a critical scalability challenge: as generated videos grow longer, the key-value (KV) cache storing previous chunks becomes prohibitively memory-intensive. While quantizing this cache to lower bit-widths reduces memory usage, it systematically degrades output quality through a phenomenon the researchers term Jensen bias—a mathematical artifact where softmax's exponential function amplifies the influence of quantization noise in cached keys, causing them to monopolize attention weights at the expense of current-chunk information.

This work emerges from the intersection of efficiency optimization and mathematical rigor in machine learning. Video generation systems increasingly rely on autoregressive chunk-wise approaches to manage computational complexity, but without memory-efficient caching solutions, these models remain impractical for commercial deployment. Previous quantization attempts treated KV-cache compression as a straightforward engineering problem, overlooking the theoretical underpinnings of why quantization specifically harms attention mechanisms.

The proposed correction mechanism directly addresses this bias through per-attention-score adjustments derived from cached key quantization step sizes and query norms. Critically, the solution adds negligible computational overhead via second-order Taylor approximation while requiring no additional memory overhead. Evaluation across multiple video models (MAGI-1, SkyReels-V2, HY-WorldPlay) demonstrates that INT2 quantization with this correction achieves near-BF16 quality, fundamentally improving the memory-quality tradeoff that previously limited video generation model deployment in resource-constrained environments.

For practitioners, this represents a pathway to scaling video diffusion without proportional hardware investment, directly addressing a current bottleneck in generative video systems.

• →Jensen bias in softmax attention causes quantization noise to disproportionately amplify cached key contributions, degrading video quality during KV-cache compression
• →A mathematical correction technique recovers near-BF16 video quality at INT2 quantization with negligible computational overhead and zero additional memory requirements
• →The method achieves equivalent quality to INT4 quantization while using 50% less memory, significantly improving efficiency for longer video generation
• →The solution is generalizable across different video diffusion architectures, validated on MAGI-1, SkyReels-V2, and HY-WorldPlay models
• →This addresses a critical bottleneck limiting practical deployment of autoregressive video generation systems at scale