AIBullisharXiv – CS AI · 14h ago7/10
🧠
Quantization Dominates Rank Reduction for KV-Cache Compression
A new study demonstrates that quantization significantly outperforms rank reduction for compressing KV caches in transformer inference, achieving 4-364 PPL improvements across multiple models. The research shows that preserving all dimensions while reducing precision is structurally superior to discarding dimensions, with INT4 quantization matching FP16 accuracy while enabling 75% total KV reduction.