AIBullisharXiv – CS AI · 14h ago7/10
🧠
Moment-KV: Momentum-Based Decode-Time KV Cache Compression for Long Generation
Researchers introduce Moment-KV, a momentum-based compression technique that optimizes Key-Value cache usage during LLM decoding phases. The method improves long-generation task performance by 2.3-3.2% while maintaining latency by dynamically tracking token importance through temporal attention patterns rather than static heuristics.