AIBullisharXiv – CS AI · 14h ago7/10
🧠
MEMENTO: Teaching LLMs to Manage Their Own Context
Researchers introduce MEMENTO, a method enabling large language models to compress their reasoning into dense summaries (mementos) organized into blocks, reducing KV cache usage by 2.5x and improving throughput by 1.75x while maintaining accuracy. The technique is validated across multiple model families using OpenMementos, a new dataset of 228K annotated reasoning traces.