AMV-L: Lifecycle-Managed Agent Memory for Tail-Latency Control in Long-Running LLM Systems

Researchers introduce AMV-L, a new memory management framework for long-running LLM systems that uses utility-based lifecycle management instead of traditional time-based retention. The system improves throughput by 3.1x and reduces latency by up to 4.7x while maintaining retrieval quality by controlling memory working-set size rather than just retention time.

• →AMV-L framework treats agent memory as a managed resource using utility scores rather than age-based TTL retention.
• →The system improves throughput by 3.1x and reduces median latency by 4.2x compared to traditional TTL approaches.
• →Extreme tail latency improvements include 4.4x reduction at p99 and reduction of >2s requests from 13.8% to 0.007%.
• →Performance gains come from bounding retrieval-set size and vector-search work rather than shortening prompts.
• →The framework demonstrates that predictable LLM performance requires explicit control of memory working-set size.