Latent Personal Memory: Represent personal memory as dynamic soft prompts

Researchers introduce Latent Personal Memory (LPM), a framework that personalizes large language models by encoding user-specific behavioral patterns as compact, interpretable latent slots converted into dynamic soft prompts. The approach achieves significant efficiency gains—outperforming LoRA and Prompt Tuning by up to 54.4% on benchmarks while reducing memory usage by 64x—making personalized LLMs more practical for deployment.

Latent Personal Memory addresses a critical challenge in AI: how to personalize frozen large language models without massive computational overhead or model retraining. Traditional approaches like LoRA require modifying model weights, while prompt tuning demands extensive context storage. LPM solves this through a novel architecture using N latent slots—a persistent, interpretable matrix representing user history—that a cross-attention network converts into input-conditioned soft prompts prepended to the frozen model.

This work emerges from the broader trend toward parameter-efficient fine-tuning (PEFT) methods, which have gained prominence as LLMs grow larger and deployment constraints tighten. LPM's innovation lies in combining interpretability with dramatic efficiency improvements. On PersonaMem v1 benchmarks, it achieves up to 8.8% accuracy gains over LoRA and 54.4% over Prompt Tuning. More strikingly, it reduces KV-cache usage by over 64x—critical for production systems managing long user histories.

For developers and organizations deploying personalized AI assistants, LPM offers a practical path to scaling personalization without prohibitive memory costs. The framework demonstrates particular advantages at extended context lengths (128K tokens), where memory efficiency becomes increasingly valuable. With 120x fewer trainable parameters than LoRA while matching accuracy, LPM reduces infrastructure requirements and computational costs significantly.

The interpretability aspect—understanding what latent slots represent about user behavior—opens possibilities for transparency and debugging in personalized AI systems. Future work likely explores scaling these latent slots across larger models and investigating what semantic patterns emerge in slot representations.

• →Latent Personal Memory reduces KV-cache usage by 64x compared to baseline approaches while improving accuracy by up to 54.4%.
• →LPM requires 120x fewer trainable parameters than LoRA while maintaining competitive accuracy on personalization benchmarks.
• →The framework uses interpretable latent slots converted to dynamic soft prompts, enabling personalization of frozen LLMs without weight modification.
• →Performance advantages grow with context length, making LPM particularly effective for handling long user histories at 128K tokens.
• →The approach is model-agnostic, demonstrated across Qwen backbones of varying sizes from 1.7B to 8B parameters.