Personalizing Embodied Multimodal Large Language Model Agents over Long-term User Interactions

Researchers introduce POLAR, a memory-augmented framework that enables multimodal AI agents to personalize their behavior based on accumulated long-term user interactions. The system organizes past interactions into semantic and episodic memory, allowing embodied agents to interpret implicit user requests and improve task execution performance across multiple interaction cycles.

POLAR addresses a fundamental limitation in current embodied AI systems: the inability to leverage personalized context accumulated over extended user interactions. While multimodal large language models have demonstrated strong capabilities in task execution, they typically operate in stateless environments where each interaction begins without memory of past exchanges. This research recognizes that real-world assistive scenarios require implicit understanding of user preferences and context that can only emerge from historical data.

The framework's innovation lies in its dual-memory architecture. Semantic memory captures user preferences and visual concepts learned over time, while episodic memory stores specific embodied experiences like agent trajectories and interaction outcomes. This distinction mirrors how human memory works, enabling more nuanced personalization than traditional fine-tuning approaches. The evaluation demonstrates that memory-augmented agents show marked improvements when tasks require reasoning across multiple prior interactions or tracking evolving user-specific context.

For the AI industry, this research has significant implications for deploying assistive robots and embodied agents in household or enterprise settings. Personalization at scale has been a barrier to adoption; users expect systems to remember preferences and adapt behavior accordingly. POLAR's approach suggests that multimodal memory systems can bridge this gap efficiently without requiring model retraining. The findings support growing investment in persistent memory architectures for AI agents, particularly as enterprises move toward long-term autonomous systems. This work indicates that memory mechanisms, rather than model scale alone, may be key to practical AI personalization.

• →POLAR enables embodied AI agents to personalize responses by organizing long-term user interactions into semantic and episodic memory structures
• →Memory-augmented agents show consistent performance improvements especially for multi-hop reasoning and tracking evolving user preferences over time
• →The framework works across multiple MLLM backends, suggesting broad applicability to different AI architectures
• →Dual-memory design separates knowledge capture from experiential learning, mirroring human memory systems for better personalization
• →Results indicate persistent memory mechanisms may be more critical than model scale for practical long-term AI assistance