Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History

Researchers introduced Persona2Web, the first benchmark for evaluating personalized web agents that can infer user preferences from historical behavior rather than explicit instructions. The framework tests how large language models handle ambiguous queries by leveraging user context, addressing a critical gap in current web agent capabilities.

Persona2Web represents a meaningful advancement in evaluating autonomous web agents built on large language models. Current web agents struggle with ambiguity because users rarely articulate every detail of their intent, forcing systems to make assumptions about context and preferences. This benchmark tackles that problem by introducing a framework where agents must resolve unclear queries using implicit information from user histories rather than relying on explicit, detailed instructions.

The research builds on the broader trend of improving AI agent autonomy and contextual reasoning. As LLMs have become more sophisticated, the focus has shifted from basic task completion to nuanced understanding of user needs. Web agents increasingly handle real-world tasks—booking travel, shopping, scheduling—where personalization directly impacts utility. Without benchmarks to measure personalization quality, developers lack clear performance metrics for this capability.

For the AI development community, Persona2Web offers a practical evaluation framework that examines multiple dimensions: how agents access user history, interpret ambiguity levels, and reason about preferences. The benchmark's emphasis on "reasoning-aware" assessment means it doesn't just measure whether agents get the right answer, but whether they demonstrate sound inference logic based on user context.

Looking ahead, this benchmark will likely influence how developers design web agents and language models. As personalization becomes a competitive differentiator in AI assistants, similar reasoning-based evaluation frameworks may become standard. The public availability of datasets and code creates opportunities for rapid iteration across the research community, potentially accelerating progress in contextual reasoning capabilities that extend beyond web agents to other autonomous systems.

• →Persona2Web is the first benchmark specifically designed to evaluate how web agents infer user preferences from historical behavior patterns.
• →The framework addresses a critical limitation in current agents: their inability to resolve ambiguous queries without explicit instructions.
• →Testing reveals significant challenges across different agent architectures and backbone models when handling varying levels of query ambiguity.
• →The benchmark's reasoning-aware evaluation goes beyond accuracy metrics to assess the quality of inference logic underlying personalization decisions.
• →Public availability of datasets and code positions Persona2Web as a potential standard for evaluating personalized AI agent capabilities.