Trip+: Benchmarking Agents in Personalized Interactive Travel Planning

Researchers introduce Trip+, a new benchmark for evaluating AI agents in travel planning that measures holistic performance across personalization, feasibility, and interaction quality. Testing 18 language models reveals a consistent gap where agents generate technically viable but exhausting itineraries that poorly match traveler preferences, highlighting limitations in how current LLMs handle complex, profile-conditioned decision-making over multiple turns.

Trip+ addresses a critical blind spot in LLM evaluation frameworks by moving beyond isolated testing of feasibility or personalization to measure end-to-end user experience in travel planning scenarios. This matters because interactive applications increasingly rely on agents to handle evolving preferences and real-world disruptions, yet existing benchmarks fail to capture whether solutions actually satisfy users holistically. The research reveals that current models optimize for technical correctness—meeting all constraints and avoiding logical conflicts—while overlooking subjective factors like traveler fatigue and preference alignment.

The benchmark reflects broader trends in AI evaluation where real-world performance diverges significantly from benchmark scores. As language models expand into personal assistant and planning roles, the gap between what models can accomplish technically and what users actually need has become increasingly problematic. Trip+ uses an LLM-based simulator to assess metrics beyond traditional binary correctness, introducing subjective quality assessment that mirrors how humans evaluate itineraries.

For AI developers and enterprises deploying agents in customer-facing applications, these findings carry significant implications. The consistent performance gap across 18 models suggests this isn't a capability issue unique to smaller models but rather a fundamental misalignment in how agents prioritize constraints. Teams building travel, logistics, or personal planning systems need to implement additional evaluation layers focused on user satisfaction metrics rather than relying solely on task completion. The research emphasizes that future model improvements must address preference-aware reasoning and fatigue modeling alongside traditional planning competencies.

• →Trip+ benchmark reveals that 18 tested language models consistently generate technically feasible but exhausting travel itineraries misaligned with user preferences.
• →Current AI evaluation frameworks inadequately measure subjective user experience metrics like fatigue and satisfaction in complex planning tasks.
• →Models optimize for constraint satisfaction rather than holistic traveler well-being, indicating a fundamental misalignment in agent objective design.
• →LLM-based simulation enables assessment of end-to-end experiences, providing a template for more realistic benchmarking of interactive applications.
• →Interactive AI systems need additional evaluation layers beyond task completion to ensure real-world deployment success in customer-facing roles.