TRACE: Tourism Recommendation with Accountable Citation Evidence

Researchers introduce TRACE, a benchmark dataset for evaluating tourism recommendation systems that combine multi-turn dialogue, verifiable review citations, and rejection recovery. The dataset reveals a significant gap in existing conversational recommender systems: LLMs excel at recall but cite weakly, while retrieval-based systems ground better but struggle with accuracy and adaptation.

TRACE addresses a critical evaluation gap in conversational recommendation systems for high-stakes domains like tourism. Unlike generic recommendation benchmarks that measure performance through single metrics like Recall@k, TRACE establishes three simultaneous competencies: accuracy in POI selection, verifiable grounding through review citations, and adaptive recovery when recommendations are rejected. This multi-dimensional approach reflects real-world requirements where incorrect suggestions carry tangible costs—wasted money and travel time.

The benchmark's construction using 10,000 dialogues across Yelp data demonstrates the practical challenge: existing systems fail to balance competing strengths. Large language models show strong recall and recovery capabilities but produce sparse citations, creating a trust deficit. Traditional retrieval systems achieve verbatim grounding but struggle with overall accuracy. This Three-Competency Gap reveals that current architectures optimize for isolated objectives rather than integrated performance.

For AI researchers and practitioners, TRACE provides both diagnostic insight and evaluation methodology. The Grounding Score metric achieving 0.80 Spearman correlation with human judgment validates the benchmark's reliability. This enables future systems to measure citation quality objectively, not just recommendation accuracy. The recovery-focused evaluation is particularly valuable—capturing mid-dialogue rejection handling separates systems that admit uncertainty from those that persist with unreliable suggestions.

Looking forward, TRACE likely influences how conversational AI systems in commerce, travel, and other high-stakes domains are evaluated. The framework suggests that trustworthiness increasingly becomes measurable and comparable. Research teams will likely adopt similar multi-competency benchmarking approaches for domains where user action carries real consequences.

• →TRACE benchmark reveals conversational recommendation systems cannot simultaneously achieve high accuracy, dense citations, and rejection recovery
• →LLM zero-shot approaches dominate on recall but cite sparsely, while retrieval systems cite accurately but with lower accuracy overall
• →The Grounding Score metric correlates 0.80 with human judgment, enabling objective measurement of citation quality beyond traditional recommendation metrics
• →High-stakes domains like tourism require multi-competency evaluation, not single-axis leaderboards focusing only on recommendation accuracy
• →Multi-turn dialogue recovery capabilities distinguish adaptive systems from those that repeat failed suggestions when recommendations are rejected