TimeSage-MT: A Multi-Turn Benchmark for Evaluating Agentic Time Series Reasoning

Researchers introduced TimeSage-MT, a multi-turn benchmark with 240 tasks designed to evaluate how well LLM agents handle time series analysis across extended conversations. The benchmark reveals significant performance gaps in current AI systems, particularly in decision-making, memory retention, and uncertainty handling across real-world domains.

TimeSage-MT addresses a critical gap in AI evaluation methodology by moving beyond single-task time series benchmarks to assess how language models perform in realistic, multi-turn analytical workflows. Traditional benchmarks focus on isolated forecasting or anomaly detection tasks, but real-world applications require agents to maintain context, refine hypotheses iteratively, and synthesize evidence into actionable insights. This benchmark's design—converting 240 real-world time series into 2,680 dialogue turns across eight domains—creates a more rigorous testing environment that mirrors actual user interactions with data analysis tools.

The benchmark's findings expose structural weaknesses in frontier LLMs that have immediate implications for AI development and deployment. Sharp performance degradation on decision-oriented tasks suggests current agents struggle with practical reasoning under uncertainty, a critical requirement for domains like finance, healthcare, and supply chain management where time series data informs consequential decisions. The study identifies three core failure modes: memory degradation across turns, poor uncertainty quantification, and weak domain-specific decision logic. These limitations indicate that scaling language models alone is insufficient for reliable agentic AI systems.

For the AI industry, TimeSage-MT establishes both a diagnostic tool and development roadmap. The public leaderboard incentivizes research into architectural improvements like enhanced memory mechanisms, uncertainty-aware reasoning, and domain knowledge integration. For enterprises evaluating AI agents for analytics workflows, the benchmark provides concrete performance metrics that expose whether commercial systems can handle multi-step reasoning reliably. The results suggest that production-grade time series agents require specialized skill libraries and structured reasoning frameworks beyond base language model capabilities, potentially driving demand for purpose-built AI solutions.

• →TimeSage-MT's 240-task, multi-turn benchmark reveals frontier LLMs fail significantly on decision-oriented time series analysis tasks
• →Current AI agents struggle with memory retention, uncertainty handling, and domain-specific reasoning across extended conversations
• →The benchmark converts real-world data into reproducible dialogue sequences, enabling rigorous comparison of agentic systems
• →Sharp performance gaps on decision tasks suggest scaling language models alone is insufficient for reliable analytical agents
• →Public leaderboard and benchmark infrastructure will drive development of specialized architectures for time series reasoning