TSRBench: A Comprehensive Multi-task Multi-modal Time Series Reasoning Benchmark for Generalist Models

TSRBench introduces a comprehensive benchmark with 4,125 problems across 14 domains to evaluate how well AI models perform at time series reasoning tasks. Testing 30+ leading models reveals that current LLMs and multimodal models struggle with numerical forecasting despite strong semantic understanding, and fail to effectively combine textual and visual data inputs.

TSRBench addresses a critical gap in AI model evaluation by systematizing how generalist models handle time series data, a capability essential for real-world applications from energy management to financial forecasting. The benchmark's scale—spanning perception, reasoning, prediction, and decision-making across 14 domains—provides a rigorous testing ground that existing benchmarks overlooked. This matters because time series reasoning directly impacts operational efficiency in critical infrastructure and financial systems, yet current evaluation frameworks largely ignore this dimension.

The research reveals fundamental limitations in today's most advanced models. Scaling laws that typically improve performance in reasoning tasks break down entirely for prediction, suggesting numerical accuracy requires different architectural approaches than semantic understanding. This decoupling indicates that strong language understanding alone cannot translate to reliable forecasting, challenging assumptions about model generalization. The finding that multimodal models fail to leverage complementary textual and visual representations of time series data highlights inefficiencies in current fusion techniques.

For developers and organizations deploying AI systems, these results suggest significant reliability concerns when applying cutting-edge models to time series applications. Companies building forecasting systems cannot assume that reasoning capability transfers to numerical accuracy. The benchmark establishes a standardized evaluation platform that enables targeted improvements, particularly in prediction layers and multimodal integration. As TSRBench becomes widely adopted, it will likely drive model development toward specialized time series reasoning architectures and better fusion mechanisms for complementary data modalities.

• →TSRBench's 4,125 problems across 14 domains reveal that current AI models struggle with time series reasoning despite strong semantic capabilities.
• →Scaling laws hold for perception and reasoning but break down for prediction tasks, indicating fundamental architectural mismatches.
• →Strong language understanding does not guarantee accurate numerical forecasting, exposing a critical gap in generalist model capabilities.
• →Multimodal models fail to effectively combine textual and visual time series representations for performance improvements.
• →The benchmark provides standardized evaluation methodology that will drive targeted improvements in time series AI development.