InterChart: Benchmarking Visual Reasoning Across Decomposed and Distributed Chart Information

Researchers introduce InterChart, a benchmark designed to evaluate how well vision-language models (VLMs) reason across multiple related charts—a capability essential for financial analysis, scientific reporting, and policy dashboards. Testing reveals that state-of-the-art VLMs struggle significantly as chart complexity increases, performing better when multi-entity charts are decomposed into simpler components, highlighting a critical gap in multimodal reasoning capabilities.

InterChart addresses a genuine limitation in how current vision-language models handle real-world visual reasoning tasks. While existing benchmarks evaluate VLMs on isolated, uniform charts, this diagnostic framework tests cross-chart integration—a fundamental requirement in financial reporting, scientific analysis, and data-driven decision-making. The benchmark's three-tier structure progresses from basic factual reasoning within single charts to complex semantic inference across visually diverse, real-world chart pairs, providing a rigorous evaluation methodology.

The research reveals a consistent pattern: VLMs experience steep accuracy declines as visual complexity increases and chart relationships become more intricate. Notably, models perform better when decomposed into simpler visual units, suggesting they struggle with architectural limitations in integrating information across multiple visual contexts. This finding has direct implications for enterprise applications relying on automated chart analysis and data interpretation.

For the AI industry, InterChart establishes a clear roadmap for improvement in multimodal reasoning systems. Financial institutions, research organizations, and government agencies increasingly depend on automated chart analysis for decision-making, making this benchmark's findings particularly relevant to developers building next-generation VLMs. The work demonstrates that model scaling alone may not solve cross-chart reasoning challenges—architectural innovations in how models process and integrate multiple visual inputs appear necessary.

Future development will likely focus on enhancing VLM architectures to better handle multi-visual contexts and establishing standardized evaluation methods for complex reasoning scenarios. Organizations building chart-analysis tools should consider these limitations when deploying current VLMs for critical applications requiring accurate multi-chart integration.

• →State-of-the-art VLMs show significant accuracy decline when reasoning across multiple related charts compared to single-chart tasks.
• →InterChart's three-tier benchmark structure progresses from basic factual reasoning to complex semantic inference across real-world chart pairs.
• →Models perform better when multi-entity charts are decomposed into simpler visual units, revealing architectural limitations in cross-chart integration.
• →The benchmark addresses a real-world gap in VLM capabilities critical for financial analysis, scientific reporting, and policy applications.
• →Results suggest that model scaling alone may be insufficient to solve multi-chart reasoning challenges without architectural innovations.