TABVERSE: Benchmarking Cross-Format Table Understanding in LLMs and VLMs

Researchers introduced TABVERSE, a new benchmark for evaluating how Large Language Models and Vision-Language Models understand tables across different formats (HTML, Markdown, LaTeX, and images). The study reveals that table representation significantly impacts model performance, with structured text formats generally outperforming rendered images, though performance varies by task and model type.

TABVERSE addresses a critical gap in AI evaluation methodology by isolating how different table representations affect model comprehension. Previous benchmarks conflated multiple variables—content, format, layout, and modality—making it impossible to determine which factors drove performance differences. This controlled approach enables researchers to measure representation effects independently, providing clearer insights into model capabilities and limitations.

The research builds on growing recognition that input format substantially influences LLM and VLM performance. As these models become central to enterprise applications requiring document understanding, financial analysis, and data extraction, understanding format sensitivity becomes operationally important. The findings that HTML consistently outperforms other text formats while image-based tables lag significantly suggests that models struggle with visual table parsing despite advances in multimodal architectures.

For AI developers and organizations deploying table-understanding systems, these results carry practical implications. The performance gaps between formats mean that preprocessing strategies and format selection directly impact accuracy and reliability. Teams cannot assume models handle all table representations equally. Additionally, the identified challenges with row-sensitive structural tasks and LaTeX reconstruction suggest specific areas where current architectures need improvement.

The benchmark itself becomes valuable infrastructure for the AI community. As LLMs and VLMs evolve, TABVERSE enables systematic tracking of progress across representation types, preventing regressions and guiding architecture design. Future work likely focuses on closing format-based performance gaps and developing models that maintain consistent accuracy regardless of input representation, which would enhance deployment flexibility in diverse document ecosystems.

• →Table representation format significantly affects LLM and VLM performance, with structured text consistently outperforming rendered images.
• →HTML emerges as the most robust text format for table understanding across tested models.
• →Current models struggle with row-sensitive structural tasks and syntactically complex LaTeX format reconstruction.
• →TABVERSE's controlled benchmark design isolates representation effects while holding table content constant, improving evaluation rigor.
• →Format-aware preprocessing strategies are essential for reliable table understanding in production AI systems.