MPDocBench-Parse: Benchmarking Practical Multi-page Document Parsing

Researchers introduce MPDocBench-Parse, a new benchmark dataset for evaluating multi-page document parsing systems across realistic, complex scenarios. The benchmark comprises 433 manually annotated documents spanning 3,246 pages in 15 document types, revealing that existing AI models excel at basic text extraction but struggle with semantic continuity, visual content preservation, and hierarchical structure recovery.

Document parsing represents a critical challenge in enterprise information systems, where converting visually complex PDFs and scanned documents into machine-readable formats directly impacts workflow efficiency and data accessibility. MPDocBench-Parse addresses a significant gap in AI evaluation infrastructure by moving beyond simplified, single-page benchmarks that dominated the field. The dataset's scale—433 documents across 3,246 pages in multiple languages and layout styles—mirrors real-world complexity that earlier benchmarks overlooked.

The benchmark's design reflects practical bottlenecks in document understanding pipelines. Traditional parsing systems handle straightforward text extraction competently, but fail when encountering truncated tables spanning multiple pages, visual elements requiring preservation, or document hierarchies demanding coherent reading order recovery. By establishing standardized evaluation protocols for these specific challenges, MPDocBench-Parse creates accountability for vendors developing document AI solutions.

For enterprises relying on automated document processing—financial services, legal tech, healthcare systems—this benchmark signals that current production systems likely have significant blind spots. Organizations cannot confidently deploy document parsing solutions without understanding performance across hierarchical structures and multi-page continuity. The research identifies market demand for more sophisticated document understanding capabilities, potentially accelerating development cycles for competitive solutions.

The study's findings suggest machine learning practitioners must prioritize architectural improvements specifically targeting semantic coherence across page boundaries and visual-semantic integration. Future model development will increasingly focus on these demonstrated weaknesses rather than optimizing single-page text extraction performance.

• →MPDocBench-Parse introduces the first large-scale benchmark specifically designed for evaluating multi-page document parsing in realistic enterprise scenarios.
• →Existing AI models successfully extract basic text but significantly underperform on semantic continuity, visual content preservation, and hierarchical structure recovery.
• →The benchmark covers 15 document types across 3,246 pages in English and Chinese, providing standardized evaluation protocols for text, tables, formulas, and figure extraction.
• →Current document parsing systems lack comprehensive evaluation frameworks for practical challenges like truncated text merging and reading order recovery across multiple pages.
• →The research identifies architectural gaps in production document AI systems, creating market opportunity for vendors developing more sophisticated multi-page understanding capabilities.