From Historical Tabular Image to Knowledge Graphs: A Provenance-Aware Modular Pipeline

Researchers present a modular, provenance-aware pipeline that converts handwritten archival tables into Knowledge Graphs while maintaining transparency through intermediate inspection points. The approach combines table structure recognition, handwriting recognition, and semantic interpretation while tracking data lineage to ensure all extracted information remains traceable to its source, addressing the opacity problem in end-to-end AI systems.

This research tackles a fundamental challenge in digital humanities and archival science: transforming unstructured historical documents into machine-readable knowledge representations without sacrificing interpretability. The authors recognize that traditional end-to-end AI approaches, while potentially more efficient, create black-box systems where intermediate steps remain hidden from human oversight. Their modular pipeline decomposition into table reconstruction, information extraction, and KG construction stages directly addresses this transparency gap.

The integration of data provenance throughout the pipeline represents a significant methodological contribution. By maintaining traceable links between extracted entities and their visual origins, the system enables human verification and correction at each stage. This is particularly valuable for historical research where accuracy and source attribution are paramount. The modular design allows researchers to swap different algorithms for each stage—demonstrated through testing three table reconstruction variants—without rebuilding the entire system.

For the broader AI and knowledge management sectors, this work exemplifies how human-AI collaboration can improve outcomes beyond what either achieves alone. The approach has implications for any domain processing legacy documents: legal archives, medical records, scientific publications, and administrative histories. Organizations managing historical data can now implement pipelines that balance automation efficiency with human expertise and oversight. The demonstrated application to military career records shows practical utility in real-world scenarios where data quality directly impacts historical scholarship and institutional memory.

• →Modular pipeline design enables inspection and correction of intermediate results, improving system transparency and trustworthiness.
• →Data provenance tracking ensures all extracted information remains traceable to original visual sources, critical for archival accuracy.
• →Multi-stage decomposition allows flexible algorithm substitution and benchmarking across different technical approaches.
• →Human-AI collaboration model advances historical research by combining computational efficiency with expert human oversight.
• →Approach generalizes beyond military archives to any domain requiring legacy document digitization with maintained source attribution.