Tracing the Roots: A Multi-Agent Framework for Uncovering Data Lineage in Post-Training LLMs

Researchers introduce a multi-agent framework to map data lineage in large language models, revealing how post-training datasets evolve and interconnect. The analysis uncovers structural redundancy, benchmark contamination propagation, and proposes lineage-aware dataset construction to improve LLM training diversity and quality.

The research addresses a critical blind spot in LLM development: understanding how training datasets relate to and influence one another. While individual datasets receive scrutiny, their systemic connections remain largely unexplored. This framework changes that by automatically reconstructing the evolutionary graph of dataset development, enabling researchers to trace how data flows, transforms, and contaminates through the training pipeline.

The findings expose two major structural problems in current dataset ecosystems. First, implicit dataset intersections create redundancy that homogenizes training corpora—datasets unknowingly reuse overlapping material, weakening diversity benefits expected from multi-source training. Second, benchmark contamination (where test data leaks into training sets) propagates upstream through lineage paths, compounding its impact across dependent datasets. These issues directly affect model performance claims and generalization capabilities.

For the broader AI development community, this work establishes data lineage as an essential governance mechanism similar to provenance tracking in scientific research. Rather than treating datasets as isolated black boxes, practitioners can now systematically understand dataset ancestry and inheritance patterns. The proposed lineage-aware sampling approach demonstrates practical value by anchoring instruction selection at root sources, reducing downstream homogenization and improving corpus diversity.

The framework's efficiency as a topological alternative to sample-level comparison matters particularly for organizations managing hundreds of datasets. Looking forward, data lineage analysis will likely become standard practice in responsible AI development, enabling better benchmark integrity, reproducibility, and strategic dataset curation. This systematic approach represents a shift toward more transparent, controllable post-training data pipelines across the industry.

• →Multi-agent framework automatically reconstructs dataset evolution graphs to map how training data interconnects and evolves.
• →Analysis reveals structural redundancy and benchmark contamination propagating through dataset lineage paths, degrading model training quality.
• →Lineage-aware sampling strategy mitigates homogenization by anchoring instruction selection at upstream root sources.
• →Data lineage serves as more efficient topological alternative to sample-level comparison for large-scale dataset ecosystems.
• →Framework advances post-training data curation toward systematic, controllable paradigm with explicit lineage structures.