Provenance-Grounded Gating and Adaptive Recovery in Synthetic Post-Training Data Curation

Researchers present a controlled study on synthetic data curation for post-training large language models, examining whether filtering decisions are grounded in source evidence and whether rejected samples can be recovered. Their findings show that provenance-aware filtering improves faithfulness detection, different gate types catch different errors, and adaptive recovery strategies significantly improve overall yield compared to simple resampling.

This research addresses a critical inefficiency in modern LLM training pipelines. Current synthetic data curation relies heavily on filtering mechanisms that discard potentially recoverable samples, representing wasted computational resources in an era where training costs directly impact model economics. The study's provenance-grounding approach—linking filtering decisions back to source evidence—represents a methodological advancement that could ripple through the industry as teams scale synthetic training data.

The broader context involves the AI industry's growing dependence on synthetic data for post-training phases. As human-annotated data becomes scarcer and more expensive, synthetic generation has become essential, but the quality-quantity tradeoff remains unresolved. This work bridges that gap by demonstrating that rejected samples aren't simply low-quality noise; they contain recoverable signal that targeted regeneration can extract.

For practitioners building LLM infrastructure, the implications are concrete: the paper shows that combining hallucination detection with reward model signals captures complementary failure modes, meaning neither gate alone suffices. The adaptive recovery pipeline that uses failure diagnosis to guide regeneration achieves meaningfully higher injection recall—the ability to catch deliberately injected errors—than baseline approaches. This translates directly to more robust models from equivalent computational budgets.

Looking ahead, the research opens questions about optimal gate architecture combinations and whether these findings generalize across different generator scales and domains. The emphasis on downstream fine-tuning quality being primarily driven by generator scale suggests diminishing returns from filtration alone, pointing toward future work optimizing the pipeline's full depth rather than isolated components.

• →Provenance-grounded filtering significantly improves faithfulness detection in synthetic data curation across stronger judges.
• →Hallucination gates and reward model gates reject largely different sample populations, making both necessary for comprehensive quality control.
• →Adaptive recovery pipelines using failure diagnosis outperform naive resampling in yield, recovery rate, and error detection.
• →Generator scale remains the primary driver of downstream fine-tuning quality, with filtration contributing meaningfully but secondarily.
• →Systematic recovery of rejected samples represents untapped efficiency gains in current synthetic post-training pipelines.