Data Mixing Agent: Learning to Re-weight Domains for Continual Pre-training

Researchers present Data Mixing Agent, an AI framework that uses reinforcement learning to automatically optimize how large language models balance training data from source and target domains during continual pre-training. The approach outperforms manual reweighting strategies while generalizing across different models, domains, and fields without requiring retraining.

Data Mixing Agent addresses a fundamental challenge in large language model development: how to improve performance on specialized tasks without degrading existing capabilities. This problem, known as catastrophic forgetting, has historically required manual tuning by experts who adjust data mixtures based on intuition and trial-and-error. The research team demonstrates that these manual heuristics can be replaced with a learned, generalizable system trained through reinforcement learning on thousands of mixing scenarios.

The breakthrough centers on automating decisions that previously demanded significant human expertise. Rather than relying on fixed rules, the agent learns patterns across diverse continual pre-training scenarios, enabling it to make optimal reweighting decisions in new contexts. Experiments on math reasoning tasks show measurable improvements over baseline approaches, while the system successfully transfers knowledge to unseen model architectures, source fields, and domains—a critical advantage for practical deployment.

For the AI research and development community, this work reduces engineering overhead and democratizes continual pre-training optimization. Organizations developing specialized language models no longer require domain experts to manually craft data mixing strategies; the agent learns these strategies automatically. The demonstrated efficiency gains—achieving superior performance with less source-domain data—directly reduce training costs and environmental impact.

Looking forward, the applicability to code generation and other specialized domains suggests broader adoption potential. The research points toward fully automated model adaptation pipelines where data curation and optimization happen without human intervention, enabling rapid deployment of capable systems across new industries and use cases.

• →Data Mixing Agent uses reinforcement learning to automatically optimize domain reweighting, replacing manual heuristic-based approaches in continual pre-training.
• →The framework generalizes across unseen models, source fields, and domains without requiring retraining, significantly improving deployment flexibility.
• →Experiments demonstrate both superior balanced performance and training efficiency, achieving better results with less source-domain data.
• →The learned heuristics align with human intuition while reducing engineering overhead for organizations developing specialized language models.
• →Successful application across math reasoning and code generation indicates broad adaptability across different AI specialization domains.