Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoning

Researchers conducted a comprehensive empirical study on scaling laws for large language models during reinforcement learning post-training, using Qwen2.5 models ranging from 0.5B to 72B parameters. The study reveals that larger models demonstrate superior learning efficiency, performance can be predicted via power-law models, and data reuse proves highly effective in constrained environments, providing practical guidelines for optimizing LLM reasoning capabilities.

This empirical investigation addresses a significant gap in LLM research by systematically studying how models scale during reinforcement learning post-training, an increasingly critical phase as developers move beyond pre-training. While scaling laws for initial model training have been well-documented, the RL fine-tuning phase remains poorly understood despite its growing importance for reasoning tasks and alignment. The Qwen2.5 model series experiments establish quantifiable relationships between model size, data volume, and computational resources, enabling more efficient resource allocation during development.

The research's most actionable finding concerns data efficiency in resource-constrained scenarios. The discovery that repeated high-quality data reuse performs comparably to fresh data samples has immediate practical implications for organizations with limited training budgets. This challenges conventional assumptions about dataset diversity requirements and suggests that strategic data curation matters more than dataset size. The identified saturation trend in learning efficiency at larger scales also warns against unlimited scaling, indicating diminishing returns become apparent even with expanded models.

For the AI industry, these findings directly influence development strategy and infrastructure investment decisions. Smaller organizations can optimize RL post-training without massive data pipelines, while larger players gain clearer cost-benefit analysis for scaling decisions. The power-law predictive model enables better forecasting of training outcomes before committing computational resources. These insights shape how teams approach mathematical reasoning capabilities and similar complex tasks requiring iterative refinement through RL.

• →Larger LLMs achieve superior learning efficiency in RL post-training across both compute and data metrics
• →Test loss, compute, and data relationships follow predictable power-law patterns applicable to both base and instruction-tuned models
• →Learning efficiency shows latent saturation as model size increases, limiting benefits from unlimited scaling
• →High-quality data reuse proves highly effective in data-constrained scenarios, with total optimization steps mattering more than sample uniqueness
• →These findings provide practical guidelines for efficient scaling of LLM reasoning capabilities through RL post-training