CLPO: Curriculum Learning meets Policy Optimization for LLM Reasoning

Researchers introduce CLPO, a curriculum learning framework that dynamically adapts training difficulty for large language models during reinforcement learning. The approach automatically identifies solved, medium, and hard problems, then strategically restructures tasks to match the model's evolving capabilities, achieving substantial improvements over existing methods on mathematical and reasoning benchmarks.

CLPO represents a meaningful advancement in how large language models develop reasoning capabilities through reinforcement learning. Traditional approaches waste computational resources by continuing to train on problems the model has already mastered or attempting problems too difficult to learn from effectively. This research addresses that inefficiency by creating a self-adjusting curriculum that evolves alongside the model's improving abilities.

The method builds on the established paradigm of online reinforcement learning with verifiable rewards, which has proven effective for enhancing LLM reasoning. However, previous approaches treat problem sets as static throughout training. CLPO introduces intelligent problem restructuring—simplifying hard problems to make them learnable and diversifying medium-difficulty problems to maximize training signal. Critically, this restructuring itself becomes optimized through rollout data, eliminating the need for additional human annotation.

The empirical results demonstrate substantial gains, with CLPO outperforming comparable methods (GRPO and DAPO) by 10+ percentage points on mathematical reasoning tasks with Qwen-3-8B. The improvements generalize beyond mathematics to out-of-domain reasoning benchmarks, suggesting the approach captures fundamental principles about effective training. Ablation studies validate that both the restructuring strategy and the associated loss function contribute meaningfully to final performance.

For the AI development community, CLPO offers a scalable pathway to improve reasoning without proportional increases in computational cost. The framework's ability to co-evolve with model capabilities suggests potential applications across various domains requiring complex reasoning. However, the research remains primarily academic, with real-world deployment implications still requiring investigation.

• →CLPO automatically adapts training difficulty by identifying and restructuring problems based on model performance, improving efficiency over static problem sets
• →The framework demonstrates 10.21 point improvement over GRPO and 7.75 point improvement over DAPO on mathematical reasoning benchmarks
• →Problem restructuring is optimized through reinforcement learning without requiring additional human annotations beyond original verifiable answers
• →The approach generalizes effectively across mathematical and out-of-domain reasoning tasks, suggesting broad applicability
• →Both problem restructuring modes and associated training loss contribute meaningfully to performance gains according to ablation studies