Stable Adaptive Thinking via Advantage Shaping and Length-Aware Gradient Regulation

Researchers developed a two-stage framework to optimize large reasoning models, reducing overthinking on simple queries while maintaining accuracy on complex problems. The approach achieved up to 3.7 accuracy point improvements while reducing token generation by over 40% through hybrid fine-tuning and adaptive reinforcement learning techniques.

• →New framework addresses overthinking behavior in large reasoning models that waste computational resources on simple queries.
• →Two-stage approach combines Hybrid Fine-Tuning with adaptive reinforcement learning using novel CPAS and LAGR techniques.
• →Testing on Qwen2.5-1.5B and 7B models showed consistent accuracy improvements up to 3.7 points with 40%+ token reduction.
• →Method demonstrates robust performance across varying problem difficulties and out-of-distribution tasks.
• →Research addresses critical efficiency challenges in deploying large reasoning models at scale.