Train at Moving Edge: Online-Verified Prompt Selection for Efficient RL Training of Large Reasoning Model

Researchers propose HIVE, a new framework for training large language models more efficiently in reinforcement learning by selecting high-utility prompts before rollout. The method uses historical reward data and prompt entropy to identify the 'learning edge' where models learn most effectively, significantly reducing computational overhead without performance loss.

• →HIVE framework reduces computational costs in RL training of large language models by selecting high-utility prompts before expensive rollout phases.
• →The research identifies that learning signals concentrate at the 'learning edge' - the intersection of intermediate difficulty and high uncertainty.
• →The method uses historical reward trajectories for coarse selection and prompt entropy as a real-time proxy for utility.
• →HIVE maintains performance while achieving significant rollout efficiency improvements across multiple math reasoning benchmarks.
• →The approach addresses the critical issue that many prompts in current RL algorithms provide negligible gradients and waste computational resources.