Miner:Mining Intrinsic Mastery for Data-Efficient RL in Large Reasoning Models

Researchers introduce Miner, a novel reinforcement learning method that leverages a model's intrinsic uncertainty as a self-supervised reward signal to improve training efficiency for large reasoning models. The approach achieves state-of-the-art results on reasoning benchmarks, with performance gains up to 4.58 points in Pass@1 metrics compared to existing methods, addressing a critical inefficiency in current critic-free RL training.

Miner represents a significant advancement in making large language model training more efficient by solving a fundamental problem in reinforcement learning: wasted computational resources when models produce consistently correct outputs. Traditional critic-free RL methods generate many rollouts that yield zero advantage estimates on homogeneous positive prompts, essentially training on data that provides no learning signal. The research team's innovation repurposes the model's own uncertainty predictions as a training signal, eliminating the need for external supervision or auxiliary models.

The method builds on established trends in AI efficiency and self-supervised learning. As reasoning models grow larger, training costs become prohibitive, making data efficiency increasingly critical. Miner's approach aligns with broader industry efforts to reduce computational overhead while maintaining performance gains. The two core innovations—token-level focal credit assignment and adaptive advantage calibration—demonstrate how granular attention to model confidence levels can guide learning more effectively.

For AI researchers and organizations developing reasoning models, this approach offers immediate practical benefits. The method requires no additional inference costs or external models, making it readily deployable in existing training pipelines. Performance improvements of up to 6.66 points in Pass@K metrics suggest meaningful advantages for complex reasoning tasks like mathematical problem-solving and logic-based applications.

The publicly available code enables rapid adoption and validation across different model architectures and scales. Future work should explore how these principles apply to larger models and whether uncertainty-driven training translates to improved real-world performance on specialized reasoning domains.

• →Miner achieves up to 4.58 absolute gains in Pass@1 metrics by using model uncertainty as a self-supervised training signal
• →The method eliminates wasted rollouts from homogeneous positive prompts through token-level focal credit assignment mechanisms
• →No external supervision, auxiliary models, or additional inference costs required for implementation
• →Demonstrated superior performance across six reasoning benchmarks on Qwen3 base models compared to four competing algorithms
• →Publicly available code enables immediate adoption for improving RL training efficiency in large language models