Learning to Explore with Parameter-Space Noise: A Deep Dive into Parameter-Space Noise for Reinforcement Learning with Verifiable Rewards

Researchers introduce PSN-RLVR, a new reinforcement learning method that uses parameter-space noise to improve AI exploration and reasoning capabilities. The technique addresses limitations in existing approaches by enabling better discovery of new problem-solving strategies rather than just reweighting existing solutions.

• →PSN-RLVR uses parameter perturbation to induce more effective exploration in reinforcement learning for AI reasoning tasks
• →The method addresses the exploration ceiling problem where existing approaches reweight solutions rather than discovering new strategies
• →Parameter-space noise better preserves long-horizon reasoning coherence compared to action-space noise approaches
• →The technique shows consistent improvements across multiple mathematical reasoning benchmarks and model families
• →PSN-GRPO outperforms existing exploration methods while remaining composable with other techniques for additional gains