Process-Verified Reinforcement Learning for Theorem Proving via Lean

Researchers demonstrate that the Lean proof assistant can provide fine-grained, process-level feedback during reinforcement learning training for theorem proving, beyond simple binary verification signals. By parsing proof attempts into tactic sequences and leveraging Lean's elaboration system, the approach delivers dense, verified credit signals grounded in type theory, showing improvements over outcome-only baselines on benchmarks like MiniF2F and ProofNet.

This research addresses a fundamental limitation in applying reinforcement learning to formal mathematics: the gap between rich, structured process feedback available in symbolic systems and the sparse binary signals typically used in RL training. By treating Lean itself as a process-level reward oracle rather than merely an evaluation-time verifier, the work bridges symbolic reasoning and machine learning in a principled way.

The approach builds on recent advances in process-based reward models and RLVR frameworks, but applies them to formal mathematics where verification is mathematically rigorous rather than heuristic. The key innovation involves extracting tactic-level feedback from Lean's elaboration system, identifying both locally sound proof steps and earliest failing points. This creates dense credit signals that guide models toward valid reasoning paths.

For the broader AI and formal reasoning community, this signals an important paradigm shift: symbolic proof assistants become active participants in training rather than passive validators. This integration could accelerate development of AI systems capable of advanced mathematical reasoning by combining language model scalability with symbolic verification guarantees. The improved performance on established benchmarks demonstrates practical benefits beyond theoretical elegance.

Looking ahead, the most significant question involves scalability to harder problems and larger proof spaces. If tactic-level feedback proves consistently beneficial as problem complexity increases, this could establish a template for formal reasoning systems where symbolic grounding continuously improves neural model quality. The work also hints at future applications beyond theorem proving, anywhere symbolic systems can provide interpretable process feedback.

• →Lean proof assistant provides tactic-level verified feedback signals during training, enabling process-based reinforcement learning for theorem proving.
• →Fine-grained process supervision outperforms outcome-only baselines on mathematical reasoning benchmarks including MiniF2F and ProofNet.
• →The approach combines language model scalability with symbolic verification guarantees, advancing formal reasoning system development.
• →Symbolic proof assistants function as process-level reward oracles, not just evaluation-time verifiers, establishing a new training paradigm.
• →Dense, type-theory-grounded credit signals improve model guidance toward valid mathematical reasoning paths.