Verifier-Backed Hard Problem Generation for Mathematical Reasoning

Researchers introduce VHG, a verifier-enhanced framework that improves how large language models generate valid and challenging mathematical problems through three-party self-play involving a setter, solver, and independent verifier. The approach addresses critical limitations in existing problem generation methods by constraining reward signals to ensure both problem validity and difficulty, demonstrating substantial improvements over baseline approaches.

The challenge of generating valid, novel, and difficult problems represents a genuine bottleneck in AI training and research automation. LLMs excel at solving existing problems but struggle when tasked with creation, often producing nonsensical or trivial problems through reward hacking in self-play scenarios. VHG addresses this by introducing a verifier role that independently validates problem soundness while a solver assesses difficulty, fundamentally changing how the problem setter receives feedback.

This work builds on growing recognition that problem generation quality directly impacts model capability advancement. Traditional approaches either require expensive human experts or rely on naive self-play where solvers and setters game each other without external constraints. The three-party architecture provides structural oversight missing from bilateral approaches, creating an adversarial-but-constrained environment.

For the AI development community, this methodology has immediate practical value. Training datasets of high-quality mathematical problems remain scarce, and automating their generation at scale could accelerate research timelines while reducing annotation costs. The dual verifier approach—combining symbolic verification with LLM-based assessment—demonstrates flexibility in implementation, allowing adaptation across different problem domains.

The technical contribution extends beyond mathematics education or training. Autonomous scientific research systems fundamentally require the ability to formulate meaningful problems within discovery spaces. By establishing verifier-backed constraints, this framework moves closer to enabling AI systems that can both solve and propose novel research directions. Future work likely involves scaling these methods to other domains and refining verifier architectures to handle increasing problem complexity.

• →VHG introduces a verifier-enhanced framework using three-party self-play to generate valid and challenging mathematical problems automatically.
• →The independent verifier role prevents reward hacking by jointly constraining the setter's rewards based on validity and difficulty assessments.
• →Dual verifier variants—symbolic and LLM-based—demonstrate the framework's flexibility across different problem types and domains.
• →Substantial improvements over baseline methods suggest practical viability for scaling automated, high-quality problem generation in AI training pipelines.
• →This approach addresses a critical bottleneck in autonomous scientific research by enabling systems to formulate meaningful problems independently.