ANCORA: Learning to Question via Manifold-Anchored Self-Play for Verifiable Reasoning

Researchers introduce ANCORA, a self-play framework enabling language models to generate verifiable problems, solve them, and improve without human supervision. The method achieves 81.5% pass rate on Dafny2Verus tasks, significantly outperforming baseline approaches and demonstrating advances in autonomous AI reasoning capabilities.

ANCORA represents a fundamental shift in how AI systems approach problem-solving by reversing the traditional learning pipeline. Rather than training models primarily to answer questions, the framework teaches models to generate meaningful problems and verify their own solutions. This self-supervised approach operates through a unified policy with two alternating roles: a Proposer creating novel specifications and a Solver producing verified solutions. The technical innovation addresses a critical bottleneck in AI development—the scarcity of high-quality training data and human-labeled examples.

The framework's effectiveness stems from three stabilizing mechanisms designed to prevent common failure modes in reinforcement learning. The two-level group-relative update couples advantages across both problem proposals and solution attempts, preventing the model from collapsing into generating trivial or redundant specifications. Self-distilled supervised fine-tuning anchors the model to valid outputs before reinforcement learning begins, establishing a foundation of correctness. The UCB-guided Curriculum DAG ensures that only novel, verified specifications enter the training pipeline, maintaining quality standards.

Results demonstrate substantial practical impact, with ANCORA lifting performance from 26.6% to 81.5% on formal verification tasks—outperforming comparable self-play methods by 15.8 percentage points despite using zero-shot evaluation versus one-shot baselines. Transfer learning experiments show the approach generalizes beyond its training domain, achieving competitive results on standard programming benchmarks.

This research carries implications for autonomous AI system development, particularly in domains requiring formal verification and rigorous problem-solving. As AI systems become more autonomous, methods enabling self-improvement without constant human supervision become increasingly valuable. Future work may explore whether these principles extend to other verification domains or multimodal learning scenarios.

• →ANCORA achieves 81.5% pass rate on Dafny2Verus formal verification tasks, a 3x improvement over SFT baseline.
• →Self-play framework enables language models to generate, solve, and verify problems autonomously without human supervision.
• →Novel stabilization mechanisms prevent common RL failures like proposer collapse through coupled advantage updates and curriculum filtering.
• →Transfer learning capabilities demonstrate generalization beyond training domain to standard programming benchmarks.
• →Approach outperforms comparable PSV self-play baseline by 15.8 points despite stricter evaluation conditions.