G-Zero: Self-Play for Open-Ended Generation from Zero Data

Researchers introduce G-Zero, a verifier-free framework that enables large language models to improve autonomously through self-play without relying on external judges or proxy models. The approach uses an intrinsic reward mechanism called Hint-δ to identify and address the Generator model's blind spots, achieving scalable self-evolution across unverifiable domains.

G-Zero addresses a fundamental limitation in current LLM self-improvement methods: their dependence on external judge models or human feedback, which creates capability bottlenecks and incentivizes reward hacking. Traditional self-play systems struggle in open-ended tasks where verification is difficult. The framework introduces two co-evolving components—a Generator model and a Proposer model—that work together without external supervision. The Proposer continuously identifies weaknesses in the Generator by creating challenging queries and helpful hints, while the Generator learns to internalize these improvements through direct preference optimization. The technical innovation centers on Hint-δ, which measures the predictive shift between unassisted responses and hint-conditioned responses, creating an intrinsic signal for improvement.

This research advances the field by demonstrating that LLMs can bootstrap their own capability improvements from internal dynamics alone. The theoretical guarantee provided—a best-iterate suboptimality bound under specified conditions—adds rigor to the approach. For AI development, this represents progress toward more autonomous and cost-effective model training, particularly valuable for domains where ground-truth verification is impossible. The removal of external judge dependencies could accelerate iterative model improvements and reduce computational overhead associated with maintaining separate judge systems.

The practical impact extends to AI companies seeking more efficient training pipelines and researchers exploring post-training optimization. By bypassing external evaluation bottlenecks, G-Zero potentially enables continuous self-improvement at scale. However, the framework's effectiveness depends on the Proposer maintaining sufficient exploration coverage and controlling noise in pseudo-labels, suggesting implementation challenges remain.

• →G-Zero enables LLM self-improvement without external judges by using co-evolutionary Proposer and Generator models with intrinsic reward signals.
• →The Hint-δ reward mechanism quantifies improvement by measuring predictive shifts between unassisted and hint-guided responses.
• →Theoretical analysis provides suboptimality guarantees under conditions of sufficient exploration and low pseudo-label noise.
• →This approach bypasses capability ceilings of external judges, enabling LLM evolution across unverifiable domains.
• →The framework reduces computational overhead and dependency on proxy models, advancing scalable autonomous AI training.