PuzzleClone: A DSL-Powered Framework for Synthesizing Verifiable Data

Researchers introduce PuzzleClone, a DSL-driven framework that automatically synthesizes large-scale, verifiable datasets for training LLMs on mathematical and logical reasoning tasks. The team generates PC-83K, a benchmark of 83,000+ diverse puzzles, and demonstrates that models fine-tuned on this dataset achieve substantial performance improvements across multiple logic and mathematical benchmarks.

PuzzleClone addresses a critical bottleneck in LLM development: the scarcity of high-quality, verifiable training data for reasoning tasks. Traditional data augmentation methods struggle with reliability and diversity, limiting their effectiveness for strengthening model capabilities. The framework's domain-specific language approach enables systematic generation of puzzle variants while maintaining logical soundness through a reproduction mechanism, solving the validation problem that plagues synthetic datasets.

The research emerges from growing recognition that LLMs require specialized training on structured reasoning problems to progress beyond pattern matching. Existing benchmarks often lack scale or diversity, and manually curated datasets cannot keep pace with model appetites. PuzzleClone represents a methodological shift toward programmatic dataset synthesis with built-in verification, addressing both quantity and quality concerns simultaneously.

For the AI development community, this framework offers tangible benefits. The PC-83K benchmark demonstrates measurable improvements: post-training SFT and reinforcement learning raise performance from 14.5% to 66.0% on the base task, with consistent gains up to 18.4 percentage points across seven external benchmarks. These results validate that synthetically generated, systematically verified data can meaningfully enhance reasoning capabilities—a finding with implications for practitioners seeking to improve model performance without human annotation bottlenecks.

Future work likely involves applying similar DSL-driven approaches to other reasoning domains beyond mathematics and logic puzzles. The open-source release enables broad adoption and potential extensions. Organizations developing reasoning-centric AI applications will benefit from understanding whether domain-specific puzzle synthesis can replicate these gains across different problem classes.

• →PuzzleClone uses domain-specific language (DSL) to systematically generate verified mathematical and logical puzzles at scale, addressing reliability and diversity limitations in synthetic datasets.
• →The PC-83K benchmark contains over 83,000 diverse puzzles with programmatic validation, enabling reproducible training data generation.
• →Fine-tuning models on PC-83K improves base task performance from 14.5% to 66.0% and delivers consistent gains up to 18.4 percentage points across seven external logic and mathematical benchmarks.
• →The reproduction mechanism ensures validity by verifying puzzle correctness before inclusion, solving a major validation challenge in LLM-generated datasets.
• →Open-source release enables broader adoption of DSL-driven synthesis approaches for reasoning-focused AI training.