Dynamic Infilling Anchors for Format-Constrained Generation in Diffusion Large Language Models

Researchers introduce Dynamic Infilling Anchors (DIA), a training-free method that improves how diffusion large language models generate structured outputs like JSON or reasoning templates. By dynamically adjusting generation length constraints, DIA achieves better format compliance and accuracy on mathematical reasoning benchmarks without requiring model retraining.

The paper addresses a fundamental challenge in constrained generation for diffusion language models: balancing structural correctness with semantic flexibility. Traditional fixed-anchor approaches enforce format constraints through rigid span limitations, but this rigidity often results in truncated reasoning or redundant content that undermines output quality. DIA solves this by dynamically estimating where formatted sections should end before the iterative infilling process begins, allowing the model to allocate computational resources more efficiently while maintaining structural integrity.

Diffusion language models represent an emerging alternative to autoregressive architectures, offering bidirectional context awareness and parallel token generation capabilities. These properties make them naturally suited for constrained tasks where output structure must be guaranteed, yet existing methods haven't fully capitalized on these advantages. The development of more sophisticated constraint-handling mechanisms directly addresses this gap in the field.

The experimental validation on reasoning benchmarks like GSM8K and MATH demonstrates practical utility beyond theoretical interest. Zero-shot improvements in both format compliance and answer accuracy suggest that DIA's dynamic approach better aligns model behavior with task requirements compared to static constraint methods. For practitioners building AI systems requiring parseable outputs—from data extraction pipelines to structured reasoning applications—this represents meaningful progress toward more reliable automation.

Looking forward, the training-free nature of DIA makes it immediately deployable to existing diffusion language model implementations. Future work may explore how dynamic anchor adjustment integrates with other constraint-satisfaction techniques and whether similar approaches could enhance other conditional generation tasks.

• →Dynamic Infilling Anchors adjust generation length constraints dynamically rather than using fixed spans, improving both format compliance and semantic coherence.
• →DIA requires no model retraining, enabling immediate deployment to existing diffusion language model systems.
• →Experimental results show significant zero-shot improvements on mathematical reasoning benchmarks GSM8K and MATH.
• →The method leverages bidirectional attention and parallel generation capabilities unique to diffusion language models.
• →This addresses the practical need for reliable structured output generation in production AI applications.