Think First, Diffuse Fast: Improving Diffusion Language Model Reasoning via Autoregressive Plan Conditioning

Researchers developed plan conditioning, a training-free method that significantly improves diffusion language model reasoning by prepending short natural-language plans from autoregressive models. The technique improved performance by 11.6 percentage points on math problems and 12.8 points on coding tasks, bringing diffusion models to competitive levels with autoregressive models.

• →Plan conditioning improves diffusion language model reasoning performance by up to 12.8 percentage points on coding and 11.6 points on math problems.
• →The method works by prepending ~100-token plans from autoregressive models to help coordinate token generation across all positions simultaneously.
• →Diffusion models benefit 2-10x more from plans than autoregressive models, supporting the coordination problem hypothesis.
• →Plan-conditioned diffusion inference shows zero standard deviation across random seeds, making results highly stable.
• →The technique requires high-quality planners to work effectively, with smaller models actually hurting performance.