Improving Classifier-Free Guidance in Masked Diffusion: Low-Dim Theoretical Insights with High-Dim Impact

Researchers have developed an improved Classifier-Free Guidance mechanism for masked diffusion models that addresses quality degradation issues in AI generation. The study reveals that high guidance early in sampling harms quality while late-stage guidance improves it, leading to a simple one-line code fix that enhances conditional image and text generation.

• →High guidance early in diffusion sampling when inputs are heavily masked reduces generation quality, while late-stage guidance improves it.
• →Current CFG implementations can cause imbalanced transitions that degrade sample quality through rapid unmasking in early generation stages.
• →The proposed method smooths transport between data distribution and initial masked distribution for better results.
• →The improvement requires only a simple one-line code change to existing implementations.
• →Experiments confirm the method's effectiveness for both conditional image and text generation tasks.