Noise Scheduling as Information-Guided Allocation in Diffusion Training

Researchers introduce InfoNoise, an adaptive noise scheduling method for diffusion model training that dynamically reallocates computational resources toward the most informative denoising levels. By estimating conditional-entropy-rate profiles during training, the approach matches or exceeds fixed schedules on image benchmarks while achieving up to 3x computational efficiency gains on diverse tasks including DNA and language generation.

InfoNoise addresses a fundamental inefficiency in diffusion model training: noise schedules are typically fixed before understanding which noise levels contribute most to learning. The research identifies that different noise levels provide varying amounts of information about clean samples, and training effort should concentrate where uncertainty reduction is steepest. Rather than requiring offline search or auxiliary models, InfoNoise estimates this information profile online using denoising losses, making schedule adaptation practical and automatic.

Diffusion models have become central to generative AI across images, text, and other modalities, but their training remains computationally expensive. Schedule design has traditionally relied on manual tuning or grid search, creating a static allocation that may not match the data distribution. This work establishes conditional-entropy-rate as the principled target for noise schedule design, grounding adaptive scheduling in information theory.

The practical implications are significant for AI infrastructure and research efficiency. On well-studied image benchmarks, InfoNoise maintains competitive performance while reducing training steps. More substantially, on less-explored domains like DNA and language generation, it shows consistent improvements, suggesting the method generalizes beyond single-modality tuning. A 3x reduction in training compute directly impacts model development timelines and resource costs, particularly valuable for researchers and organizations with limited GPU budgets.

Future work likely involves integrating InfoNoise into production training pipelines and extending the approach to other generative frameworks. The theoretical foundation around entropy-rate profiles may inspire related efficiency improvements across deep learning, while the online adaptation mechanism demonstrates how modern training can leverage realtime data signals rather than predetermined schedules.

• →InfoNoise dynamically optimizes noise schedules during training by identifying and prioritizing the most informative denoising levels
• →The method achieves up to 3x training efficiency gains on sequence, representation, and multimodal tasks without auxiliary models
• →Conditional-entropy-rate profiling provides a principled, information-theoretic foundation for adaptive noise schedule design
• →Performance on image benchmarks matches established baselines while reducing computational cost, indicating broad applicability
• →Online adaptation eliminates the need for offline search and manual tuning, making diffusion model training more accessible