Researchers identify and resolve a critical instability in MeanFlow training for one-step generative models by correcting how the conditional velocity field is used in loss calculations. The fix, derived in closed form, improves sample quality by up to 54% on benchmarks and produces monotonic FID improvements across diffusion transformer checkpoints, though revealing a practical FID-MSE landscape mismatch.
This paper addresses a fundamental theoretical problem in generative modeling that has practical implications for efficient model distillation. MeanFlow training, a distillation-free approach to one-step generation, has suffered from unstable training dynamics characterized by non-decreasing loss and unbounded gradient variance. The researchers identify the root cause: the conditional velocity field serves dual statistical roles in the loss function but receives incorrect weighting, leading to suboptimal variance reduction. By deriving the optimal coefficient mathematically and validating it empirically, the work provides both theoretical insight and practical improvement.
The research builds on the broader trend toward efficient generative models that reduce computational costs during inference. Diffusion models and flow-matching approaches have proven powerful but expensive; one-step methods promise dramatic speedups. However, training stability has constrained adoption. This work directly tackles that bottleneck through rigorous analysis rather than heuristics. The theoretical contribution—establishing the connection between coefficient assignment and control variate optimization—advances understanding of variance reduction in machine learning more broadly.
For practitioners developing generative models, these findings offer immediate value through concrete architectural improvements. The results on latent Diffusion Transformers, a practically relevant model class, demonstrate real-world applicability. However, the discovered FID-MSE landscape mismatch introduces nuance: the coefficient minimizing gradient variance differs from the one optimizing final sample quality, suggesting practitioners cannot blindly follow theoretical optima without considering downstream metrics. This insight prevents overconfidence in purely theoretical prescriptions and encourages empirical validation during deployment.
- →MeanFlow training instability stems from incorrect coefficient assignment to the conditional velocity field's dual statistical roles.
- →The optimal coefficient can be derived in closed form through proper variance reduction theory.
- →Fixes in concurrent works correspond to different practical implementations of the same theoretical optimum.
- →Up to 54% sample quality improvement achieved on 2D benchmarks with monotone FID trends on DiT checkpoints.
- →Gradient variance optimization and FID optimization prefer different coefficient values, requiring empirical validation over pure theory.