Finite Difference Flow Optimization for RL Post-Training of Text-to-Image Models

Researchers propose a new online reinforcement learning method for improving text-to-image diffusion models that reduces variance by comparing paired trajectories and treating the entire sampling process as a single action. The approach demonstrates faster convergence and better image quality and prompt alignment compared to existing methods.

• →New RL variant reduces variance in model updates by sampling paired trajectories and optimizing flow velocity toward better images
• →Method treats entire sampling process as single action rather than treating each step separately
• →Approach shows faster convergence than previous reinforcement learning methods for diffusion models
• →Evaluation using vision language models and quality metrics demonstrates improved output quality
• →Results show better prompt alignment compared to existing post-training techniques