FlowPortrait: Reinforcement Learning for Audio-Driven Portrait Video Generation

FlowPortrait is a new reinforcement learning framework that uses Multimodal Large Language Models for evaluation to generate more realistic talking-head videos with better lip synchronization. The system combines human-aligned assessment with policy optimization techniques to address persistent issues in audio-driven portrait animation.

• →FlowPortrait addresses key challenges in talking-head video generation including poor lip sync and unnatural motion.
• →The framework uses Multimodal Large Language Models to create human-aligned evaluation metrics for video quality assessment.
• →Group Relative Policy Optimization is employed to post-train the generator using composite reward signals.
• →Extensive experiments show consistent improvements in video quality compared to existing methods.
• →The approach demonstrates the effectiveness of reinforcement learning for portrait animation tasks.