AsymTalker: Identity-Consistent Long-Term Talking Head Generation via Asymmetric Distillation

AsymTalker introduces a diffusion-based method for generating long-form talking head videos with consistent identity and synchronized audio. The approach solves critical challenges in extended video synthesis through temporal reference encoding and asymmetric knowledge distillation, achieving real-time performance at 66 FPS on videos up to 10 minutes long.

AsymTalker addresses a significant technical bottleneck in generative video synthesis. Previous diffusion-based talking head systems process videos in chunks, which creates two cascading problems: the static identity reference becomes temporally misaligned with dynamic audio streams, and identity drift compounds across chunks as self-generated frames feed into subsequent generations. This represents a fundamental limitation preventing practical deployment for long-form content.

The proposed solution employs an elegant asymmetric design where a teacher model anchors to ground-truth references during training while the student model learns under inference-aligned conditions with only self-generated frames. This separation prevents train-test mismatch while avoiding the identity degradation that plagued previous approaches. Temporal Reference Encoding further stabilizes identity by encoding a pseudo-video rather than treating the identity image as static, maintaining coherence across the temporal dimension without additional parameters.

The technical achievement extends beyond academic merit into practical applicability. Achieving 600-second video synthesis with real-time inference (66 FPS) on established benchmarks (HDTF, VFHQ) positions this work at the frontier of generative video capabilities. The implications ripple across entertainment, content creation, and digital communication sectors where synthetic talking head videos enable scalable personalization and accessibility applications.

Future development likely focuses on scaling to longer contexts, improving audio-visual synchronization precision, and reducing computational requirements for broader adoption. The asymmetric distillation framework may inspire similar approaches in other sequential generation tasks facing train-inference mismatch problems.

• →AsymTalker solves temporal-spatial misalignment and identity drift in long-form talking head generation through asymmetric knowledge distillation
• →The method achieves state-of-the-art results on HDTF and VFHQ benchmarks with consistent identity over 600-second videos
• →Real-time inference speed of 66 FPS enables practical deployment for content creation applications
• →Asymmetric teacher-student training design prevents both train-test mismatch and cascading identity degradation across video chunks
• →Temporal Reference Encoding maintains identity coherence without introducing additional model parameters