MovieTeller: Tool-augmented Movie Synopsis with ID Consistent Progressive Abstraction

Researchers introduce MovieTeller, a new AI framework that generates accurate movie synopses by combining face recognition tools with Vision-Language Models to maintain character consistency and narrative coherence. The training-free approach uses progressive abstraction to overcome current VLM limitations in processing long-form video content.

• →MovieTeller framework addresses critical failures in existing Vision-Language Models for long-duration video summarization.
• →The system uses face recognition tools to establish factual character groundings and consistent ID tracking throughout movies.
• →Progressive abstraction pipeline breaks down full-length movie summarization into manageable multi-stage processes.
• →The approach requires no costly model fine-tuning and works with off-the-shelf models in plug-and-play manner.
• →Experiments show significant improvements in factual accuracy, character consistency, and narrative coherence over baseline methods.