Abstract
With the explosive growth of digital entertainment, automated video summarization has become indispensable for applications such as content indexing, personalized recommendation, and efficient media archiving. Automatic synopsis generation for long-form videos, such as movies and TV series, presents a significant challenge for existing Vision-Language Models (VLMs). While proficient at single-image captioning, these general-purpose models often exhibit critical failures in long-duration contexts, primarily a lack of ID-consistent character identification and a fractured narrative coherence. To overcome these limitations, we propose MovieTeller, a novel framework for generating movie synopses via tool-augmented progressive abstraction. Our core contribution is a training-free, tool-augmented, fact-grounded generation process. Instead of requiring costly model fine-tuning, our framework directly leverages off-the-shelf models in a plug-and-play manner. We first invoke a specialized face recognition model as an external "tool" to establish Factual Groundings--precise character identities and their corresponding bounding boxes. These groundings are then injected into the prompt to steer the VLM's reasoning, ensuring the generated scene descriptions are anchored to verifiable facts. Furthermore, our progressive abstraction pipeline decomposes the summarization of a full-length movie into a multi-stage process, effectively mitigating the context length limitations of current VLMs. Experiments demonstrate that our approach yields significant improvements in factual accuracy, character consistency, and overall narrative coherence compared to end-to-end baselines.
Abstract (translated)
随着数字娱乐的爆炸性增长，自动视频摘要在内容索引、个性化推荐和高效媒体存档等方面变得不可或缺。对于长篇视频（如电影和电视剧）的自动情节生成，现有视觉-语言模型（VLMs）面临着重大挑战。虽然这些通用模型在单张图片描述方面表现出色，但在长时间背景下往往出现关键性失败，主要表现为人物识别一致性不足以及叙事连贯性的断裂。 为了克服这些限制，我们提出了MovieTeller框架，它通过工具增强的渐进式抽象来生成电影简介。我们的核心贡献是一个无需训练、基于事实且利用外部工具辅助的过程。与需要昂贵模型微调的方法不同，我们的框架可以直接以即插即用的方式使用现成的模型。 具体来说，我们首先引入一个专门的人脸识别模型作为外部“工具”，用于建立事实基础——精确的人物身份及其对应的边界框。这些基础信息随后被注入到提示中，引导VLM进行推理，确保生成的场景描述与可验证的事实相一致。 此外，我们的渐进式抽象流水线将一部完整电影的摘要工作分解为多阶段过程，有效地缓解了现有VLM在处理长文本时的上下文长度限制问题。实验表明，相较于端到端的基础方法，我们的方法显著提高了事实准确性、人物一致性以及整体叙事连贯性。
URL
https://arxiv.org/abs/2602.23228
