Abstract
We present "Narrative Weaver", a novel framework that addresses a fundamental challenge in generative AI: achieving multi-modal controllable, long-range, and consistent visual content generation. While existing models excel at generating high-fidelity short-form visual content, they struggle to maintain narrative coherence and visual consistency across extended sequences - a critical limitation for real-world applications such as filmmaking and e-commerce advertising. Narrative Weaver introduces the first holistic solution that seamlessly integrates three essential capabilities: fine-grained control, automatic narrative planning, and long-range coherence. Our architecture combines a Multimodal Large Language Model (MLLM) for high-level narrative planning with a novel fine-grained control module featuring a dynamic Memory Bank that prevents visual drift. To enable practical deployment, we develop a progressive, multi-stage training strategy that efficiently leverages existing pre-trained models, achieving state-of-the-art performance even with limited training data. Recognizing the absence of suitable evaluation benchmarks, we construct and release the E-commerce Advertising Video Storyboard Dataset (EAVSD) - the first comprehensive dataset for this task, containing over 330K high-quality images with rich narrative annotations. Through extensive experiments across three distinct scenarios (controllable multi-scene generation, autonomous storytelling, and e-commerce advertising), we demonstrate our method's superiority while opening new possibilities for AI-driven content creation.
Abstract (translated)
我们介绍了“叙事织工”(Narrative Weaver)，这是一个新颖的框架，旨在解决生成式人工智能中的一个基本挑战：实现多模态可控、长跨度和视觉内容的一致性生成。虽然现有的模型在生成高保真度的短格式视觉内容方面表现出色，但在扩展序列中保持叙述连贯性和视觉一致性方面存在困难——这对于电影制作和电子商务广告等现实世界应用而言是一个关键限制。“叙事织工”引入了第一个全面解决方案，无缝整合三种基本能力：细粒度控制、自动叙述规划以及长跨度的一致性。我们的架构结合了一种多模态大型语言模型（MLLM）用于高层次的叙述规划，并配备了一个具有动态记忆库的新颖细粒度控制模块，以防止视觉漂移。 为了实现实际部署，“叙事织工”开发了一种渐进式的、多阶段训练策略，高效利用现有的预训练模型，在数据有限的情况下也能达到最先进的性能。由于缺乏合适的评估基准，“我们构建并发布了电子商务广告视频故事版数据集（EAVSD）——这是该任务的第一个综合性数据集，包含超过33万张高质量图像及丰富的叙述注释。 通过在三种不同的场景中进行广泛的实验（可控多场景生成、自主叙事和电子商务广告），我们展示了“叙事织工”的优越性能，并开辟了人工智能驱动内容创作的新可能。
URL
https://arxiv.org/abs/2603.06688
