Abstract
Video outpainting presents a unique challenge of extending the borders while maintaining consistency with the given content. In this paper, we suggest the use of video inpainting models that excel in object flow learning and reconstruction in outpainting rather than solely generating the background as in existing methods. However, directly applying or fine-tuning inpainting models to outpainting has shown to be ineffective, often leading to blurry results. Our extensive experiments on discriminator designs reveal that a critical component missing in the outpainting fine-tuning process is a discriminator capable of effectively assessing the perceptual quality of the extended areas. To tackle this limitation, we differentiate the objectives of adversarial training into global and local goals and introduce a hierarchical discriminator that meets both objectives. Additionally, we develop a specialized outpainting loss function that leverages both local and global features of the discriminator. Fine-tuning on this adversarial loss function enhances the generator's ability to produce both visually appealing and globally coherent outpainted scenes. Our proposed method outperforms state-of-the-art methods both quantitatively and qualitatively. Supplementary materials including the demo video and the code are available in SigPort.
Abstract (translated)
视频扩展（outpainting）在保持与给定内容一致性的同时，具有扩展边界的独特挑战。本文建议采用擅长于对象流动学习和重构的视频修复模型来应对这一问题，而不是像现有方法那样仅生成背景部分。然而，直接应用或微调这些修复模型到扩展任务中被证明是无效的，并且经常导致模糊的结果。我们通过大量关于判别器设计的实验发现，在微调过程中缺失的关键组件是一个能够有效评估扩展区域感知质量的判别器。 为了解决这一局限性，我们将对抗训练的目标分为全局和局部目标，并引入了一个既能满足全局又能满足局部目标的分层判别器。此外，我们开发了一种专门针对视频扩展任务的损失函数，该函数利用了判别器的本地和全球特征。在这一对抗损失函数上的微调提升了生成器的能力，使其能够产生既美观又具有全局一致性的视频扩展场景。 我们的方法在定量和定性评估中都优于现有的最先进的技术。补充材料包括演示视频和代码可在SigPort上获取。
URL
https://arxiv.org/abs/2508.00418
