Abstract
Appearance editing according to user needs is a pivotal task in video editing. Existing text-guided methods often lead to ambiguities regarding user intentions and restrict fine-grained control over editing specific aspects of objects. To overcome these limitations, this paper introduces a novel approach named {Zero-to-Hero}, which focuses on reference-based video editing that disentangles the editing process into two distinct problems. It achieves this by first editing an anchor frame to satisfy user requirements as a reference image and then consistently propagating its appearance across other frames. We leverage correspondence within the original frames to guide the attention mechanism, which is more robust than previously proposed optical flow or temporal modules in memory-friendly video generative models, especially when dealing with objects exhibiting large motions. It offers a solid ZERO-shot initialization that ensures both accuracy and temporal consistency. However, intervention in the attention mechanism results in compounded imaging degradation with over-saturated colors and unknown blurring issues. Starting from Zero-Stage, our Hero-Stage Holistically learns a conditional generative model for vidEo RestOration. To accurately evaluate the consistency of the appearance, we construct a set of videos with multiple appearances using Blender, enabling a fine-grained and deterministic evaluation. Our method outperforms the best-performing baseline with a PSNR improvement of 2.6 dB. The project page is at this https URL.
Abstract (translated)
根据用户需求进行外观编辑是视频编辑中的关键任务之一。现有的文本引导方法常常导致关于用户意图的模糊性，并限制了对特定对象方面进行精细控制的能力。为了克服这些局限，本文提出了一种名为{Zero-to-Hero}的新方法，该方法专注于基于参考的视频编辑，将编辑过程分解为两个不同的问题：首先编辑一个锚帧以满足用户的需要作为参考图像，然后在其他帧中一致地传播其外观。我们利用原始帧之间的对应关系来指导注意力机制，在处理具有大幅度运动的对象时，这种方法比以前提出的光流或时间模块更稳健，尤其是在内存友好的视频生成模型中更为适用。它提供了一个坚实的零样本初始化，确保了准确性和时间一致性。然而，干预注意力机制会导致图像退化的问题加重，表现为过饱和的颜色和未知的模糊问题。 从Zero-Stage开始，我们的Hero-Stage全面学习了一种条件生成模型用于视频恢复。为了精确评估外观的一致性，我们使用Blender构建了一系列具有多种外观变化的视频集，从而进行精细且确定性的评估。我们的方法超越了最佳基线，在PSNR（峰值信噪比）上提高了2.6分贝。 该项目页面可在以下链接访问：[https URL] （请注意，实际URL需要被替换为项目的具体网址）。
URL
https://arxiv.org/abs/2505.23134
