Abstract
Temporal consistency is essential for video editing applications. Existing work on layered representation of videos allows propagating edits consistently to each frame. These methods, however, can only edit object appearance rather than object shape changes due to the limitation of using a fixed UV mapping field for texture atlas. We present a shape-aware, text-driven video editing method to tackle this challenge. To handle shape changes in video editing, we first propagate the deformation field between the input and edited keyframe to all frames. We then leverage a pre-trained text-conditioned diffusion model as guidance for refining shape distortion and completing unseen regions. The experimental results demonstrate that our method can achieve shape-aware consistent video editing and compare favorably with the state-of-the-art.
Abstract (translated)
时间一致性对于视频编辑应用至关重要。现有的视频分层表示方法允许连续地传播编辑到每个帧。然而,这些方法只能编辑物体外观而不是物体形状变化,因为使用固定的UV映射 field 用于纹理集网格的限制。我们提出了一种形状意识的文本驱动的视频编辑方法来解决这个问题。为了处理视频编辑中的形状变化,我们首先将输入和编辑关键帧之间的变形 field 传播到所有帧。然后,我们利用预训练的文本Conditioned扩散模型作为指导,以改进形状扭曲和完成未知的区域。实验结果表明,我们的方法可以实现形状意识的连续视频编辑,并与当前技术水平相当。
URL
https://arxiv.org/abs/2301.13173
