Abstract
Video object removal and inpainting are critical tasks in the fields of computer vision and multimedia processing, aimed at restoring missing or corrupted regions in video sequences. Traditional methods predominantly rely on flow-based propagation and spatio-temporal Transformers, but these approaches face limitations in effectively leveraging long-term temporal features and ensuring temporal consistency in the completion results, particularly when dealing with large masks. Consequently, performance on extensive masked areas remains suboptimal. To address these challenges, this paper introduces a novel video inpainting approach leveraging the Diffusion Transformer (DiT). DiT synergistically combines the advantages of diffusion models and transformer architectures to maintain long-term temporal consistency while ensuring high-quality inpainting results. We propose a Circular Position-Shift strategy to further enhance long-term temporal consistency during the inference stage. Additionally, the proposed method automatically detects objects within videos, interactively removes specified objects, and generates corresponding prompts. In terms of processing speed, it takes only 180 seconds (testing on one NVIDIA A100 GPU) to complete a video with a resolution of $1080 \times 1920$ with 121 frames without any acceleration method. Experimental results indicate that the proposed method demonstrates superior performance in content fidelity, texture restoration, and temporal consistency. Project page: this https URL.
Abstract (translated)
视频对象移除和修复是计算机视觉与多媒体处理领域中的关键任务，旨在恢复视频序列中丢失或损坏的区域。传统方法主要依赖于基于流的传播和时空Transformer，但这些方法在有效利用长期时间特征及确保完成结果的时间一致性方面存在局限性，特别是在处理大范围遮罩时尤为明显。因此，在广泛的被屏蔽区域内性能仍然不理想。 为解决这些问题，本文介绍了一种新颖的视频修复方法，该方法利用扩散变换器（Diffusion Transformer, DiT）。DiT 结合了扩散模型和变换器架构的优势，在保持长期时间一致性的同时确保高质量的修复结果。我们还提出了一种圆形位置偏移策略，以进一步增强推理阶段中的长时一致性。此外，所提出的这种方法能够自动检测视频内的对象，并交互式地移除指定的对象并生成相应的提示。 在处理速度方面，在没有加速方法的情况下使用一个 NVIDIA A100 GPU 测试分辨率为 1080x1920、包含 121 帧的视频，整个修复过程仅需 180 秒。实验结果表明，所提出的方法在内容保真度、纹理恢复和时间一致性方面表现出色。 项目页面：[点击这里查看](https://this-url.com/)（请将“this https URL”替换为实际链接）。
URL
https://arxiv.org/abs/2506.12853
