Abstract
Stereo video generation has been gaining increasing attention with recent advancements in video diffusion models. However, most existing methods focus on generating 3D stereoscopic videos from monocular 2D videos. These approaches typically assume that the input monocular video is of high quality, making the task primarily about inpainting occluded regions in the warped video while preserving disoccluded areas. In this paper, we introduce a new pipeline that not only generates stereo videos but also enhances both left-view and right-view videos consistently with a single model. Our approach achieves this by fine-tuning the model on degraded data for restoration, as well as conditioning the model on warped masks for consistent stereo generation. As a result, our method can be fine-tuned on a relatively small synthetic stereo video datasets and applied to low-quality real-world videos, performing both stereo video generation and restoration. Experiments demonstrate that our method outperforms existing approaches both qualitatively and quantitatively in stereo video generation from low-resolution inputs.
Abstract (translated)
最近在视频扩散模型方面的进展使得立体视频生成越来越受到关注。然而，大多数现有的方法主要集中在从单目二维视频生成三维立体视频上。这些方法通常假设输入的单目视频质量较高，并且任务主要是修复扭曲视频中被遮挡区域的同时保留新显露出来的区域。在这篇论文中，我们介绍了一种新的流水线技术，它不仅可以生成立体视频，还可以通过单一模型同时增强左视图和右视图视频的质量。我们的方法通过在退化数据上进行微调来进行恢复，并通过对扭曲掩码进行条件设置来实现一致的立体生成。因此，我们的方法可以使用相对较小的人工合成立体视频数据集进行微调，并应用于低质量的真实世界视频中，从而同时完成立体视频生成和修复的任务。实验结果表明，在从低分辨率输入生成立体视频方面，我们的方法在定性和定量评估上都优于现有的方法。
URL
https://arxiv.org/abs/2506.06023
