Abstract
Recent advances in Novel View Synthesis (NVS) and 3D generation have significantly improved editing tasks, with a primary emphasis on maintaining cross-view consistency throughout the generative process. Contemporary methods typically address this challenge using a dual-strategy framework: performing consistent 2D inpainting across all views guided by embedded priors either explicitly in pixel space or implicitly in latent space; and conducting 3D reconstruction with additional consistency guidance. Previous strategies, in particular, often require an initial 3D reconstruction phase to establish geometric structure, introducing considerable computational overhead. Even with the added cost, the resulting reconstruction quality often remains suboptimal. In this paper, we present VEIGAR, a computationally efficient framework that outperforms existing methods without relying on an initial reconstruction phase. VEIGAR leverages a lightweight foundation model to reliably align priors explicitly in the pixel space. In addition, we introduce a novel supervision strategy based on scale-invariant depth loss, which removes the need for traditional scale-and-shift operations in monocular depth regularization. Through extensive experimentation, VEIGAR establishes a new state-of-the-art benchmark in reconstruction quality and cross-view consistency, while achieving a threefold reduction in training time compared to the fastest existing method, highlighting its superior balance of efficiency and effectiveness.
Abstract (translated)
最近在新颖视图合成（NVS）和3D生成方面取得的进展显著提升了编辑任务的效果，特别是在整个生成过程中的跨视角一致性维护方面。当前的方法通常采用双管齐下的框架来应对这一挑战：一种是通过像素空间中显式的嵌入先验或潜在空间中的隐式嵌入先验来进行所有视图的一致性2D修补；另一种是在进行3D重建时增加了一致性的指导。以往的策略往往需要一个初始的3D重建阶段来建立几何结构，这会带来相当大的计算开销。即使在增加了这一成本的情况下，最终得到的重建质量仍然常常不尽人意。 在这篇论文中，我们提出了一种名为VEIGAR的新框架，它比现有的方法更加高效且无需依赖于最初的重建阶段。VEIGAR利用一个轻量级的基础模型，在像素空间内可靠地对齐先验。此外，我们还引入了一种基于尺度不变深度损失的新型监督策略，这种方法消除了传统单目深度正则化中所需的缩放和平移操作。 通过广泛的实验，VEIGAR在重建质量和跨视角一致性方面建立了新的行业标准，并且训练时间比最快的现有方法减少了三倍，突显了其效率与效果之间的卓越平衡。
URL
https://arxiv.org/abs/2506.15821
