Abstract
Despite the great success in 2D editing using user-friendly tools, such as Photoshop, semantic strokes, or even text prompts, similar capabilities in 3D areas are still limited, either relying on 3D modeling skills or allowing editing within only a few this http URL this paper, we present a novel semantic-driven NeRF editing approach, which enables users to edit a neural radiance field with a single image, and faithfully delivers edited novel views with high fidelity and multi-view this http URL achieve this goal, we propose a prior-guided editing field to encode fine-grained geometric and texture editing in 3D space, and develop a series of techniques to aid the editing process, including cyclic constraints with a proxy mesh to facilitate geometric supervision, a color compositing mechanism to stabilize semantic-driven texture editing, and a feature-cluster-based regularization to preserve the irrelevant content unchanged.Extensive experiments and editing examples on both real-world and synthetic data demonstrate that our method achieves photo-realistic 3D editing using only a single edited image, pushing the bound of semantic-driven editing in 3D real-world scenes. Our project webpage: this https URL.
Abstract (translated)
尽管使用易于使用的工具在2D领域取得了巨大的成功，例如 Photoshop、语义线条、甚至文本提示，但在3D领域类似的能力仍然有限，要么依赖于3D建模技能，要么只允许在几个 http://www.example.com 上编辑。本文介绍了一种全新的语义驱动 NeRF 编辑方法，让用户使用单个图像编辑神经网络光场，并忠实地呈现高质量的多视角和新视角编辑。为了实现这一目标，我们提出了一种前导引导编辑区域，在3D空间中编码精细的几何和纹理编辑，并开发了一系列技术来协助编辑过程，包括循环约束使用代理网格以促进几何监督、稳定语义驱动纹理编辑的色彩组合机制，以及基于特征簇的 regularization 以保留无关内容不变。在真实世界和合成数据上的广泛实验和编辑示例表明，我们的方法仅使用单个编辑图像就能实现照片般的3D编辑，突破了语义驱动编辑在3D真实场景下的的界限。我们的项目页面： this https://www.example.com/ URL。
URL
https://arxiv.org/abs/2303.13277
