Abstract
We study the problem of reconstructing 3D feature curves of an object from a set of calibrated multi-view images. To do so, we learn a neural implicit field representing the density distribution of 3D edges which we refer to as Neural Edge Field (NEF). Inspired by NeRF, NEF is optimized with a view-based rendering loss where a 2D edge map is rendered at a given view and is compared to the ground-truth edge map extracted from the image of that view. The rendering-based differentiable optimization of NEF fully exploits 2D edge detection, without needing a supervision of 3D edges, a 3D geometric operator or cross-view edge correspondence. Several technical designs are devised to ensure learning a range-limited and view-independent NEF for robust edge extraction. The final parametric 3D curves are extracted from NEF with an iterative optimization method. On our benchmark with synthetic data, we demonstrate that NEF outperforms existing state-of-the-art methods on all metrics. Project page: this https URL.
Abstract (translated)
我们研究从一组校准多视角图像中恢复物体的3D特征曲线的问题。为了解决这个问题,我们学习了一个神经网络隐含区域,代表3D边缘密度分布,我们称之为神经网络边缘场(NEF)。受到NeRF启发,NEF使用视点渲染损失优化,其中2D边缘图在一个给定视角下渲染,并与从该视角图像中提取的基线边缘图进行比较。基于渲染的可区分优化NEF完全利用2D边缘检测,不需要对3D边缘进行监督,也不需要3D几何操作或跨视角边缘对应。几种技术设计旨在确保学习限制范围且视点独立的NEF,以 robust 3D边缘提取。最后参数化的3D曲线从NEF中通过迭代优化方法提取。在我们的合成数据基准测试中,我们证明NEF在所有指标上都比现有的先进方法表现更好。项目页面:这个https URL。
URL
https://arxiv.org/abs/2303.07653
