Abstract
Recent advances in neural rendering have shown great potential for reconstructing scenes from multiview images. However, accurately representing objects with glossy surfaces remains a challenge for existing methods. In this work, we introduce ENVIDR, a rendering and modeling framework for high-quality rendering and reconstruction of surfaces with challenging specular reflections. To achieve this, we first propose a novel neural renderer with decomposed rendering components to learn the interaction between surface and environment lighting. This renderer is trained using existing physically based renderers and is decoupled from actual scene representations. We then propose an SDF-based neural surface model that leverages this learned neural renderer to represent general scenes. Our model additionally synthesizes indirect illuminations caused by inter-reflections from shiny surfaces by marching surface-reflected rays. We demonstrate that our method outperforms state-of-art methods on challenging shiny scenes, providing high-quality rendering of specular reflections while also enabling material editing and scene relighting.
Abstract (translated)
神经网络渲染最近的进展已经表明可以从多视角图像中重建场景的巨大潜力。然而，准确代表具有光滑表面的物体仍然是现有方法所面临的挑战。在本文中，我们介绍了ENVIDR，一个渲染和建模框架，用于高质量渲染和重建具有挑战性的光滑反射表面的表面。为了实现这一点，我们提出了一种新的神经网络渲染器，分解渲染组件以学习表面和环境照明之间的交互。这个渲染器使用现有的物理基渲染器进行训练，并将其与实际场景表示分离。随后，我们提出了基于SDF的神经网络表面模型，该模型利用已学习的神经网络渲染器来代表一般场景。我们的模型此外合成由光滑表面间的反射引起的间接照明。我们证明了我们的方法在挑战性的光滑场景方面优于最先进的方法，提供高质量的镜面反射渲染，同时还可以编辑材料和重新照亮场景。
URL
https://arxiv.org/abs/2303.13022
