Abstract
Decomposing an object's appearance into representations of its materials and the surrounding illumination is difficult, even when the object's 3D shape is known beforehand. This problem is ill-conditioned because diffuse materials severely blur incoming light, and is ill-posed because diffuse materials under high-frequency lighting can be indistinguishable from shiny materials under low-frequency lighting. We show that it is possible to recover precise materials and illumination -- even from diffuse objects -- by exploiting unintended shadows, like the ones cast onto an object by the photographer who moves around it. These shadows are a nuisance in most previous inverse rendering pipelines, but here we exploit them as signals that improve conditioning and help resolve material-lighting ambiguities. We present a method based on differentiable Monte Carlo ray tracing that uses images of an object to jointly recover its spatially-varying materials, the surrounding illumination environment, and the shapes of the unseen light occluders who inadvertently cast shadows upon it.
Abstract (translated)
将物体的外观分解成其材料和周围环境的表示方法是困难的，即使物体的三维形状已知。这个问题是Conditioning不足的，因为扩散材料严重模糊入射光，也因为高频照明下的扩散材料可以与低频照明下的闪亮材料分辨不清。我们表明，可以利用意外生成的 shadows，例如摄影师围绕物体移动时生成的 shadows。这些 shadows 在大多数先前的反渲染管道中都是令人困扰的，但在这里我们利用它们作为改善 conditioning 和解决材料照明混淆的信号。我们提出了基于不同变的蒙特卡罗射线渲染的方法，该方法使用物体的图像一起恢复其空间 varying 的材料、周围的照明环境，以及无意中对物体生成的光遮蔽器的形状。
URL
https://arxiv.org/abs/2305.16321
