Abstract
Computing the gradients of a rendering process is paramount for diverse applications in computer vision and graphics. However, accurate computation of these gradients is challenging due to discontinuities and rendering approximations, particularly for surface-based representations and rasterization-based rendering. We present a novel method for computing gradients at visibility discontinuities for rasterization-based differentiable renderers. Our method elegantly simplifies the traditionally complex problem through a carefully designed approximation strategy, allowing for a straightforward, effective, and performant solution. We introduce a novel concept of micro-edges, which allows us to treat the rasterized images as outcomes of a differentiable, continuous process aligned with the inherently non-differentiable, discrete-pixel rasterization. This technique eliminates the necessity for rendering approximations or other modifications to the forward pass, preserving the integrity of the rendered image, which makes it applicable to rasterized masks, depth, and normals images where filtering is prohibitive. Utilizing micro-edges simplifies gradient interpretation at discontinuities and enables handling of geometry intersections, offering an advantage over the prior art. We showcase our method in dynamic human head scene reconstruction, demonstrating effective handling of camera images and segmentation masks.
Abstract (translated)
计算渲染过程中梯度的计算对于计算机视觉和图形学中的各种应用至关重要。然而，由于不连续性和渲染近似，准确计算这些梯度具有挑战性，特别是在基于表面的表示和基于元组织的渲染中。我们提出了一种新的方法，用于计算基于元组织的渲染中的可见性断点处的梯度。我们的方法通过精心设计的近似策略，将通常复杂的问题简化为一个简单而有效的解决方案。我们引入了一个名为微边（micro-edges）的新概念，使我们可以将栅化图像视为与固有非可导性离散像素渲染过程的连续过程的输出。这种技术消除了对前向传播的渲染近似或其他修改的需要，保留了渲染图像的完整性，使其适用于无法进行滤波的栅化mask、深度和法线图像。利用微边简化了在断点处的梯度解释，并使处理几何交涉及纳，提供了与先例技术相比的优势。我们在动态人类头部场景重构中展示我们的方法，证明了在处理相机图像和分割掩码方面具有有效的处理能力。
URL
https://arxiv.org/abs/2405.02508
