Abstract
We introduce Intrinsic Image Fusion, a method that reconstructs high-quality physically based materials from multi-view images. Material reconstruction is highly underconstrained and typically relies on analysis-by-synthesis, which requires expensive and noisy path tracing. To better constrain the optimization, we incorporate single-view priors into the reconstruction process. We leverage a diffusion-based material estimator that produces multiple, but often inconsistent, candidate decompositions per view. To reduce the inconsistency, we fit an explicit low-dimensional parametric function to the predictions. We then propose a robust optimization framework using soft per-view prediction selection together with confidence-based soft multi-view inlier set to fuse the most consistent predictions of the most confident views into a consistent parametric material space. Finally, we use inverse path tracing to optimize for the low-dimensional parameters. Our results outperform state-of-the-art methods in material disentanglement on both synthetic and real scenes, producing sharp and clean reconstructions suitable for high-quality relighting.
Abstract (translated)
我们介绍了内在图像融合（Intrinsic Image Fusion）方法，该方法可以从多视角图像中重建高质量的基于物理原理的材料。材料重建问题高度欠约束，并且通常依赖于综合分析（analysis-by-synthesis），这需要昂贵且噪声较多的路径追踪计算。为了更好地约束优化过程，我们在重建过程中加入了单视图先验信息。我们利用了一种扩散基础的材质估计器，该估计器能够为每个视角生成多个候选分解方案，但这些方案往往并不一致。为了减少这种不一致性，我们将显式的低维参数化函数拟合到预测结果上。 接下来，我们提出了一种鲁棒的优化框架，通过软视图预测选择与基于置信度的软多视图内点集来融合最具置信视角中最一致性的预测值，在一致化的参数化材质空间中。最后，我们使用逆路径追踪技术对低维参数进行优化。 我们的方法在合成场景和真实场景中的材料分离任务上均超越了现有最先进的方法，并生成了适用于高质量重光照的清晰且干净的重建结果。
URL
https://arxiv.org/abs/2512.13157
