Abstract
We propose a learning-based method to recover normals, specularity, and roughness from a single diffuse image of a material, using microgeometry appearance as our primary cue. Previous methods that work on single images tend to produce over-smooth outputs with artifacts, operate at limited resolution, or train one model per class with little room for generalization. Previous methods that work on single images tend to produce over-smooth outputs with artifacts, operate at limited resolution, or train one model per class with little room for generalization. In contrast, in this work, we propose a novel capture approach that leverages a generative network with attention and a U-Net discriminator, which shows outstanding performance integrating global information at reduced computational complexity. We showcase the performance of our method with a real dataset of digitized textile materials and show that a commodity flatbed scanner can produce the type of diffuse illumination required as input to our method. Additionally, because the problem might be illposed -more than a single diffuse image might be needed to disambiguate the specular reflection- or because the training dataset is not representative enough of the real distribution, we propose a novel framework to quantify the model's confidence about its prediction at test time. Our method is the first one to deal with the problem of modeling uncertainty in material digitization, increasing the trustworthiness of the process and enabling more intelligent strategies for dataset creation, as we demonstrate with an active learning experiment.
Abstract (translated)
我们提出了一种基于学习的方法,从材料的一个均匀图像中恢复正则性、亮度和粗糙度,使用微几何外观作为主要线索。以前的方法和在单个图像上工作的方法和以前的方法都倾向于产生平滑的输出带有 artifacts,只能在有限的分辨率下操作或训练每个类别的单个模型,几乎没有泛化能力。相比之下,在本文中,我们提出了一种新的捕获方法,利用注意力机制和 U-Net 分类器,表现出卓越的性能,在减少计算复杂度的情况下整合全球信息。我们使用数字纺织材料数据集展示了我们方法的性能,并表明商品平板扫描仪可以产生所需的均匀照明类型,此外,由于问题可能不存在或不完备,可能需要多个均匀图像才能区分正则反射,或者训练数据集不足以代表真实分布,我们提出了一种新的框架,以量化模型在测试时的自信程度。我们的方法是第一个处理材料数字化中的不确定性问题的方法,提高了过程的可靠性,并 enabling 更聪明的数据集创建策略,正如我们使用主动学习实验所证明的那样。
URL
https://arxiv.org/abs/2305.16312