Paper Reading AI Learner

UMat: Uncertainty-Aware Single Image High Resolution Material Capture

2023-05-25 17:59:04
Carlos Rodriguez-Pardo, Henar Dominguez-Elvira, David Pascual-Hernandez, Elena Garces


We propose a learning-based method to recover normals, specularity, and roughness from a single diffuse image of a material, using microgeometry appearance as our primary cue. Previous methods that work on single images tend to produce over-smooth outputs with artifacts, operate at limited resolution, or train one model per class with little room for generalization. Previous methods that work on single images tend to produce over-smooth outputs with artifacts, operate at limited resolution, or train one model per class with little room for generalization. In contrast, in this work, we propose a novel capture approach that leverages a generative network with attention and a U-Net discriminator, which shows outstanding performance integrating global information at reduced computational complexity. We showcase the performance of our method with a real dataset of digitized textile materials and show that a commodity flatbed scanner can produce the type of diffuse illumination required as input to our method. Additionally, because the problem might be illposed -more than a single diffuse image might be needed to disambiguate the specular reflection- or because the training dataset is not representative enough of the real distribution, we propose a novel framework to quantify the model's confidence about its prediction at test time. Our method is the first one to deal with the problem of modeling uncertainty in material digitization, increasing the trustworthiness of the process and enabling more intelligent strategies for dataset creation, as we demonstrate with an active learning experiment.

Abstract (translated)

我们提出了一种基于学习的方法,从材料的一个均匀图像中恢复正则性、亮度和粗糙度,使用微几何外观作为主要线索。以前的方法和在单个图像上工作的方法和以前的方法都倾向于产生平滑的输出带有 artifacts,只能在有限的分辨率下操作或训练每个类别的单个模型,几乎没有泛化能力。相比之下,在本文中,我们提出了一种新的捕获方法,利用注意力机制和 U-Net 分类器,表现出卓越的性能,在减少计算复杂度的情况下整合全球信息。我们使用数字纺织材料数据集展示了我们方法的性能,并表明商品平板扫描仪可以产生所需的均匀照明类型,此外,由于问题可能不存在或不完备,可能需要多个均匀图像才能区分正则反射,或者训练数据集不足以代表真实分布,我们提出了一种新的框架,以量化模型在测试时的自信程度。我们的方法是第一个处理材料数字化中的不确定性问题的方法,提高了过程的可靠性,并 enabling 更聪明的数据集创建策略,正如我们使用主动学习实验所证明的那样。



3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot