Paper Reading AI Learner

Learning Self-Prior for Mesh Inpainting Using Self-Supervised Graph Convolutional Networks

2023-05-01 02:51:38
Shota Hattori, Tatsuya Yatagawa, Yutaka Ohtake, Hiromasa Suzuki

Abstract

This study presents a self-prior-based mesh inpainting framework that requires only an incomplete mesh as input, without the need for any training datasets. Additionally, our method maintains the polygonal mesh format throughout the inpainting process without converting the shape format to an intermediate, such as a voxel grid, a point cloud, or an implicit function, which are typically considered easier for deep neural networks to process. To achieve this goal, we introduce two graph convolutional networks (GCNs): single-resolution GCN (SGCN) and multi-resolution GCN (MGCN), both trained in a self-supervised manner. Our approach refines a watertight mesh obtained from the initial hole filling to generate a completed output mesh. Specifically, we train the GCNs to deform an oversmoothed version of the input mesh into the expected completed shape. To supervise the GCNs for accurate vertex displacements, despite the unknown correct displacements at real holes, we utilize multiple sets of meshes with several connected regions marked as fake holes. The correct displacements are known for vertices in these fake holes, enabling network training with loss functions that assess the accuracy of displacement vectors estimated by the GCNs. We demonstrate that our method outperforms traditional dataset-independent approaches and exhibits greater robustness compared to other deep-learning-based methods for shapes that less frequently appear in shape datasets.

Abstract (translated)

这项研究提出了一种基于自我先验性的网格填充框架,只需要不完整的网格作为输入,而不需要任何训练数据集。此外,我们的方法和传统的数据驱动方法不同,不会将形状格式转换为中间格式,如立方点网格、点云或隐含函数,这些通常被认为是深度学习网络处理更简单的格式。为了实现这一目标,我们引入了两个图卷积神经网络(GCNs):单分辨率GCN(SGCN)和多分辨率GCN(MGCN),均通过自我监督训练完成。我们改进了从初始漏洞填充中获得的密封网格,生成完整的输出网格。具体来说,我们训练GCNs将输入网格的过度平滑版本变形为预期的完整形状。为了监督GCNs准确的顶点位移,尽管真实的漏洞中未知的正确位移,我们使用了多个带有多个连接区域标记为假漏洞的网格集。这些正确的位移对于在这些假漏洞中的顶点是已知的,因此可以使用损失函数训练网络,评估由GCNs估计的位移向量的准确性。我们证明了我们的方法和传统的数据无关的方法相比,对于在形状数据集中较少出现的形状,表现出更强的鲁棒性。

URL

https://arxiv.org/abs/2305.00635

PDF

https://arxiv.org/pdf/2305.00635.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot