Paper Reading AI Learner

Graph-based Inpainting for 3D Dynamic Point Clouds

2019-04-23 08:18:56
Zeqing Fu, Wei Hu, Zongming Guo

Abstract

With the development of depth sensors and 3D laser scanning techniques, 3D dynamic point clouds have attracted increasing attention as a format for the representation of 3D objects in motion, with applications in various fields such as 3D immersive tele-presence, navigation, animation, gaming and virtual reality. However, dynamic point clouds usually exhibit holes of missing data, mainly due to the fast motion, the limitation of acquisition techniques and complicated structure. Further, point clouds are defined on irregular non-Euclidean domain, which is challenging to address with conventional methods for regular data. Hence, leveraging on graph signal processing tools, we propose an efficient dynamic point cloud inpainting method, exploiting both the inter-frame coherence and the intra-frame self-similarity in 3D dynamic point clouds. Specifically, for each frame in a point cloud sequence, we first split it into cubes of fixed size as the processing unit, and treat cubes with holes inside as target cubes. Secondly, we take advantage of the intra-frame self-similarity in the target frame, by globally searching for the most similar cube to each target cube as the intra-source cube. Thirdly, we exploit the inter-frame coherence among every three consecutive frames, by searching the corresponding cubes in the previous and subsequent frames for each target cube as the inter-source cubes, which contains most nearest neighbors of the target cube in the relative location. Finally, we formulate dynamic point cloud inpainting as an optimization problem based on both intra- and inter-source cubes, which is regularized by the graph-signal smoothness prior. Experimental results show that the proposed approach outperforms three competing methods significantly, both in objective and subjective quality.

Abstract (translated)

随着深度传感器和三维激光扫描技术的发展,三维动态点云作为一种运动中三维物体的表示格式越来越受到人们的关注,其应用领域包括三维沉浸式远程存在、导航、动画、游戏和虚拟现实。然而,动态点云往往会出现数据丢失的空穴,主要是由于运动速度快、采集技术的局限和结构复杂等原因。此外,点云是在不规则非欧几里得域上定义的,用常规的规则数据方法很难解决这一问题。因此,利用图形信号处理工具,我们提出了一种有效的动态点云绘制方法,利用三维动态点云的帧间相干性和帧内自相似性。具体地说,对于点云序列中的每一帧,我们首先将其分割成固定大小的立方体作为处理单元,并将内部有孔的立方体作为目标立方体。其次,利用目标帧内部的自相似性,全局搜索每个目标立方体中最相似的立方体作为源内立方体。第三,我们利用每三个连续帧之间的帧间相干性,通过搜索每个目标立方体的前帧和后帧中对应的立方体作为源间立方体,其中包含相对位置上目标立方体的最接近邻居。最后,我们将动态点云绘制作为一个基于源内和源间多维数据集的优化问题,并利用图信号平滑先验规则化。实验结果表明,该方法在客观和主观两个方面均优于三种竞争方法。

URL

https://arxiv.org/abs/1904.10795

PDF

https://arxiv.org/pdf/1904.10795.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot