Abstract
With the development of depth sensors and 3D laser scanning techniques, 3D dynamic point clouds have attracted increasing attention as a format for the representation of 3D objects in motion, with applications in various fields such as 3D immersive tele-presence, navigation, animation, gaming and virtual reality. However, dynamic point clouds usually exhibit holes of missing data, mainly due to the fast motion, the limitation of acquisition techniques and complicated structure. Further, point clouds are defined on irregular non-Euclidean domain, which is challenging to address with conventional methods for regular data. Hence, leveraging on graph signal processing tools, we propose an efficient dynamic point cloud inpainting method, exploiting both the inter-frame coherence and the intra-frame self-similarity in 3D dynamic point clouds. Specifically, for each frame in a point cloud sequence, we first split it into cubes of fixed size as the processing unit, and treat cubes with holes inside as target cubes. Secondly, we take advantage of the intra-frame self-similarity in the target frame, by globally searching for the most similar cube to each target cube as the intra-source cube. Thirdly, we exploit the inter-frame coherence among every three consecutive frames, by searching the corresponding cubes in the previous and subsequent frames for each target cube as the inter-source cubes, which contains most nearest neighbors of the target cube in the relative location. Finally, we formulate dynamic point cloud inpainting as an optimization problem based on both intra- and inter-source cubes, which is regularized by the graph-signal smoothness prior. Experimental results show that the proposed approach outperforms three competing methods significantly, both in objective and subjective quality.
Abstract (translated)
随着深度传感器和三维激光扫描技术的发展,三维动态点云作为一种运动中三维物体的表示格式越来越受到人们的关注,其应用领域包括三维沉浸式远程存在、导航、动画、游戏和虚拟现实。然而,动态点云往往会出现数据丢失的空穴,主要是由于运动速度快、采集技术的局限和结构复杂等原因。此外,点云是在不规则非欧几里得域上定义的,用常规的规则数据方法很难解决这一问题。因此,利用图形信号处理工具,我们提出了一种有效的动态点云绘制方法,利用三维动态点云的帧间相干性和帧内自相似性。具体地说,对于点云序列中的每一帧,我们首先将其分割成固定大小的立方体作为处理单元,并将内部有孔的立方体作为目标立方体。其次,利用目标帧内部的自相似性,全局搜索每个目标立方体中最相似的立方体作为源内立方体。第三,我们利用每三个连续帧之间的帧间相干性,通过搜索每个目标立方体的前帧和后帧中对应的立方体作为源间立方体,其中包含相对位置上目标立方体的最接近邻居。最后,我们将动态点云绘制作为一个基于源内和源间多维数据集的优化问题,并利用图信号平滑先验规则化。实验结果表明,该方法在客观和主观两个方面均优于三种竞争方法。
URL
https://arxiv.org/abs/1904.10795