Abstract
Integrating an RGB camera into a ToF imaging system has become a significant technique for perceiving the real world. The RGB guided ToF imaging system is crucial to several applications, including face anti-spoofing, saliency detection, and trajectory prediction. Depending on the distance of the working range, the implementation schemes of the RGB guided ToF imaging systems are different. Specifically, ToF sensors with a uniform field of illumination, which can output dense depth but have low resolution, are typically used for close-range measurements. In contrast, LiDARs, which emit laser pulses and can only capture sparse depth, are usually employed for long-range detection. In the two cases, depth quality improvement for RGB guided ToF imaging corresponds to two sub-tasks: guided depth super-resolution and guided depth completion. In light of the recent significant boost to the field provided by deep learning, this paper comprehensively reviews the works related to RGB guided ToF imaging, including network structures, learning strategies, evaluation metrics, benchmark datasets, and objective functions. Besides, we present quantitative comparisons of state-of-the-art methods on widely used benchmark datasets. Finally, we discuss future trends and the challenges in real applications for further research.
Abstract (translated)
将RGB相机集成到ToF成像系统中已成为感知现实世界的重要技术。RGB引导ToF成像系统对于多个应用场景至关重要，包括面部抗伪造、轮廓检测和轨迹预测。根据工作范围的不同，RGB引导ToF成像系统的实现方案是不同的。具体来说，具有均匀场照明的ToF传感器通常用于近距离测量。相反，激光雷达，它们只能捕获稀疏深度，通常用于远距离检测。在这两种情况下，RGB引导ToF成像系统的深度质量改进相当于两个子任务：引导深度超分辨率 和引导深度完成。 鉴于最近深度学习在领域提供的重大提升，本文全面回顾了与RGB引导ToF成像相关的论文，包括网络结构、学习策略、评估指标、基准数据集和目标函数。此外，我们还在广泛使用的基准数据集上对最先进的方法进行了定量比较。最后，我们讨论了在实际应用中未来的趋势和挑战，为进一步研究提供了指导。
URL
https://arxiv.org/abs/2405.10357
