Abstract
Missing values remain a common challenge for depth data across its wide range of applications, stemming from various causes like incomplete data acquisition and perspective alteration. This work bridges this gap with DepthLab, a foundation depth inpainting model powered by image diffusion priors. Our model features two notable strengths: (1) it demonstrates resilience to depth-deficient regions, providing reliable completion for both continuous areas and isolated points, and (2) it faithfully preserves scale consistency with the conditioned known depth when filling in missing values. Drawing on these advantages, our approach proves its worth in various downstream tasks, including 3D scene inpainting, text-to-3D scene generation, sparse-view reconstruction with DUST3R, and LiDAR depth completion, exceeding current solutions in both numerical performance and visual quality. Our project page with source code is available at this https URL.
Abstract (translated)
缺失值仍然是深度数据在广泛应用中常见的挑战，这些问题源于多种原因，如不完整数据采集和视角变化。本研究通过DepthLab填补了这一空白，DepthLab是一款基于图像扩散先验的基础深度插补模型。我们的模型具有两个显著优势：(1) 对于缺深度区域表现出强大的适应性，能够可靠地完成连续区域和孤立点的填充；(2) 在填充缺失值时，能忠实保持与已知条件深度的一致比例。凭借这些优势，我们的方法在多种下游任务中证明了其价值，包括3D场景插补、文本到3D场景生成、使用DUST3R进行稀疏视图重建以及LiDAR深度完成，在数值性能和视觉质量方面均超越现有解决方案。我们的项目页面（含源代码）可在以下链接获取：[此 https URL]。
URL
https://arxiv.org/abs/2412.18153
