Abstract
Depth completion plays a vital role in 3D perception systems, especially in scenarios where sparse depth data must be densified for tasks such as autonomous driving, robotics, and augmented reality. While many existing approaches rely on semantic segmentation to guide depth completion, they often overlook the benefits of object-level understanding. In this work, we introduce an instance-aware depth completion framework that explicitly integrates binary instance masks as spatial priors to refine depth predictions. Our model combines four main components: a frozen YOLO V11 instance segmentation branch, a U-Net-based depth completion backbone, a cross-attention fusion module, and an attention-guided prediction head. The instance segmentation branch generates per-image foreground masks that guide the depth branch via cross-attention, allowing the network to focus on object-centric regions during refinement. We validate our method on the Virtual KITTI 2 dataset, showing that it achieves lower RMSE compared to both a U-Net-only baseline and previous semantic-guided methods, while maintaining competitive MAE. Qualitative and quantitative results demonstrate that the proposed model effectively enhances depth accuracy near object boundaries, occlusions, and thin structures. Our findings suggest that incorporating instance-aware cues offers a promising direction for improving depth completion without relying on dense semantic labels.
Abstract (translated)
深度完成在3D感知系统中扮演着至关重要的角色，特别是在需要将稀疏的深度数据密集化的场景下，例如自动驾驶、机器人技术和增强现实。尽管许多现有的方法依赖于语义分割来指导深度完成任务，但它们往往忽略了对象级别理解的好处。在这项工作中，我们介绍了一种实例感知的深度完成框架，该框架明确地整合了二值实例掩模作为空间先验，以细化深度预测。我们的模型结合了四个主要组件：冻结版YOLO V11实例分割分支、基于U-Net的深度完成骨干网络、跨注意力融合模块和注意力引导的预测头。实例分割分支生成每张图像的前景掩模，并通过跨注意力机制指导深度分支，使网络在细化过程中能够专注于对象中心区域。 我们在虚拟KITTI 2数据集上验证了我们的方法，结果显示与仅基于U-Net的方法以及之前的语义导向方法相比，其均方根误差（RMSE）更低，同时保持了竞争性的平均绝对误差（MAE）。定性和定量结果表明，所提出的模型能够有效提升对象边界、遮挡和细长结构附近的深度精度。我们的研究结果表明，在不依赖密集的语义标签的情况下，融入实例感知线索为提高深度完成性能提供了一条有前景的方向。
URL
https://arxiv.org/abs/2512.04734
