Abstract
RGB-D scene parsing methods effectively capture both semantic and geometric features of the environment, demonstrating great potential under challenging conditions such as extreme weather and low lighting. However, existing RGB-D scene parsing methods predominantly rely on supervised training strategies, which require a large amount of manually annotated pixel-level labels that are both time-consuming and costly. To overcome these limitations, we introduce DepthMatch, a semi-supervised learning framework that is specifically designed for RGB-D scene parsing. To make full use of unlabeled data, we propose complementary patch mix-up augmentation to explore the latent relationships between texture and spatial features in RGB-D image pairs. We also design a lightweight spatial prior injector to replace traditional complex fusion modules, improving the efficiency of heterogeneous feature fusion. Furthermore, we introduce depth-guided boundary loss to enhance the model's boundary prediction capabilities. Experimental results demonstrate that DepthMatch exhibits high applicability in both indoor and outdoor scenes, achieving state-of-the-art results on the NYUv2 dataset and ranking first on the KITTI Semantics benchmark.
Abstract (translated)
RGB-D场景解析方法能够有效捕捉环境的语义和几何特征，在极端天气和低光照等挑战性条件下表现出巨大潜力。然而，现有的RGB-D场景解析方法主要依赖于监督学习策略，需要大量的手动像素级标签，这些标签既耗时又昂贵。为克服这些限制，我们引入了DepthMatch，这是一种专门为RGB-D场景解析设计的半监督学习框架。为了充分利用未标注的数据，我们提出了互补补丁混合增强技术，以探索RGB-D图像对中纹理和空间特征之间的潜在关系。此外，我们还设计了一种轻量级的空间先验注入器来替代传统的复杂融合模块，提高了异构特征融合的效率。同时，我们引入了深度引导边界损失，以提高模型的边界预测能力。 实验结果表明，DepthMatch在室内和室外场景中都具有很高的适用性，在NYUv2数据集上取得了最先进的成果，并且在KITTI Semantics基准测试中排名第一。
URL
https://arxiv.org/abs/2505.20041
