Abstract
Scene understanding plays a critical role in enabling intelligence and autonomy in robotic systems. Traditional approaches often face challenges, including occlusions, ambiguous boundaries, and the inability to adapt attention based on task-specific requirements and sample variations. To address these limitations, this paper presents an efficient RGB-D scene understanding model that performs a range of tasks, including semantic segmentation, instance segmentation, orientation estimation, panoptic segmentation, and scene classification. The proposed model incorporates an enhanced fusion encoder, which effectively leverages redundant information from both RGB and depth inputs. For semantic segmentation, we introduce normalized focus channel layers and a context feature interaction layer, designed to mitigate issues such as shallow feature misguidance and insufficient local-global feature representation. The instance segmentation task benefits from a non-bottleneck 1D structure, which achieves superior contour representation with fewer parameters. Additionally, we propose a multi-task adaptive loss function that dynamically adjusts the learning strategy for different tasks based on scene variations. Extensive experiments on the NYUv2, SUN RGB-D, and Cityscapes datasets demonstrate that our approach outperforms existing methods in both segmentation accuracy and processing speed.
Abstract (translated)
场景理解在增强机器人系统的智能和自主性方面扮演着关键角色。传统方法经常面临遮挡、边界模糊以及无法根据任务特定需求和样本变化调整注意力等问题的挑战。为了克服这些限制，本文提出了一种高效的RGB-D场景理解模型，该模型能够执行一系列任务，包括语义分割、实例分割、姿态估计、全景分割和场景分类。所提出的模型包含了一个增强融合编码器，可以有效利用来自RGB和深度输入数据的冗余信息。 对于语义分割任务，我们引入了标准化关注通道层和上下文特征交互层，旨在解决浅层特征误导以及局部-全局特征表示不足等问题。在实例分割任务中，我们的模型采用了一种非瓶颈1D结构，在减少参数的同时实现了更优的轮廓表示。此外，我们还提出了一种多任务自适应损失函数，可以根据场景变化动态调整不同的学习策略。 通过在NYUv2、SUN RGB-D和Cityscapes数据集上的大量实验验证，我们的方法不仅在分割准确性上超越了现有技术，而且在处理速度方面也表现出色。
URL
https://arxiv.org/abs/2603.07570
