Abstract
Semantic segmentation is a fundamental problem in computer vision and it requires high-resolution feature maps for dense prediction. Current coordinate-guided low-resolution feature interpolation methods, e.g., bilinear interpolation, produce coarse high-resolution features which suffer from feature misalignment and insufficient context information. Moreover, enriching semantics to high-resolution features requires a high computation burden, so that it is challenging to meet the requirement of lowlatency inference. We propose a novel Guided Attentive Interpolation (GAI) method to adaptively interpolate fine-grained high-resolution features with semantic features to tackle these issues. Guided Attentive Interpolation determines both spatial and semantic relations of pixels from features of different resolutions and then leverages these relations to interpolate high-resolution features with rich semantics. GAI can be integrated with any deep convolutional network for efficient semantic segmentation. In experiments, the GAI-based semantic segmentation networks, i.e., GAIN, can achieve78.8 mIoU with 22.3 FPS on Cityscapes and 80.6 mIoU with 64.5 on CamVid using an NVIDIA 1080Ti GPU, which are the new state-of-the-art results of low-latency semantic segmentation. Code and models are available at: this https URL.
Abstract (translated)
语义分割是计算机视觉中的一个基本问题，它需要高分辨率的特征图来进行密集预测。当前的坐标引导低分辨率特征插值方法（如双线性插值），会产生粗略的高分辨率特征，这些特征存在特征错位和上下文信息不足的问题。此外，增强高分辨率特征的语义信息会带来较高的计算负担，从而难以满足低延迟推理的需求。为此，我们提出了一种新颖的引导注意力插值（Guided Attentive Interpolation, GAI）方法，该方法能够自适应地将精细粒度的高分辨率特征与语义特征相结合来解决这些问题。引导注意力插值确定了不同分辨率特征图中像素的空间和语义关系，并利用这些关系插值生成包含丰富语义信息的高分辨率特征。GAI可以与任何深度卷积网络结合使用，以实现高效的语义分割。在实验中，基于GAI的语义分割网络（即GAIN）在Cityscapes数据集上达到了78.8 mIoU和22.3 FPS，在CamVid数据集上达到了80.6 mIoU和64.5 FPS的结果，使用NVIDIA 1080Ti GPU进行测试。这些结果代表了低延迟语义分割的最新技术水平。 代码及模型可以在以下链接获取：this https URL.
URL
https://arxiv.org/abs/2601.01167
