Abstract
Panoptic segmentation is one of the most challenging scene parsing tasks, combining the tasks of semantic segmentation and instance segmentation. While much progress has been made, few works focus on the real-time application of panoptic segmentation methods. In this paper, we revisit the recently introduced K-Net architecture. We propose vital changes to the architecture, training, and inference procedure, which massively decrease latency and improve performance. Our resulting RT-K-Net sets a new state-of-the-art performance for real-time panoptic segmentation methods on the Cityscapes dataset and shows promising results on the challenging Mapillary Vistas dataset. On Cityscapes, RT-K-Net reaches 60.2 % PQ with an average inference time of 32 ms for full resolution 1024x2048 pixel images on a single Titan RTX GPU. On Mapillary Vistas, RT-K-Net reaches 33.2 % PQ with an average inference time of 69 ms. Source code is available at this https URL.
Abstract (translated)
Panoptic segmentation 是处理场景解析任务中最具挑战性的之一，将语义分割和实例分割任务结合在一起。尽管已经取得了很多进展，但只有少数工作关注实时 Panoptic segmentation 方法的应用。在本文中，我们重新审视了最近引入的 K-Net 架构。我们提出了关键的变化，修改了架构、训练和推理程序，极大地减少了延迟并提高了性能。我们得到的 RT-K-Net 在 Cityscapes 数据集上实现了实时 Panoptic segmentation 方法的最新前沿技术性能，并在挑战性的 Mapillary Vistas 数据集上取得了令人期望的结果。在 Cityscapes 中，RT-K-Net 的 PQ 准确率达到了 60.2%，平均推理时间为 32 毫秒，对于一张 1024x2048 像素的全分辨率图像，可以在单个 Titan RTX 显卡上执行。在 Mapillary Vistas 中，RT-K-Net 的 PQ 准确率达到了 33.2%，平均推理时间为 69 毫秒。源代码可在 this https URL 获取。
URL
https://arxiv.org/abs/2305.01255
