Abstract
This work addresses the task of completely weakly supervised class-incremental learning for semantic segmentation to learn segmentation for both base and additional novel classes using only image-level labels. While class-incremental semantic segmentation (CISS) is crucial for handling diverse and newly emerging objects in the real world, traditional CISS methods require expensive pixel-level annotations for training. To overcome this limitation, partially weakly-supervised approaches have recently been proposed. However, to the best of our knowledge, this is the first work to introduce a completely weakly-supervised method for CISS. To achieve this, we propose to generate robust pseudo-labels by combining pseudo-labels from a localizer and a sequence of foundation models based on their uncertainty. Moreover, to mitigate catastrophic forgetting, we introduce an exemplar-guided data augmentation method that generates diverse images containing both previous and novel classes with guidance. Finally, we conduct experiments in three common experimental settings: 15-5 VOC, 10-10 VOC, and COCO-to-VOC, and in two scenarios: disjoint and overlap. The experimental results demonstrate that our completely weakly supervised method outperforms even partially weakly supervised methods in the 15-5 VOC and 10-10 VOC settings while achieving competitive accuracy in the COCO-to-VOC setting.
Abstract (translated)
这项工作致力于完全弱监督的类增量学习在语义分割中的任务，利用仅有的图像级别标签来为基本和新增的类别进行语义分割。虽然类增量语义分割（CISS）对于处理现实世界中多样且不断出现的新物体至关重要，但传统的方法需要昂贵的像素级标注来进行训练。为了克服这一限制，最近提出了一些部分弱监督方法。然而，据我们所知，这是首次介绍了一种完全弱监督方法用于CISS。 为实现这一点，我们提议通过结合来自定位器和一系列基础模型的伪标签，并根据它们的不确定性进行选择来生成稳健的伪标签。此外，为了减轻灾难性遗忘问题，我们引入了一种基于样本引导的数据增强方法，该方法能生成包含先前类与新类别多样图像。 最后，我们在三种常见的实验设置下进行了测试：15-5 VOC、10-10 VOC 和 COCO-to-VOC，并且在两种场景中进行验证：离散和重叠。实验结果表明，在 15-5 VOC 和 10-10 VOC 设置下，我们的完全弱监督方法超越了部分弱监督的方法，同时在 COCO-to-VOC 设置中也取得了具有竞争力的准确性。
URL
https://arxiv.org/abs/2505.10781
