Abstract
Weakly Incremental Learning for Semantic Segmentation (WILSS) leverages a pre-trained segmentation model to segment new classes using cost-effective and readily available image-level labels. A prevailing way to solve WILSS is the generation of seed areas for each new class, serving as a form of pixel-level supervision. However, a scenario usually arises where a pixel is concurrently predicted as an old class by the pre-trained segmentation model and a new class by the seed areas. Such a scenario becomes particularly problematic in WILSS, as the lack of pixel-level annotations on new classes makes it intractable to ascertain whether the pixel pertains to the new class or not. To surmount this issue, we propose an innovative, tendency-driven relationship of mutual exclusivity, meticulously tailored to govern the behavior of the seed areas and the predictions generated by the pre-trained segmentation model. This relationship stipulates that predictions for the new and old classes must not conflict whilst prioritizing the preservation of predictions for the old classes, which not only addresses the conflicting prediction issue but also effectively mitigates the inherent challenge of incremental learning - catastrophic forgetting. Furthermore, under the auspices of this tendency-driven mutual exclusivity relationship, we generate pseudo masks for the new classes, allowing for concurrent execution with model parameter updating via the resolution of a bi-level optimization problem. Extensive experiments substantiate the effectiveness of our framework, resulting in the establishment of new benchmarks and paving the way for further research in this field.
Abstract (translated)
我们的研究"Weakly Incremental Learning for Semantic Segmentation (WILSS)"利用预训练的分割模型对新的类别进行分割，使用成本效益高且易得的开源图像级标签进行有效的分割。解决WILSS的一种方法是为新每个类别生成种子区域，作为一种像素级别的监督。然而，在WILSS中，预训练的分割模型预测像素为旧类和新类的情况通常会发生。这种情况在WILSS中变得尤为严重，因为新类缺乏像素级别的注释，因此无法确定像素是否属于新类。为了克服这个问题，我们提出了一个创新的分歧驱动关系，精心设计以管理种子区域和预训练分割模型生成的预测的行为。该关系规定，新旧类的预测不能冲突，同时优先考虑保留旧类的预测，这不仅解决了冲突预测问题，还有效地缓解了逐步学习固有的挑战 - 灾难性遗忘。此外，在分歧驱动 mutual exclusivity关系的帮助下，我们生成新类的伪掩码，使得通过解决双层优化问题对模型参数进行更新时，可以实现同时执行。大量实验证实了我们在该领域的有效性和创新性，从而为该领域建立了新的基准，并为进一步研究铺平道路。
URL
https://arxiv.org/abs/2404.11981
