Abstract
Scribble-based weakly supervised semantic segmentation leverages only a few annotated pixels as labels to train a segmentation model, presenting significant potential for reducing the human labor involved in the annotation process. This approach faces two primary challenges: first, the sparsity of scribble annotations can lead to inconsistent predictions due to limited supervision; second, the variability in scribble annotations, reflecting differing human annotator preferences, can prevent the model from consistently capturing the discriminative regions of objects, potentially leading to unstable predictions. To address these issues, we propose a holistic framework, the class-driven scribble promotion network, for robust scribble-supervised semantic segmentation. This framework not only utilizes the provided scribble annotations but also leverages their associated class labels to generate reliable pseudo-labels. Within the network, we introduce a localization rectification module to mitigate noisy labels and a distance perception module to identify reliable regions surrounding scribble annotations and pseudo-labels. In addition, we introduce new large-scale benchmarks, ScribbleCOCO and ScribbleCityscapes, accompanied by a scribble simulation algorithm that enables evaluation across varying scribble styles. Our method demonstrates competitive performance in both accuracy and robustness, underscoring its superiority over existing approaches. The datasets and the codes will be made publicly available.
Abstract (translated)
基于涂鸦的弱监督语义分割方法仅利用少量标注像素作为标签来训练分割模型，从而大大减少了人工标注过程中的劳动量。然而，这种方法面临两大挑战：首先，由于涂鸦标注稀疏导致有限的指导性，可能会产生不一致的预测；其次，不同的人工标注者偏好可能导致涂鸦标注的变化，这可能使模型难以稳定地捕捉到物体的判别区域，进而造成不稳定预测。为了解决这些问题，我们提出了一种全面框架——类别驱动涂鸦推广网络，用于鲁棒的基于涂鸦标签的语义分割。该框架不仅利用给定的涂鸦注释，还通过它们关联的类标签生成可靠的伪标签。在网络内部，我们引入了一个定位修正模块来减少噪声标签的影响，并设计了距离感知模块以识别围绕涂鸦标注和伪标签的可靠区域。此外，我们提出了新的大规模基准测试数据集——ScribbleCOCO 和 ScribbleCityscapes，并附带了一种涂鸦模拟算法，能够在不同风格的涂鸦下进行评估。我们的方法在准确性和鲁棒性方面均表现出色，证明了其相对于现有方法的优势。我们将公开发布这些数据集和代码。
URL
https://arxiv.org/abs/2503.13895
