Abstract
Sketches, with their expressive potential, allow humans to convey the essence of an object through even a rough contour. For the first time, we harness this expressive potential to improve segmentation performance in challenging tasks like camouflaged object detection (COD). Our approach introduces an innovative sketch-guided interactive segmentation framework, allowing users to intuitively annotate objects with freehand sketches (drawing a rough contour of the object) instead of the traditional bounding boxes or points used in classic interactive segmentation models like SAM. We demonstrate that sketch input can significantly improve performance in existing iterative segmentation methods, outperforming text or bounding box annotations. Additionally, we introduce key modifications to network architectures and a novel sketch augmentation technique to fully harness the power of sketch input and further boost segmentation accuracy. Remarkably, our model' s output can be directly used to train other neural networks, achieving results comparable to pixel-by-pixel annotations--while reducing annotation time by up to 120 times, which shows great potential in democratizing the annotation process and enabling model training with less reliance on resource-intensive, laborious pixel-level annotations. We also present KOSCamo+, the first freehand sketch dataset for camouflaged object detection. The dataset, code, and the labeling tool will be open sourced.
Abstract (translated)
草图，凭借其强大的表现力，允许人类通过粗糙的轮廓来传达一个物体的本质。首次，我们利用这一表现潜力以提升在具有挑战性的任务如伪装目标检测（COD）中的分割性能。我们的方法引入了一个创新的基于草图引导的交互式分割框架，该框架让用户能够用自由手绘的草图（绘制物体的大致轮廓），而不是经典交互式分割模型如SAM中使用的传统边界框或点，来进行直观地标注对象。 我们展示了草图输入可以显著提升现有迭代分割方法的表现，并且在现有的文本或边界框注释方式下表现更佳。此外，我们对网络架构进行了关键修改，并引入了一种新颖的草图增强技术，以充分利用草图输入并进一步提高分割准确性。令人印象深刻的是，我们的模型输出可以直接用于训练其他神经网络，在结果上媲美逐像素标注—同时将标注时间减少多达120倍，这显示了在注释过程中实现民主化以及使模型训练更少依赖于资源密集型、劳动密集的逐像素级标注方面具有巨大潜力。 此外，我们还发布了KOSCamo+，这是首个用于伪装目标检测的手绘草图数据集。该数据集、代码和标记工具将会开源提供。
URL
https://arxiv.org/abs/2501.19329
