Abstract
Human sketch has already proved its worth in various visual understanding tasks (e.g., retrieval, segmentation, image-captioning, etc). In this paper, we reveal a new trait of sketches - that they are also salient. This is intuitive as sketching is a natural attentive process at its core. More specifically, we aim to study how sketches can be used as a weak label to detect salient objects present in an image. To this end, we propose a novel method that emphasises on how "salient object" could be explained by hand-drawn sketches. To accomplish this, we introduce a photo-to-sketch generation model that aims to generate sequential sketch coordinates corresponding to a given visual photo through a 2D attention mechanism. Attention maps accumulated across the time steps give rise to salient regions in the process. Extensive quantitative and qualitative experiments prove our hypothesis and delineate how our sketch-based saliency detection model gives a competitive performance compared to the state-of-the-art.
Abstract (translated)
人类 Sketch 已经在各种视觉理解任务中证明了其价值(例如检索、分割、图像标题等)。在本文中,我们揭示了 Sketch 的新特征——它们也是引人注目的。这是因为 Sketch 本质上是一个自然的注意力过程的核心。更具体地说,我们旨在研究如何通过 Sketch 用作弱标签来检测图像中的引人注目物体。为此,我们提出了一种新方法,强调了如何通过手绘 Sketch 解释“引人注目的物体”。为了实现这一点,我们引入了一个照片到 Sketch 生成模型,旨在通过 2D 注意力机制生成与给定视觉照片相应的Sequential Sketch 坐标。注意力图在时间步上累积,从而产生引人注目的区域。广泛的定量和定性实验证明了我们的假设,并概述了我们 Sketch based 引人注目检测模型相对于最先进的技术提供的竞争性能。
URL
https://arxiv.org/abs/2303.11502
