Abstract
Modelling human free-hand sketches has become topical recently, driven by practical applications such as fine-grained sketch based image retrieval (FG-SBIR). Sketches are clearly related to photo edge-maps, but a human free-hand sketch of a photo is not simply a clean rendering of that photo's edge map. Instead there is a fundamental process of abstraction and iconic rendering, where overall geometry is warped and salient details are selectively included. In this paper we study this sketching process and attempt to invert it. We model this inversion by translating iconic free-hand sketches to contours that resemble more geometrically realistic projections of object boundaries, and separately factorise out the salient added details. This factorised re-representation makes it easier to match a free-hand sketch to a photo instance of an object. Specifically, we propose a novel unsupervised image style transfer model based on enforcing a cyclic embedding consistency constraint. A deep FG-SBIR model is then formulated to accommodate complementary discriminative detail from each factorised sketch for better matching with the corresponding photo. Our method is evaluated both qualitatively and quantitatively to demonstrate its superiority over a number of state-of-the-art alternatives for style transfer and FG-SBIR.
Abstract (translated)
最近，由于基于细粒度草图的图像检索（FG-SBIR）等实际应用，人体徒手草图的建模已成为热门话题。草图与照片边缘图明显相关，但是照片的人手绘草图并不仅仅是该照片边缘图的干净渲染。相反，有一个抽象和标志性渲染的基本过程，其中整体几何体被扭曲，并且有选择地包括显着的细节。在本文中，我们研究了这个草图绘制过程并试图将其反转。我们通过将标志性的自由手草图转换为类似于对象边界的更几何逼真投影的轮廓来模拟这种反演，并且单独地分解出显着增加的细节。这种分解的重新表示使得将自由手绘草图与对象的照片实例相匹配变得更容易。具体地，我们提出了一种基于实施循环嵌入一致性约束的新型无监督图像样式传递模型。然后制定深度FG-SBIR模型以适应来自每个分解草图的互补辨别细节，以更好地与相应照片匹配。我们的方法在质量和数量上进行了评估，以证明其优于大多数最先进的风格转移和FG-SBIR替代方案。
URL
https://arxiv.org/abs/1808.02313
