Abstract
In this paper, we present a deep coupled framework to address the problem of matching sketch image against a gallery of mugshots. Face sketches have the essential in- formation about the spatial topology and geometric details of faces while missing some important facial attributes such as ethnicity, hair, eye, and skin color. We propose a cou- pled deep neural network architecture which utilizes facial attributes in order to improve the sketch-photo recognition performance. The proposed Attribute-Assisted Deep Con- volutional Neural Network (AADCNN) method exploits the facial attributes and leverages the loss functions from the facial attributes identification and face verification tasks in order to learn rich discriminative features in a common em- bedding subspace. The facial attribute identification task increases the inter-personal variations by pushing apart the embedded features extracted from individuals with differ- ent facial attributes, while the verification task reduces the intra-personal variations by pulling together all the fea- tures that are related to one person. The learned discrim- inative features can be well generalized to new identities not seen in the training data. The proposed architecture is able to make full use of the sketch and complementary fa- cial attribute information to train a deep model compared to the conventional sketch-photo recognition methods. Exten- sive experiments are performed on composite (E-PRIP) and semi-forensic (IIIT-D semi-forensic) datasets. The results show the superiority of our method compared to the state- of-the-art models in sketch-photo recognition algorithms
Abstract (translated)
在本文中,我们提出了一个深度耦合框架,以解决草图图像与面部照片相匹配的问题。面部草图具有关于面部的空间拓扑和几何细节的基本信息,同时缺少一些重要的面部属性,例如种族,头发,眼睛和肤色。我们提出了一种耦合深度神经网络架构,该架构利用面部属性来提高草图照片识别性能。所提出的属性辅助深度卷积神经网络(AADCNN)方法利用面部属性并利用面部属性识别和面部验证任务中的损失函数,以便在共同的嵌入子空间中学习丰富的判别特征。面部属性识别任务通过推开从具有不同面部属性的个体提取的嵌入特征来增加人际变化,而验证任务通过将所有与一个特征相关的特征拉在一起来减少个人内部变化。人。学习的辨别特征可以很好地推广到训练数据中未见的新身份。与传统的草图照片识别方法相比,所提出的架构能够充分利用草图和互补的特征属性信息来训练深度模型。在复合(E-PRIP)和半法医(IIIT-D半法医)数据集上进行了广泛的实验。结果表明,与草图 - 照片识别算法中的最新模型相比,我们的方法具有优越性
URL
https://arxiv.org/abs/1808.00059