Abstract
Deep learning methods have led to significant improvements in the performance on the facial landmark detection (FLD) task. However, detecting landmarks in challenging settings, such as head pose changes, exaggerated expressions, or uneven illumination, continue to remain a challenge due to high variability and insufficient samples. This inadequacy can be attributed to the model's inability to effectively acquire appropriate facial structure information from the input images. To address this, we propose a novel image augmentation technique specifically designed for the FLD task to enhance the model's understanding of facial structures. To effectively utilize the newly proposed augmentation technique, we employ a Siamese architecture-based training mechanism with a Deep Canonical Correlation Analysis (DCCA)-based loss to achieve collective learning of high-level feature representations from two different views of the input images. Furthermore, we employ a Transformer + CNN-based network with a custom hourglass module as the robust backbone for the Siamese framework. Extensive experiments show that our approach outperforms multiple state-of-the-art approaches across various benchmark datasets.
Abstract (translated)
深度学习方法已经在面部关键点检测（FLD）任务上取得了显著的改进。然而，在具有挑战性的环境中检测关键点，例如头部姿态变化、夸张表情或不均匀照明，仍然是一个挑战，因为存在高度的变异性不足的样本。这种不足可以归因于模型无法有效地从输入图像中获取适当的面部结构信息。为了应对这个问题，我们提出了一个专门针对FLD任务设计的图像增强技术，以增强模型对面部结构的认知。为了有效地利用新提出的增强技术，我们采用了一种基于Siamese网络架构的训练方法，结合了基于深度卷积分析（DCCA）的损失，以实现从输入图像的两个不同视角进行集体学习。此外，我们还使用了一个自定义的时钟模块的Transformer + CNN网络作为Siamese框架的 robust 骨干网络。大量实验证明，我们的方法在各种基准数据集上超过了多个最先进的方法的性能。
URL
https://arxiv.org/abs/2402.15044
