Abstract
Recently, deep learning based facial landmark detection (FLD) methods have achieved considerable success. However, in challenging scenarios such as large pose variations, illumination changes, and facial expression variations, they still struggle to accurately capture the geometric structure of the face, resulting in performance degradation. Moreover, the limited size and diversity of existing FLD datasets hinder robust model training, leading to reduced detection accuracy. To address these challenges, we propose a Frequency-Guided Task-Balancing Transformer (FGTBT), which enhances facial structure perception through frequency-domain modeling and multi-dataset unified training. Specifically, we propose a novel Fine-Grained Multi-Task Balancing loss (FMB-loss), which moves beyond coarse task-level balancing by assigning weights to individual landmarks based on their occurrence across datasets. This enables more effective unified training and mitigates the issue of inconsistent gradient magnitudes. Additionally, a Frequency-Guided Structure-Aware (FGSA) model is designed to utilize frequency-guided structure injection and regularization to help learn facial structure constraints. Extensive experimental results on popular benchmark datasets demonstrate that the integration of the proposed FMB-loss and FGSA model into our FGTBT framework achieves performance comparable to state-of-the-art methods. The code is available at this https URL.
Abstract (translated)
最近，基于深度学习的面部标志检测（FLD）方法已经取得了显著的成功。然而，在诸如大幅度姿态变化、光照变化和面部表情变化等挑战性场景中，这些方法仍然难以准确捕捉面部的几何结构，从而导致性能下降。此外，现有FLD数据集的规模有限且多样性不足，阻碍了鲁棒模型的训练，并降低了检测精度。为了应对这些挑战，我们提出了一种频率引导的任务平衡变换器（FGTBT），该方法通过频域建模和多数据集统一训练来增强面部结构感知能力。 具体而言，我们提出了一个新颖的细粒度多任务平衡损失（FMB-loss），它超越了粗略的任务级平衡，并根据各个地标在不同数据集中出现的情况为它们分配权重。这使得统一训练更加有效，并减轻了一致性梯度大小问题。此外，设计了一个频率引导结构感知模型（FGSA），利用频率引导的结构注入和正则化来帮助学习面部结构约束。 广泛的实验结果表明，在流行的基准数据集上将所提出的FMB-loss和FGSA模型集成到我们的FGTBT框架中可以实现与当前最佳方法相当的性能。代码可在提供的链接处获取。
URL
https://arxiv.org/abs/2601.12863
