Abstract
Facial expression recognition (FER) is a challenging task due to pervasive occlusion and dataset biases. Especially when facial information is partially occluded, existing FER models struggle to extract effective facial features, leading to inaccurate classifications. In response, we present ORSANet, which introduces the following three key contributions: First, we introduce auxiliary multi-modal semantic guidance to disambiguate facial occlusion and learn high-level semantic knowledge, which is two-fold: 1) we introduce semantic segmentation maps as dense semantics prior to generate semantics-enhanced facial representations; 2) we introduce facial landmarks as sparse geometric prior to mitigate intrinsic noises in FER, such as identity and gender biases. Second, to facilitate the effective incorporation of these two multi-modal priors, we customize a Multi-scale Cross-interaction Module (MCM) to adaptively fuse the landmark feature and semantics-enhanced representations within different scales. Third, we design a Dynamic Adversarial Repulsion Enhancement Loss (DARELoss) that dynamically adjusts the margins of ambiguous classes, further enhancing the model's ability to distinguish similar expressions. We further construct the first occlusion-oriented FER dataset to facilitate specialized robustness analysis on various real-world occlusion conditions, dubbed Occlu-FER. Extensive experiments on both public benchmarks and Occlu-FER demonstrate that our proposed ORSANet achieves SOTA recognition performance. Code is publicly available at this https URL.
Abstract (translated)
面部表情识别（FER）是一个具有挑战性的任务，由于普遍存在遮挡和数据集偏差的问题。特别是在脸部信息部分被遮挡的情况下，现有的面部表情识别模型难以提取有效的面部特征，导致分类不准确。为此，我们提出了ORSANet，并引入了以下三个关键贡献： 第一，我们引入了辅助多模态语义引导来区分面部遮挡并学习高层次的语义知识，具体来说： 1）我们将语义分割图作为稠密语义先验以生成增强语义的面部表示； 2）我们将面部特征点（landmarks）作为稀疏几何先验来缓解FER中的固有噪声问题，如身份和性别偏差。 第二，为了有效地整合这两种多模态先验，我们定制了一种多尺度交叉交互模块（MCM），该模块可以在不同尺度下自适应地融合特征点特征与增强语义的表示。 第三，我们设计了一种动态对抗排斥强化损失函数（DARELoss），该函数可以动态调整模糊类别之间的边界，进一步增强了模型区分相似表情的能力。此外，为了进行特定的鲁棒性分析，我们在各种真实世界遮挡条件下构建了首个面向遮挡的面部表情识别数据集，命名为Occlu-FER。 在公开基准测试和Occlu-FER上的广泛实验表明，我们提出的ORSANet达到了最先进的（SOTA）识别性能。代码可在提供的链接中找到。
URL
https://arxiv.org/abs/2507.15401
