Abstract
With wearing masks becoming a new cultural norm, facial expression recognition (FER) while taking masks into account has become a significant challenge. In this paper, we propose a unified multi-branch vision transformer for facial expression recognition and mask wearing classification tasks. Our approach extracts shared features for both tasks using a dual-branch architecture that obtains multi-scale feature representations. Furthermore, we propose a cross-task fusion phase that processes tokens for each task with separate branches, while exchanging information using a cross attention module. Our proposed framework reduces the overall complexity compared with using separate networks for both tasks by the simple yet effective cross-task fusion phase. Extensive experiments demonstrate that our proposed model performs better than or on par with different state-of-the-art methods on both facial expression recognition and facial mask wearing classification task.
Abstract (translated)
随着戴口罩成为一种新的文化规范，同时考虑戴口罩的情况下的面部表情识别（FER）已成为一个重要的挑战。在本文中，我们提出了一种统一的跨分支视觉Transformer，用于面部表情识别和戴口罩分类任务。我们的方法通过采用双分支架构提取共享特征，获得多尺度特征表示。此外，我们还提出了一种跨任务融合阶段，处理每个任务使用单独的分支，同时通过交叉注意模块交换信息。与使用单独网络处理两个任务相比，我们提出的框架通过简单的但有效的跨任务融合阶段减少了整体复杂性。 丰富的实验结果表明，与不同 state-of-the-art 方法相比，我们提出的模型在面部表情识别和戴口罩分类任务上都表现更好，或者处于同一水平。
URL
https://arxiv.org/abs/2404.14606
