Abstract
Micro-expression recognition (MER) is a challenging task due to the subtle and fleeting nature of micro-expressions. Traditional input modalities, such as Apex Frame, Optical Flow, and Dynamic Image, often fail to adequately capture these brief facial movements, resulting in suboptimal performance. In this study, we introduce the Micro-expression Spatio-Temporal Image (MESTI), a novel dynamic input modality that transforms a video sequence into a single image while preserving the essential characteristics of micro-movements. Additionally, we present the Micro-expression Gradient Attention Network (MEGANet), which incorporates a novel Gradient Attention block to enhance the extraction of fine-grained motion features from micro-expressions. By combining MESTI and MEGANet, we aim to establish a more effective approach to MER. Extensive experiments were conducted to evaluate the effectiveness of MESTI, comparing it with existing input modalities across three CNN architectures (VGG19, ResNet50, and EfficientNetB0). Moreover, we demonstrate that replacing the input of previously published MER networks with MESTI leads to consistent performance improvements. The performance of MEGANet, both with MESTI and Dynamic Image, is also evaluated, showing that our proposed network achieves state-of-the-art results on the CASMEII and SAMM datasets. The combination of MEGANet and MESTI achieves the highest accuracy reported to date, setting a new benchmark for micro-expression recognition. These findings underscore the potential of MESTI as a superior input modality and MEGANet as an advanced recognition network, paving the way for more effective MER systems in a variety of applications.
Abstract (translated)
微表情识别（MER）是一项具有挑战性的任务，原因在于微表情的微妙性和短暂性。传统的输入模式，如Apex Frame、光学流和动态图像，常常无法充分捕捉这些短暂的面部运动，从而导致性能不佳。在这项研究中，我们引入了一种新的动态输入模式——微表情时空图（MESTI），该模式能够将视频序列转换为单张图片，并保留了细微动作的关键特性。此外，我们还提出了微表情梯度注意力网络（MEGANet），它包含一个新颖的梯度注意力块来增强从微表情中提取细粒度运动特征的能力。 通过结合MESTI和MEGANet，我们的目标是建立一种更有效的MER方法。为了评估MESTI的有效性，我们在三种CNN架构（VGG19、ResNet50和EfficientNetB0）上进行了广泛的实验，并将其与现有的输入模式进行了比较。此外，我们还展示了用MESTI替换先前发布的MER网络的输入会导致性能的一致提高。我们也对MEGANet在使用MESTI和动态图像时的表现进行了评估，结果表明我们的提议网络在CASMEII和SAMM数据集上实现了最先进的成果。 结合了MEGANet与MESTI的方法取得了迄今为止报告的最高准确率，并为微表情识别设定了新的基准。这些发现强调了MESTI作为更优输入模式以及MEGANet作为高级别识别网络的潜力，为各种应用场景中的MER系统铺平了道路。
URL
https://arxiv.org/abs/2509.00056
