Abstract
Micro-expressions (MEs) are involuntary movements revealing people's hidden feelings, which has attracted numerous interests for its objectivity in emotion detection. However, despite its wide applications in various scenarios, micro-expression recognition (MER) remains a challenging problem in real life due to three reasons, including (i) data-level: lack of data and imbalanced classes, (ii) feature-level: subtle, rapid changing, and complex features of MEs, and (iii) decision-making-level: impact of individual differences. To address these issues, we propose a dual-branch meta-auxiliary learning method, called LightmanNet, for fast and robust micro-expression recognition. Specifically, LightmanNet learns general MER knowledge from limited data through a dual-branch bi-level optimization process: (i) In the first level, it obtains task-specific MER knowledge by learning in two branches, where the first branch is for learning MER features via primary MER tasks, while the other branch is for guiding the model obtain discriminative features via auxiliary tasks, i.e., image alignment between micro-expressions and macro-expressions since their resemblance in both spatial and temporal behavioral patterns. The two branches of learning jointly constrain the model of learning meaningful task-specific MER knowledge while avoiding learning noise or superficial connections between MEs and emotions that may damage its generalization ability. (ii) In the second level, LightmanNet further refines the learned task-specific knowledge, improving model generalization and efficiency. Extensive experiments on various benchmark datasets demonstrate the superior robustness and efficiency of LightmanNet.
Abstract (translated)
微表情（MEs）是指不经意的运动，揭示了人们隐藏的感受，其对于情感检测的客观性吸引了众多关注。然而，尽管它在各种场景中具有广泛的应用，但在现实生活中，微表情识别（MER）仍然是一个具有挑战性的问题，由于以下三个原因： 1. 数据层面：数据不足和数据不平衡； 2. 特征层面：微表情的微妙、快速变化和复杂特征； 3. 决策层面：个体差异的影响。 为了应对这些问题，我们提出了一个双分支元辅助学习方法，称为LightmanNet，用于快速且稳健的微表情识别。具体来说，LightmanNet通过双分支生物级优化过程从有限的数据中学习通用MER知识：（i）在第一层，它通过两个分支获得任务特定的MER知识，第一个分支通过学习主要MER任务中的MER特征来获得，而另一个分支则通过引导模型通过辅助任务获得具有区分性的特征，即通过它们在空间和时间行为模式中的相似性来获得。两个分支的学习共同约束了学习有意义的任务特定MER知识的同时，避免了学习噪声或浅层连接可能会损害其泛化能力的可能性。（ii）在第二层，LightmanNet进一步优化了已学习的任务特定知识，提高了模型的泛化能力和效率。在各种基准数据集上的广泛实验证明，LightmanNet具有卓越的稳健性和效率。
URL
https://arxiv.org/abs/2404.12024
