Abstract
Event-based cameras are bio-inspired vision sensors that asynchronously capture per-pixel intensity changes with microsecond latency, high temporal resolution, and high dynamic range, providing valuable information about the spatio-temporal dynamics of the scene. In the present work, we propose evTransFER, a transfer learning-based framework and architecture for face expression recognition using event-based cameras. The main contribution is a feature extractor designed to encode the spatio-temporal dynamics of faces, built by training an adversarial generative method on a different problem (facial reconstruction) and then transferring the trained encoder weights to the face expression recognition system. We show that this proposed transfer learning method greatly improves the ability to recognize facial expressions compared to training a network from scratch. In addition, we propose an architecture that incorporates an LSTM to capture longer-term facial expression dynamics, and we introduce a new event-based representation, referred to as TIE, both of which further improve the results. We evaluate the proposed framework on the event-based facial expression database e-CK+ and compare it to state-of-the-art methods. The results show that the proposed framework evTransFER achieves a 93.6\% recognition rate on the e-CK+ database, significantly improving the accuracy (25.9\% points or more) when compared to state-of-the-art performance for similar problems.
Abstract (translated)
基于事件的相机是受生物启发的视觉传感器，能够异步捕获像素级别的强度变化，并具有微秒级延迟、高时间分辨率和大动态范围的特点，为场景的空间-时间动力学提供了有价值的信息。在当前的工作中，我们提出了evTransFER框架与架构，用于基于事件的摄像机的人脸表情识别，采用了基于迁移学习的方法。主要贡献是一个特征提取器的设计，该设计能够编码人脸的空间-时间动态特性，通过在一个不同的问题（面部重建）上训练一种对抗性生成方法，并将训练好的编码器权重转移到人脸识别系统中来实现这一目标。我们展示了这种方法与从头开始训练网络相比，在识别面部表情的能力上有显著提升。 此外，还提出了一种结合LSTM的架构以捕捉更长期的人脸表情动态，并引入了一个新的基于事件表示的形式——TIE（Time-Indexed Event），这两种方法进一步提高了结果。我们在基于事件的表情数据库e-CK+上评估了所提出的框架，并将其与最先进的方法进行了比较。结果显示，我们的框架evTransFER在e-CK+数据库上的识别率达到93.6%，相较于同类问题中的最佳性能，在准确率上有显著的提高（25.9个百分点或更多）。
URL
https://arxiv.org/abs/2508.03609
