Abstract
Remote Photoplethysmography (rPPG) is a non-contact technique for extracting physiological signals from facial videos, used in applications like emotion monitoring, medical assistance, and anti-face spoofing. Unlike controlled laboratory settings, real-world environments often contain motion artifacts and noise, affecting the performance of existing methods. To address this, we propose PhysMamba, a dual-stream time-frequency interactive model based on Mamba. PhysMamba integrates the state-of-the-art Mamba-2 model and employs a dual-stream architecture to learn diverse rPPG features, enhancing robustness in noisy conditions. Additionally, we designed the Cross-Attention State Space Duality (CASSD) module to improve information exchange and feature complementarity between the two streams. We validated PhysMamba using PURE, UBFC-rPPG and MMPD. Experimental results show that PhysMamba achieves state-of-the-art performance across various scenarios, particularly in complex environments, demonstrating its potential in practical remote heart rate monitoring applications.
Abstract (translated)
远距离心率测量（rPPG）是一种非接触技术，用于从面部视频中提取生理信号，应用于情感监测、医疗协助和反伪造应用。与实验室控制环境不同，现实世界环境通常包含运动伪迹和噪声，影响现有方法的性能。为了解决这个问题，我们提出了 PhysMamba，一种基于 Mamba 的双流时频交互模型。 PhysMamba 集成了最先进的 Mamba-2 模型，并采用双流架构学习多样 rPPG 特征，提高了在噪音环境中的鲁棒性。此外，我们还设计了 Cross-Attention State Space Duality（CASSD）模块，以提高两个流之间的信息交流和特征互补。我们使用 PURE、UBFC-rPPG 和 MMPD 对 PhysMamba 进行验证。实验结果表明，PhysMamba 在各种场景中均取得了与最新技术相当的表现，尤其是在复杂环境中，这表明其在实际远程心率监测应用中的巨大潜力。
URL
https://arxiv.org/abs/2408.01077
