Abstract
Modern surveillance systems increasingly rely on multi-wavelength sensors and deep neural networks to recognize faces in infrared images captured at night. However, most facial recognition models are trained on visible light datasets, leading to substantial performance degradation on infrared inputs due to significant domain shifts. Early feature-based methods for infrared face recognition proved ineffective, prompting researchers to adopt generative approaches that convert infrared images into visible light images for improved recognition. This paradigm, known as Heterogeneous Face Recognition (HFR), faces challenges such as model and modality discrepancies, leading to distortion and feature loss in generated images. To address these limitations, this paper introduces a novel latent diffusion-based model designed to generate high-quality visible face images from thermal inputs while preserving critical identity features. A multi-attribute classifier is incorporated to extract key facial attributes from visible images, mitigating feature loss during infrared-to-visible image restoration. Additionally, we propose the Self-attn Mamba module, which enhances global modeling of cross-modal features and significantly improves inference speed. Experimental results on two benchmark datasets demonstrate the superiority of our approach, achieving state-of-the-art performance in both image quality and identity preservation.
Abstract (translated)
现代监控系统越来越多地依赖多波段传感器和深度神经网络来识别夜间捕捉的红外图像中的面部。然而，大多数面部识别模型是在可见光数据集上训练的，因此在处理红外输入时性能显著下降，这是由于领域偏移（domain shift）造成的。早期基于特征的方法对于红外人脸识别效果不佳，促使研究人员采用生成式方法将红外图像转换为可见光图像以提高识别率。这种范式被称为异质面部识别（Heterogeneous Face Recognition, HFR），它面临着模型和模式差异带来的挑战，导致在生成的图像中出现失真和特征丢失的问题。为了克服这些限制，本文介绍了一种基于潜在扩散的新模型，旨在从热输入生成高质量的可见人脸图像，并保留关键的身份特征。此外，我们还引入了一个多属性分类器来提取来自可见光图像的关键面部特征，从而在红外转可见光图像恢复过程中减少特征丢失。另外，我们提出了Self-attn Mamba模块，它增强了跨模态特征的整体建模能力，并显著提高了推理速度。在两个基准数据集上的实验结果证明了我们方法的优越性，在图像质量和身份保留方面均达到了当前最先进的性能水平。
URL
https://arxiv.org/abs/2512.21032
