Abstract
Infrared imaging plays a critical role in low-light and adverse weather conditions. However, due to the distinct characteristics of infrared images, existing foundation models such as Masked Autoencoder (MAE) trained on visible data perform suboptimal in infrared image interpretation tasks. To bridge this gap, an infrared foundation model known as InfMAE was developed and pre-trained on large-scale infrared datasets. Despite its effectiveness, InfMAE still faces several limitations, including the omission of informative tokens, insufficient modeling of global associations, and neglect of non-uniform noise. In this paper, we propose a Dual-domain Guided Infrared foundation model based on MAE (DuGI-MAE). First, we design a deterministic masking strategy based on token entropy, preserving only high-entropy tokens for reconstruction to enhance informativeness. Next, we introduce a Dual-Domain Guidance (DDG) module, which simultaneously captures global token relationships and adaptively filters non-uniform background noise commonly present in infrared imagery. To facilitate large-scale pretraining, we construct Inf-590K, a comprehensive infrared image dataset encompassing diverse scenes, various target types, and multiple spatial resolutions. Pretrained on Inf-590K, DuGI-MAE demonstrates strong generalization capabilities across various downstream tasks, including infrared object detection, semantic segmentation, and small target detection. Experimental results validate the superiority of the proposed method over both supervised and self-supervised comparison methods. Our code is available in the supplementary material.
Abstract (translated)
红外成像在低光和恶劣天气条件下发挥着关键作用。然而，由于红外图像具有独特的特性，现有的基于可见光数据训练的Masked Autoencoder (MAE)等基础模型，在处理红外图像解释任务时表现不佳。为了弥合这一差距，开发了一个名为InfMAE的红外基础模型，并在大规模红外数据集上进行了预训练。尽管效果显著，InfMAE仍然面临一些限制，包括信息性令牌缺失、全局关联建模不足以及忽视非均匀噪声的问题。 为此，在本文中我们提出了一种基于双域引导的Masked Autoencoder (MAE)红外基础模型（DuGI-MAE）。首先，我们设计了一个基于令牌熵的确定性掩码策略，仅保留高熵令牌进行重建以增强信息量。接下来，我们引入了双域指导模块（DDG），该模块同时捕捉全局令牌关系并自适应地过滤掉通常存在于红外图像中的非均匀背景噪声。 为了支持大规模预训练，我们构建了一个名为Inf-590K的全面红外图像数据集，涵盖了各种场景、不同类型的目标和多个空间分辨率。在Inf-590K上进行预训练后，DuGI-MAE在包括红外目标检测、语义分割以及小目标检测在内的多种下游任务中展示了强大的泛化能力。 实验结果验证了所提出方法相较于监督学习及自监督对比方法的优越性。我们的代码可在补充材料中获取。
URL
https://arxiv.org/abs/2512.04511
