Abstract
Thermal imaging plays a crucial role in various applications, but the inherent low resolution of commonly available infrared (IR) cameras limits its effectiveness. Conventional super-resolution (SR) methods often struggle with thermal images due to their lack of high-frequency details. Guided SR leverages information from a high-resolution image, typically in the visible spectrum, to enhance the reconstruction of a high-res IR image from the low-res input. Inspired by SwinFusion, we propose SwinFuSR, a guided SR architecture based on Swin transformers. In real world scenarios, however, the guiding modality (e.g. RBG image) may be missing, so we propose a training method that improves the robustness of the model in this case. Our method has few parameters and outperforms state of the art models in terms of Peak Signal to Noise Ratio (PSNR) and Structural SIMilarity (SSIM). In Track 2 of the PBVS 2024 Thermal Image Super-Resolution Challenge, it achieves 3rd place in the PSNR metric. Our code and pretained weights are available at this https URL.
Abstract (translated)
热成像在各种应用中扮演着关键角色，但通常可用的红外（IR）相机固有的低分辨率限制了其效果。传统的超分辨率（SR）方法往往由于其缺乏高频细节，在热图像上表现不佳。引导SR利用高分辨率图像上的信息，通常在可见光谱范围内，增强低分辨率输入的热红外图像的重建。受到SwinFusion的启发，我们提出了SwinFuSR，一种基于Swin变换器的引导SR架构。然而，在现实世界的场景中，引导模式（例如RGB图像）可能缺失，因此我们提出了一种改进模型的方法，以提高其在这种情况下的一致性。我们的方法具有很少的参数，并且在PSNR和结构相似性（SSIM）方面优于最先进的模型。在2024年PBVS thermal image super-resolution challenge的跟踪2中，它在PSNR指标上获得了第3名。我们的代码和预训练权重可在此https:// URL上找到。
URL
https://arxiv.org/abs/2404.14533
