Abstract
Optics-guided thermal UAV image super-resolution has attracted significant research interest due to its potential in all-weather monitoring applications. However, existing methods typically compress optical features to match thermal feature dimensions for cross-modal alignment and fusion, which not only causes the loss of high-frequency information that is beneficial for thermal super-resolution, but also introduces physically inconsistent artifacts such as texture distortions and edge blurring by overlooking differences in the imaging physics between modalities. To address these challenges, we propose PCNet to achieve cross-resolution mutual enhancement between optical and thermal modalities, while physically constraining the optical guidance process via thermal conduction to enable robust thermal UAV image super-resolution. In particular, we design a Cross-Resolution Mutual Enhancement Module (CRME) to jointly optimize thermal image super-resolution and optical-to-thermal modality conversion, facilitating effective bidirectional feature interaction across resolutions while preserving high-frequency optical priors. Moreover, we propose a Physics-Driven Thermal Conduction Module (PDTM) that incorporates two-dimensional heat conduction into optical guidance, modeling spatially-varying heat conduction properties to prevent inconsistent artifacts. In addition, we introduce a temperature consistency loss that enforces regional distribution consistency and boundary gradient smoothness to ensure generated thermal images align with real-world thermal radiation principles. Extensive experiments on VGTSR2.0 and DroneVehicle datasets demonstrate that PCNet significantly outperforms state-of-the-art methods on both reconstruction quality and downstream tasks including semantic segmentation and object detection.
Abstract (translated)
基于光学引导的热气球无人机图像超分辨率技术因其在全天候监测应用中的潜力而引起了广泛的研究兴趣。然而，现有的方法通常通过压缩光学特征以匹配热特征维度来进行跨模态对齐和融合，这不仅导致了有益于热超分辨率的高频信息丢失，而且还因忽视了不同成像物理之间的差异而导致诸如纹理扭曲和边缘模糊等物理不一致的现象。 为了解决这些挑战，我们提出了PCNet（Physics-Driven Cross-modal Network），它在光学与热模态之间实现了跨分辨率相互增强，并通过基于热传导的物理约束来强化光学引导过程，从而能够实现鲁棒性的热气球无人机图像超分辨率。具体来说，我们设计了跨分辨率互增强模块（CRME），以共同优化热图的超分辨率和从光学到热的模态转换，促进不同分辨率间的有效双向特征交互，并保持高频光学先验信息。 此外，我们还提出了物理驱动的热传导模块（PDTM），该模块将二维热传导融入光学引导过程中，通过建模空间变化的热传导特性来防止不一致的现象。同时，我们引入了温度一致性损失，确保生成的热图在区域分布和边界梯度平滑性方面与现实世界的热辐射原理保持一致。 我们在VGTSR2.0和DroneVehicle数据集上的广泛实验表明，PCNet在重建质量和包括语义分割和目标检测在内的下游任务上都显著优于现有最先进的方法。
URL
https://arxiv.org/abs/2601.03526
