Abstract
Image fusion aims to integrate comprehensive information from images acquired through multiple sources. However, images captured by diverse sensors often encounter various degradations that can negatively affect fusion quality. Traditional fusion methods generally treat image enhancement and fusion as separate processes, overlooking the inherent correlation between them; notably, the dominant regions in one modality of a fused image often indicate areas where the other modality might benefit from enhancement. Inspired by this observation, we introduce the concept of dominant regions for image enhancement and present a Dynamic Relative EnhAnceMent framework for Image Fusion (Dream-IF). This framework quantifies the relative dominance of each modality across different layers and leverages this information to facilitate reciprocal cross-modal enhancement. By integrating the relative dominance derived from image fusion, our approach supports not only image restoration but also a broader range of image enhancement applications. Furthermore, we employ prompt-based encoding to capture degradation-specific details, which dynamically steer the restoration process and promote coordinated enhancement in both multi-modal image fusion and image enhancement scenarios. Extensive experimental results demonstrate that Dream-IF consistently outperforms its counterparts.
Abstract (translated)
图像融合的目标是整合通过多种来源获取的图像中的全面信息。然而，不同传感器捕获的图像经常会遇到各种退化问题，这些问题可能会影响融合质量。传统的方法通常将图像增强和融合视为两个独立的过程，忽视了它们之间的内在关联；特别地，在一幅融合图像中某一模式下的主导区域往往表明其他模式下该区域可能存在需要提升的地方。基于这一观察，我们引入了用于图像增强的主导区域的概念，并提出了一种名为动态相对增强框架（Dynamic Relative EnhAnceMent framework for Image Fusion，简称Dream-IF）的方法。此框架量化了不同层次上每一种模式相对于另一种模式的优势，并利用这些信息来促进相互之间的跨模态增强。通过将从图像融合中得出的相对优势融入其中，我们的方法不仅支持图像恢复，还能够广泛应用于其他图像增强应用场合。 此外，我们采用基于提示编码的方法捕捉特定退化的细节，这可以动态地引导恢复过程并推动多模式图像融合和图像增强场景中的协调性提升。广泛的实验结果表明，Dream-IF在各种对比测试中始终表现出色。
URL
https://arxiv.org/abs/2503.10109
