Image_Enhancement
-
Towards Scale-Aware Low-Light Enhancement via Structure-Guided Transformer Design
2025-04-18 20:57:16Wei Dong, Yan Min, Han Zhou, Jun ChenAbstract
Current Low-light Image Enhancement (LLIE) techniques predominantly rely on either direct Low-Light (LL) to Normal-Light (NL) mappings or guidance from semantic features or illumination maps. Nonetheless, the intrinsic ill-posedness of LLIE and the difficulty in retrieving robust semantics from heavily corrupted images hinder their effectiveness in extremely low-light environments. To tackle this challenge, we present SG-LLIE, a new multi-scale CNN-Transformer hybrid framework guided by structure priors. Different from employing pre-trained models for the extraction of semantics or illumination maps, we choose to extract robust structure priors based on illumination-invariant edge detectors. Moreover, we develop a CNN-Transformer Hybrid Structure-Guided Feature Extractor (HSGFE) module at each scale with in the UNet encoder-decoder architecture. Besides the CNN blocks which excels in multi-scale feature extraction and fusion, we introduce a Structure-Guided Transformer Block (SGTB) in each HSGFE that incorporates structural priors to modulate the enhancement process. Extensive experiments show that our method achieves state-of-the-art performance on several LLIE benchmarks in both quantitative metrics and visual quality. Our solution ranks second in the NTIRE 2025 Low-Light Enhancement Challenge. Code is released at this https URL.
Abstract (translated)
目前的低光图像增强（LLIE）技术主要依赖于直接从低光照（LL）到正常光照（NL）的映射，或者通过语义特征或照明图进行指导。然而，LLIE的基本问题和从中提取稳健语义的困难使得这些方法在极端低光环境中效果不佳。 为了解决这一挑战，我们提出了SG-LLIE，这是一种新的多尺度CNN-Transformer混合框架，由结构先验引导。与使用预训练模型来提取语义或光照图不同，我们选择基于不变于光照的边缘检测器来提取稳健的结构先验信息。此外，在U-Net编码器-解码器架构中的每一层级上，我们都开发了一种称为CNN-Transformer混合结构指导特征提取器（HSGFE）的新模块。 除了擅长多尺度特征提取和融合的CNN块之外，我们还在每个HSGFE中引入了结构导向变换器块（SGTB），它将结构先验信息融入到增强过程中以调节图像增强过程。广泛的实验表明，我们的方法在多个LLIE基准测试中，在定量指标和视觉质量方面均达到了最先进的性能。 我们在NTIRE 2025低光增强挑战赛中排名第二。代码可在此URL获取：[此处应为具体链接，请访问原始发布来源以查看准确的链接地址]。
URL
https://arxiv.org/abs/2504.14075
PDF
-
Towards Realistic Low-Light Image Enhancement via ISP Driven Data Modeling
2025-04-16 15:53:53Zhihua Wang, Yu Long, Qinghua Lin, Kai Zhang, Yazhu Zhang, Yuming Fang, Li Liu, Xiaochun CaoAbstract
Deep neural networks (DNNs) have recently become the leading method for low-light image enhancement (LLIE). However, despite significant progress, their outputs may still exhibit issues such as amplified noise, incorrect white balance, or unnatural enhancements when deployed in real world applications. A key challenge is the lack of diverse, large scale training data that captures the complexities of low-light conditions and imaging pipelines. In this paper, we propose a novel image signal processing (ISP) driven data synthesis pipeline that addresses these challenges by generating unlimited paired training data. Specifically, our pipeline begins with easily collected high-quality normal-light images, which are first unprocessed into the RAW format using a reverse ISP. We then synthesize low-light degradations directly in the RAW domain. The resulting data is subsequently processed through a series of ISP stages, including white balance adjustment, color space conversion, tone mapping, and gamma correction, with controlled variations introduced at each stage. This broadens the degradation space and enhances the diversity of the training data, enabling the generated data to capture a wide range of degradations and the complexities inherent in the ISP pipeline. To demonstrate the effectiveness of our synthetic pipeline, we conduct extensive experiments using a vanilla UNet model consisting solely of convolutional layers, group normalization, GeLU activation, and convolutional block attention modules (CBAMs). Extensive testing across multiple datasets reveals that the vanilla UNet model trained with our data synthesis pipeline delivers high fidelity, visually appealing enhancement results, surpassing state-of-the-art (SOTA) methods both quantitatively and qualitatively.
Abstract (translated)
最近，深度神经网络（DNN）已成为低光图像增强（LLIE）的领先方法。然而，尽管取得了显著进展，它们在实际应用中的输出仍然可能出现诸如放大噪声、错误白平衡或不自然增强等问题。关键挑战之一是缺乏能够捕捉低光条件和成像流程复杂性的多样化大规模训练数据。 为此，本文提出了一种新颖的基于图像信号处理（ISP）的数据合成管道，通过生成无限制配对训练数据来解决这些问题。具体来说，我们的管道从易于收集的高质量正常光照图像开始，并使用反向ISP首先将其未加工为RAW格式。然后，在RAW域直接合成低光退化情况。随后，生成的数据会经过一系列ISP阶段处理，包括白平衡调整、颜色空间转换、色调映射和伽马校正等，同时在每个阶段引入受控变化。这拓宽了降级范围，并增强了训练数据的多样性，使得生成的数据能够捕捉到广泛的降级情况以及ISP流程中的固有复杂性。 为了证明我们合成管道的有效性，我们在一个简单的UNet模型上进行了大量实验，该模型仅由卷积层、组归一化、GeLU激活和卷积块注意模块（CBAMs）组成。在多个数据集上的广泛测试表明，使用我们的数据合成管道训练的简单UNet模型能够提供高保真度且视觉效果良好的增强结果，在量化和定性评估中均超越了最先进的方法（SOTA）。
URL
https://arxiv.org/abs/2504.12204
PDF
-
Learning Physics-Informed Color-Aware Transforms for Low-Light Image Enhancement
2025-04-16 09:23:38Xingxing Yang, Jie Chen, Zaifeng YangAbstract
Image decomposition offers deep insights into the imaging factors of visual data and significantly enhances various advanced computer vision tasks. In this work, we introduce a novel approach to low-light image enhancement based on decomposed physics-informed priors. Existing methods that directly map low-light to normal-light images in the sRGB color space suffer from inconsistent color predictions and high sensitivity to spectral power distribution (SPD) variations, resulting in unstable performance under diverse lighting conditions. To address these challenges, we introduce a Physics-informed Color-aware Transform (PiCat), a learning-based framework that converts low-light images from the sRGB color space into deep illumination-invariant descriptors via our proposed Color-aware Transform (CAT). This transformation enables robust handling of complex lighting and SPD variations. Complementing this, we propose the Content-Noise Decomposition Network (CNDN), which refines the descriptor distributions to better align with well-lit conditions by mitigating noise and other distortions, thereby effectively restoring content representations to low-light images. The CAT and the CNDN collectively act as a physical prior, guiding the transformation process from low-light to normal-light domains. Our proposed PiCat framework demonstrates superior performance compared to state-of-the-art methods across five benchmark datasets.
Abstract (translated)
图像分解为视觉数据的成像因素提供了深刻见解，并显著增强了各种先进的计算机视觉任务。在这项工作中，我们介绍了一种基于分解物理信息先验的新方法来改进低光条件下的图像增强技术。现有直接将低光图像映射到正常光照条件下图像的方法在sRGB色彩空间中存在颜色预测不一致以及对光谱功率分布（SPD）变化高度敏感的问题，导致其在不同照明条件下表现不稳定。为解决这些问题，我们引入了物理信息感知的颜色变换框架——Physics-informed Color-aware Transform (PiCat)，它通过我们提出的Color-aware Transform (CAT) 将低光图像从sRGB色彩空间转换成深度的光照不变描述符。这种变换能够更好地处理复杂的光照和SPD变化问题，从而增强其鲁棒性。 此外，我们提出了内容噪声分解网络（CNDN），该网络通过减少噪声和其他失真来优化描述符分布，使低光图像的内容表示与良好照明条件下的表示更加一致，进而有效地恢复了这些内容的表示。CAT和CNDN共同作为物理先验知识，在从低光照域到正常光照领域的转换过程中提供指导。 我们的PiCat框架在五个基准数据集上展示出优于现有最佳方法的表现。
URL
https://arxiv.org/abs/2504.11896
PDF
-
UKDM: Underwater keypoint detection and matching using underwater image enhancement techniques
2025-04-15 10:52:19Pedro Diaz-Garcia, Felix Escalona, Miguel CazorlaAbstract
The purpose of this paper is to explore the use of underwater image enhancement techniques to improve keypoint detection and matching. By applying advanced deep learning models, including generative adversarial networks and convolutional neural networks, we aim to find the best method which improves the accuracy of keypoint detection and the robustness of matching algorithms. We evaluate the performance of these techniques on various underwater datasets, demonstrating significant improvements over traditional methods.
Abstract (translated)
本文的目的是探讨使用水下图像增强技术来提升关键点检测和匹配的效果。通过应用包括生成对抗网络（GAN）和卷积神经网络（CNN）在内的先进深度学习模型，我们旨在找到能够提高关键点检测准确性及匹配算法鲁棒性的最佳方法。我们在多种水下数据集上评估了这些技术的性能，并展示了相对于传统方法有显著的进步。
URL
https://arxiv.org/abs/2504.11063
PDF
-
AgentPolyp: Accurate Polyp Segmentation via Image Enhancement Agent
2025-04-15 08:39:35Pu Wang, Zhihua Zhang, Dianjie Lu, Guijuan Zhang, Youshan Zhang, Zhuoran ZhengAbstract
Since human and environmental factors interfere, captured polyp images usually suffer from issues such as dim lighting, blur, and overexposure, which pose challenges for downstream polyp segmentation tasks. To address the challenges of noise-induced degradation in polyp images, we present AgentPolyp, a novel framework integrating CLIP-based semantic guidance and dynamic image enhancement with a lightweight neural network for segmentation. The agent first evaluates image quality using CLIP-driven semantic analysis (e.g., identifying ``low-contrast polyps with vascular textures") and adapts reinforcement learning strategies to dynamically apply multi-modal enhancement operations (e.g., denoising, contrast adjustment). A quality assessment feedback loop optimizes pixel-level enhancement and segmentation focus in a collaborative manner, ensuring robust preprocessing before neural network segmentation. This modular architecture supports plug-and-play extensions for various enhancement algorithms and segmentation networks, meeting deployment requirements for endoscopic devices.
Abstract (translated)
由于人类和环境因素的干扰，捕获到的息肉图像通常会遇到诸如光线昏暗、模糊和过度曝光等问题，这些问题给下游的息肉分割任务带来了挑战。为了应对由噪声引起的息肉图像退化问题，我们提出了AgentPolyp，这是一种新型框架，它结合了基于CLIP（Contrastive Language-Image Pre-training）的语义指导以及使用轻量级神经网络进行分割的动态图像增强技术。该框架首先利用CLIP驱动的语义分析评估图像质量（例如，识别“低对比度且具有血管纹理的息肉”），并根据强化学习策略自适应地应用多模态增强操作（如去噪和调整对比度）。一个质量评估反馈循环以协作的方式优化像素级别的增强处理以及分割聚焦点，在神经网络分割前确保进行稳健的预处理。这种模块化架构支持各种增强算法和分割网络的即插即用扩展，满足内窥镜设备部署的需求。
URL
https://arxiv.org/abs/2504.10978
PDF
-
Learning to Harmonize Cross-vendor X-ray Images by Non-linear Image Dynamics Correction
2025-04-14 10:24:57Yucheng Lu, Shunxin Wang, Dovile Juodelyte, Veronika CheplyginaAbstract
In this paper, we explore how conventional image enhancement can improve model robustness in medical image analysis. By applying commonly used normalization methods to images from various vendors and studying their influence on model generalization in transfer learning, we show that the nonlinear characteristics of domain-specific image dynamics cannot be addressed by simple linear transforms. To tackle this issue, we reformulate the image harmonization task as an exposure correction problem and propose a method termed Global Deep Curve Estimation (GDCE) to reduce domain-specific exposure mismatch. GDCE performs enhancement via a pre-defined polynomial function and is trained with the help of a ``domain discriminator'', aiming to improve model transparency in downstream tasks compared to existing black-box methods.
Abstract (translated)
在这篇论文中，我们探讨了传统图像增强技术如何能提高医学影像分析中的模型鲁棒性。通过将常用的归一化方法应用于不同供应商提供的图像，并研究这些方法对迁移学习中模型泛化能力的影响，我们发现简单的线性变换无法解决特定领域内图像动态变化的非线性特征问题。为了解决这一问题，我们将图像校准任务重新定义为曝光调整问题，并提出了一种名为全局深度曲线估计（Global Deep Curve Estimation, GDCE）的方法来减少特定领域的曝光差异。GDCE通过预定义的多项式函数进行增强，并在“领域鉴别器”的帮助下进行训练，旨在与现有的黑盒方法相比，在下游任务中提升模型的透明度。
URL
https://arxiv.org/abs/2504.10080
PDF
-
Low-Light Image Enhancement using Event-Based Illumination Estimation
2025-04-13 00:01:33Lei Sun, Yuhan Bao, Jiajun Zhai, Jingyun Liang, Yulun Zhang, Kaiwei Wang, Danda Pani Paudel, Luc Van GoolAbstract
Low-light image enhancement (LLIE) aims to improve the visibility of images captured in poorly lit environments. Prevalent event-based solutions primarily utilize events triggered by motion, i.e., ''motion events'' to strengthen only the edge texture, while leaving the high dynamic range and excellent low-light responsiveness of event cameras largely unexplored. This paper instead opens a new avenue from the perspective of estimating the illumination using ''temporal-mapping'' events, i.e., by converting the timestamps of events triggered by a transmittance modulation into brightness values. The resulting fine-grained illumination cues facilitate a more effective decomposition and enhancement of the reflectance component in low-light images through the proposed Illumination-aided Reflectance Enhancement module. Furthermore, the degradation model of temporal-mapping events under low-light condition is investigated for realistic training data synthesizing. To address the lack of datasets under this regime, we construct a beam-splitter setup and collect EvLowLight dataset that includes images, temporal-mapping events, and motion events. Extensive experiments across 5 synthetic datasets and our real-world EvLowLight dataset substantiate that the devised pipeline, dubbed RetinEV, excels in producing well-illuminated, high dynamic range images, outperforming previous state-of-the-art event-based methods by up to 6.62 dB, while maintaining an efficient inference speed of 35.6 frame-per-second on a 640X480 image.
Abstract (translated)
低光图像增强（LLIE）旨在改善在光线不足环境中捕获的图像的可见度。目前流行的事件驱动解决方案主要利用由运动触发的事件，即“运动事件”，来强化边缘纹理，而未充分利用事件相机高动态范围和卓越的低光响应性能。 本文则从通过“时间映射”事件估算照明的角度开辟了一条新途径，即将透射率调制触发的事件的时间戳转换为亮度值。由此产生的精细光照线索有助于更有效地分解并增强低光图像中的反射分量，通过提出辅助光照增强模块实现这一点。此外，还研究了在低光条件下时间映射事件退化模型，以进行现实训练数据合成。为了弥补此类数据集的不足，我们构建了一个分束器装置，并收集了包含图像、时间映射事件和运动事件的EvLowLight数据集。 跨5个合成数据集以及我们的真实世界EvLowLight数据集进行了广泛的实验，证实所设计的管道——称为RetinEV，在生成光照良好且动态范围高的图像方面表现出色，优于之前的最先进的事件驱动方法高达6.62 dB，并在分辨率为640x480的图像上保持每秒35.6帧的有效推理速度。
URL
https://arxiv.org/abs/2504.09379
PDF
-
Nonlocal Retinex-Based Variational Model and its Deep Unfolding Twin for Low-Light Image Enhancement
2025-04-10 14:48:26Daniel Torres, Joan Duran, Julia Navarro, Catalina SbertAbstract
Images captured under low-light conditions present significant limitations in many applications, as poor lighting can obscure details, reduce contrast, and hide noise. Removing the illumination effects and enhancing the quality of such images is crucial for many tasks, such as image segmentation and object detection. In this paper, we propose a variational method for low-light image enhancement based on the Retinex decomposition into illumination, reflectance, and noise components. A color correction pre-processing step is applied to the low-light image, which is then used as the observed input in the decomposition. Moreover, our model integrates a novel nonlocal gradient-type fidelity term designed to preserve structural details. Additionally, we propose an automatic gamma correction module. Building on the proposed variational approach, we extend the model by introducing its deep unfolding counterpart, in which the proximal operators are replaced with learnable networks. We propose cross-attention mechanisms to capture long-range dependencies in both the nonlocal prior of the reflectance and the nonlocal gradient-based constraint. Experimental results demonstrate that both methods compare favorably with several recent and state-of-the-art techniques across different datasets. In particular, despite not relying on learning strategies, the variational model outperforms most deep learning approaches both visually and in terms of quality metrics.
Abstract (translated)
在低光条件下拍摄的图像在许多应用中存在重大限制，因为不良照明会遮盖细节、降低对比度并隐藏噪声。去除光照效果并提升这类图像的质量对于诸如图像分割和目标检测等任务至关重要。本文提出了一种基于Retinex分解（将图像分为光照、反射率和噪声组件）的低光图像增强变分方法。在处理过程中，首先对低光图像进行颜色校正预处理，然后将其用作分解中的观察输入。此外，我们的模型整合了一个新颖的非局部梯度型保真项，旨在保留结构细节。我们还提出了一种自动gamma校正模块。基于所提出的变分方法，我们将该模型扩展为深度解包版本，在此版本中，近似算子被可学习网络取代。我们在反射率的非局部先验和基于非局部梯度约束中引入了跨注意力机制来捕捉长距离依赖关系。实验结果表明，无论是在多个数据集上与多种近期及最先进的技术相比，这两种方法都表现出色。特别地，尽管不依赖于学习策略，变分模型在视觉质量和质量指标方面均优于大多数深度学习方法。
URL
https://arxiv.org/abs/2504.07810
PDF
-
Lumina-OmniLV: A Unified Multimodal Framework for General Low-Level Vision
2025-04-07 10:22:00Yuandong Pu, Le Zhuo, Kaiwen Zhu, Liangbin Xie, Wenlong Zhang, Xiangyu Chen, Pneg Gao, Yu Qiao, Chao Dong, Yihao LiuAbstract
We present Lunima-OmniLV (abbreviated as OmniLV), a universal multimodal multi-task framework for low-level vision that addresses over 100 sub-tasks across four major categories: image restoration, image enhancement, weak-semantic dense prediction, and stylization. OmniLV leverages both textual and visual prompts to offer flexible and user-friendly interactions. Built on Diffusion Transformer (DiT)-based generative priors, our framework supports arbitrary resolutions -- achieving optimal performance at 1K resolution -- while preserving fine-grained details and high fidelity. Through extensive experiments, we demonstrate that separately encoding text and visual instructions, combined with co-training using shallow feature control, is essential to mitigate task ambiguity and enhance multi-task generalization. Our findings also reveal that integrating high-level generative tasks into low-level vision models can compromise detail-sensitive restoration. These insights pave the way for more robust and generalizable low-level vision systems.
Abstract (translated)
我们介绍了一种名为Lunima-OmniLV（简称OmniLV）的通用多模态多任务框架，该框架专门用于低级视觉处理，并涵盖了四个主要类别中的超过100个子任务：图像恢复、图像增强、弱语义密集预测以及风格化。OmniLV利用文本和视觉提示来提供灵活且用户友好的交互方式。基于扩散变换器（DiT）的生成先验知识构建，我们的框架支持任意分辨率，并在1K分辨率下实现最佳性能，同时保持了精细细节与高保真度。 通过广泛的实验研究，我们证明分别编码文本和视觉指令，并结合浅层特征控制协同训练方法，对于缓解任务模糊性以及增强多任务泛化能力至关重要。此外，我们的研究成果还表明，在低级视觉模型中整合高级生成任务可能会损害细节敏感型恢复的效果。这些见解为构建更为稳健且通用的低级视觉系统铺平了道路。
URL
https://arxiv.org/abs/2504.04903
PDF
-
Inland Waterway Object Detection in Multi-environment: Dataset and Approach
2025-04-07 08:45:00Shanshan Wang, Haixiang Xu, Hui Feng, Xiaoqian Wang, Pei Song, Sijie Liu, Jianhua HeAbstract
The success of deep learning in intelligent ship visual perception relies heavily on rich image data. However, dedicated datasets for inland waterway vessels remain scarce, limiting the adaptability of visual perception systems in complex environments. Inland waterways, characterized by narrow channels, variable weather, and urban interference, pose significant challenges to object detection systems based on existing datasets. To address these issues, this paper introduces the Multi-environment Inland Waterway Vessel Dataset (MEIWVD), comprising 32,478 high-quality images from diverse scenarios, including sunny, rainy, foggy, and artificial lighting conditions. MEIWVD covers common vessel types in the Yangtze River Basin, emphasizing diversity, sample independence, environmental complexity, and multi-scale characteristics, making it a robust benchmark for vessel detection. Leveraging MEIWVD, this paper proposes a scene-guided image enhancement module to improve water surface images based on environmental conditions adaptively. Additionally, a parameter-limited dilated convolution enhances the representation of vessel features, while a multi-scale dilated residual fusion method integrates multi-scale features for better detection. Experiments show that MEIWVD provides a more rigorous benchmark for object detection algorithms, and the proposed methods significantly improve detector performance, especially in complex multi-environment scenarios.
Abstract (translated)
深度学习在智能船舶视觉感知中的成功很大程度上依赖于丰富的图像数据。然而，专门针对内河航道船只的数据集仍然稀缺，限制了视觉感知系统在复杂环境下的适应性。内河水道的特点包括狭窄的航道、多变的天气以及城市干扰等因素，这些都对基于现有数据集的对象检测系统提出了重大挑战。 为解决这些问题，本文引入了多环境内河航道船舶数据集（MEIWVD），该数据集包含来自各种场景的32,478张高质量图像，涵盖了晴天、雨天、雾天和人工照明条件。MEIWVD涵盖长江流域常见的船只类型，并强调多样性、样本独立性、环境复杂性和多尺度特性，使其成为船舶检测的强大基准。 利用MEIWVD，本文提出了一种基于场景引导的图像增强模块，可根据环境条件自适应地改善水面图像质量。此外，限参数膨胀卷积增强了对船特征的表现力，而多层次膨胀残差融合方法则集成了多层次特征以实现更好的检测效果。 实验表明，MEIWVD为物体检测算法提供了更为严格的基准测试，并且所提出的方法显著提高了探测器在复杂多环境场景中的性能。
URL
https://arxiv.org/abs/2504.04835
PDF
-
Noise Calibration and Spatial-Frequency Interactive Network for STEM Image Enhancement
2025-04-03 13:11:57Hesong Li, Ziqi Wu, Ruiwen Shao, Tao Zhang, Ying FuAbstract
Scanning Transmission Electron Microscopy (STEM) enables the observation of atomic arrangements at sub-angstrom resolution, allowing for atomically resolved analysis of the physical and chemical properties of materials. However, due to the effects of noise, electron beam damage, sample thickness, etc, obtaining satisfactory atomic-level images is often challenging. Enhancing STEM images can reveal clearer structural details of materials. Nonetheless, existing STEM image enhancement methods usually overlook unique features in the frequency domain, and existing datasets lack realism and generality. To resolve these issues, in this paper, we develop noise calibration, data synthesis, and enhancement methods for STEM images. We first present a STEM noise calibration method, which is used to synthesize more realistic STEM images. The parameters of background noise, scan noise, and pointwise noise are obtained by statistical analysis and fitting of real STEM images containing atoms. Then we use these parameters to develop a more general dataset that considers both regular and random atomic arrangements and includes both HAADF and BF mode images. Finally, we design a spatial-frequency interactive network for STEM image enhancement, which can explore the information in the frequency domain formed by the periodicity of atomic arrangement. Experimental results show that our data is closer to real STEM images and achieves better enhancement performances together with our network. Code will be available at this https URL}{this https URL.
Abstract (translated)
扫描透射电子显微镜（STEM）能够以亚埃级分辨率观察材料的原子排列，从而实现对物质物理和化学性质的原子级别分析。然而，由于噪声、电子束损伤、样品厚度等因素的影响，获取满意的原子级图像往往具有挑战性。增强STEM图像可以揭示材料更清晰的结构细节。不过，现有的STEM图像增强方法通常忽视了频域中的独特特征，并且现有数据集缺乏真实性和普遍性。为解决这些问题，在本文中，我们开发了一种用于STEM图像的噪声校准、数据合成和增强的方法。 首先，我们提出一种STEM噪声校准方法，该方法被用来合成更逼真的STEM图像。通过统计分析和拟合含有原子的真实STEM图像中的背景噪声、扫描噪声及点噪声参数来获得这些参数。然后利用这些参数开发了一个更具普遍性的数据集，这个数据集考虑了规则和随机的原子排列，并包括高角度环形暗场（HAADF）模式和背散射（BF）模式下的图像。 最后，我们设计了一种空间频域互动网络用于STEM图像增强，该方法可以探索由原子排列周期性形成的频率域中的信息。实验结果显示我们的数据更接近真实的STEM图像，并且与我们开发的网络一起实现了更好的增强性能。代码可在[此处](https://this https URL)获取。
URL
https://arxiv.org/abs/2504.02555
PDF
-
HPGN: Hybrid Priors-Guided Network for Compressed Low-Light Image Enhancement
2025-04-03 08:06:24Hantang Li, Jinhua Hao, Lei Xiong, Shuyuan ZhuAbstract
In practical applications, conventional methods generate large volumes of low-light images that require compression for efficient storage and transmission. However, most existing methods either disregard the removal of potential compression artifacts during the enhancement process or fail to establish a unified framework for joint task enhancement of images with varying compression qualities. To solve this problem, we propose the hybrid priors-guided network (HPGN), which enhances compressed low-light images by integrating both compression and illumination priors. Our approach fully utilizes the JPEG quality factor (QF) and DCT quantization matrix (QM) to guide the design of efficient joint task plug-and-play modules. Additionally, we employ a random QF generation strategy to guide model training, enabling a single model to enhance images across different compression levels. Experimental results confirm the superiority of our proposed method.
Abstract (translated)
在实际应用中，传统方法生成了大量的低光图像，这些图像需要压缩以实现高效存储和传输。然而，大多数现有方法要么忽略了在增强过程中去除潜在的压缩伪影，要么未能建立一个统一的框架来处理不同压缩质量下的图像任务联合增强问题。为了解决这个问题，我们提出了混合先验引导网络（HPGN），该网络通过结合压缩和光照先验来增强被压缩的低光图像。我们的方法充分利用JPEG质量因子（QF）和DCT量化矩阵（QM）来指导高效的任务联合插件模块的设计。此外，我们采用随机生成QF策略来引导模型训练，使单一模型能够处理不同压缩级别的图像增强任务。实验结果证实了我们所提出的方法的优越性。
URL
https://arxiv.org/abs/2504.02373
PDF
-
Brightness Perceiving for Recursive Low-Light Image Enhancement
2025-04-03 07:53:33Haodian Wang, Long Peng, Yuejin Sun, Zengyu Wan, Yang Wang, Yang CaoAbstract
Due to the wide dynamic range in real low-light scenes, there will be large differences in the degree of contrast degradation and detail blurring of captured images, making it difficult for existing end-to-end methods to enhance low-light images to normal exposure. To address the above issue, we decompose low-light image enhancement into a recursive enhancement task and propose a brightness-perceiving-based recursive enhancement framework for high dynamic range low-light image enhancement. Specifically, our recursive enhancement framework consists of two parallel sub-networks: Adaptive Contrast and Texture enhancement network (ACT-Net) and Brightness Perception network (BP-Net). The ACT-Net is proposed to adaptively enhance image contrast and details under the guidance of the brightness adjustment branch and gradient adjustment branch, which are proposed to perceive the degradation degree of contrast and details in low-light images. To adaptively enhance images captured under different brightness levels, BP-Net is proposed to control the recursive enhancement times of ACT-Net by exploring the image brightness distribution properties. Finally, in order to coordinate ACT-Net and BP-Net, we design a novel unsupervised training strategy to facilitate the training procedure. To further validate the effectiveness of the proposed method, we construct a new dataset with a broader brightness distribution by mixing three low-light datasets. Compared with eleven existing representative methods, the proposed method achieves new SOTA performance on six reference and no reference metrics. Specifically, the proposed method improves the PSNR by 0.9 dB compared to the existing SOTA method.
Abstract (translated)
由于真实低光场景中存在的宽动态范围，拍摄到的图像在对比度退化和细节模糊程度上会有很大的差异，这使得现有的端到端方法难以将低光图像增强至正常曝光。为了解决上述问题，我们将低光图像增强任务分解成一个递归增强的任务，并提出了一种基于亮度感知的递归增强框架来改进高动态范围下的低光图像质量。 具体而言，我们的递归增强框架由两个并行的子网络组成：自适应对比度和纹理增强网络（ACT-Net）以及亮度感知网络（BP-Net）。其中，ACT-Net旨在通过亮度调整分支和梯度调整分支的指导来自适应地增强图像的对比度和细节。这两个分支被设计用于感知低光条件下图像中对比度和细节退化的程度。为了根据不同的亮度水平自适应地增强图像，我们提出了BP-Net，它通过探索图像亮度分布特性来控制ACT-Net的递归增强次数。 最后，为了协调ACT-Net和BP-Net的工作，我们设计了一种新颖的无监督训练策略，以简化并促进整个训练过程。为进一步验证所提方法的有效性，我们将三个低光数据集混合在一起，构建了一个具有更宽亮度分布的新数据集。与现有的11个代表性方法相比，在六个参考和非参考评估指标上，我们提出的方法取得了新的SOTA（State-of-the-Art）性能表现，特别是在PSNR方面，比现有最优方法提高了0.9 dB。
URL
https://arxiv.org/abs/2504.02362
PDF
-
SemiISP/SemiIE: Semi-Supervised Image Signal Processor and Image Enhancement Leveraging One-to-Many Mapping sRGB-to-RAW
2025-04-03 07:28:16Masakazu Yoshimura, Junji Otsuka, Radu Berdan, Takeshi OhashiAbstract
DNN-based methods have been successful in Image Signal Processor (ISP) and image enhancement (IE) tasks. However, the cost of creating training data for these tasks is considerably higher than for other tasks, making it difficult to prepare large-scale datasets. Also, creating personalized ISP and IE with minimal training data can lead to new value streams since preferred image quality varies depending on the person and use case. While semi-supervised learning could be a potential solution in such cases, it has rarely been utilized for these tasks. In this paper, we realize semi-supervised learning for ISP and IE leveraging a RAW image reconstruction (sRGB-to-RAW) method. Although existing sRGB-to-RAW methods can generate pseudo-RAW image datasets that improve the accuracy of RAW-based high-level computer vision tasks such as object detection, their quality is not sufficient for ISP and IE tasks that require precise image quality definition. Therefore, we also propose a sRGB-to-RAW method that can improve the image quality of these tasks. The proposed semi-supervised learning with the proposed sRGB-to-RAW method successfully improves the image quality of various models on various datasets.
Abstract (translated)
基于深度神经网络（DNN）的方法在图像信号处理（ISP）和图像增强（IE）任务中取得了成功。然而，为这些任务创建训练数据的成本远高于其他任务，这使得准备大规模的数据集变得困难。此外，在使用少量训练数据的情况下创造个性化的ISP和IE可以产生新的价值流，因为不同的个人和用例对理想图像质量的需求不同。尽管半监督学习在这种情况下可能是一个潜在的解决方案，但其在这些任务中的应用却很少见。在这篇论文中，我们通过利用RAW图像重建（sRGB到RAW）方法实现了ISP和IE领域的半监督学习。虽然现有的sRGB到RAW的方法可以生成伪RAW图像数据集以提高基于RAW的高级计算机视觉任务（如目标检测）的准确性，但这些数据的质量不足以满足需要精确图像质量定义的ISP和IE任务的需求。因此，我们还提出了一种改进这些任务中图像质量的sRGB到RAW方法。结合提出的半监督学习和sRGB到RAW的方法，在各种数据集上对多种模型的图像质量得到了显著提升。
URL
https://arxiv.org/abs/2504.02345
PDF
-
A GAN-Enhanced Deep Learning Framework for Rooftop Detection from Historical Aerial Imagery
2025-03-29 19:51:39Pengyu Chen, Sicheng Wang, Cuizhen Wang, Senrong Wang, Beiao Huang, Lu Huang, Zhe ZangAbstract
Accurate rooftop detection from historical aerial imagery is vital for examining long-term urban development and human settlement patterns. However, black-and-white analog photographs pose significant challenges for modern object detection frameworks due to their limited spatial resolution, lack of color information, and archival degradation. To address these limitations, this study introduces a two-stage image enhancement pipeline based on Generative Adversarial Networks (GANs): image colorization using DeOldify, followed by super-resolution enhancement with Real-ESRGAN. The enhanced images were then used to train and evaluate rooftop detection models, including Faster R-CNN, DETReg, and YOLOv11n. Results show that combining colorization with super-resolution substantially improves detection performance, with YOLOv11n achieving a mean Average Precision (mAP) exceeding 85%. This reflects an improvement of approximately 40% over original black-and-white images and 20% over images enhanced through colorization alone. The proposed method effectively bridges the gap between archival imagery and contemporary deep learning techniques, enabling more reliable extraction of building footprints from historical aerial photographs.
Abstract (translated)
从历史航拍图像中准确检测屋顶对于考察长期城市发展和人类居住模式至关重要。然而，黑白胶片照片由于其有限的空间分辨率、缺乏颜色信息以及档案退化等问题，给现代物体检测框架带来了重大挑战。为了解决这些问题，本研究引入了一种基于生成对抗网络（GAN）的两阶段图像增强管道：首先使用DeOldify进行图像着色，然后采用Real-ESRGAN进行超分辨率增强。经过这些处理后的图像被用来训练和评估屋顶检测模型，包括Faster R-CNN、DETReg 和 YOLOv11n。实验结果显示，将着色与超分辨率相结合可以显著提高检测性能，其中YOLOv11n的平均精度均值（mAP）超过了85%。这比原始黑白图像提高了约40%，相比仅通过着色增强的图像也提升了20%。本方法有效地弥合了档案影像和现代深度学习技术之间的差距，使得从历史航拍照片中可靠地提取建筑轮廓成为可能。
URL
https://arxiv.org/abs/2503.23200
PDF
-
Underwater Image Enhancement by Convolutional Spiking Neural Networks
2025-03-26 12:15:38Vidya Sudevan, Fakhreddine Zayer, Rizwana Kausar, Sajid Javed, Hamad Karki, Giulia De Masi, Jorge DiasAbstract
Underwater image enhancement (UIE) is fundamental for marine applications, including autonomous vision-based navigation. Deep learning methods using convolutional neural networks (CNN) and vision transformers advanced UIE performance. Recently, spiking neural networks (SNN) have gained attention for their lightweight design, energy efficiency, and scalability. This paper introduces UIE-SNN, the first SNN-based UIE algorithm to improve visibility of underwater images. UIE-SNN is a 19- layered convolutional spiking encoder-decoder framework with skip connections, directly trained using surrogate gradient-based backpropagation through time (BPTT) strategy. We explore and validate the influence of training datasets on energy reduction, a unique advantage of UIE-SNN architecture, in contrast to the conventional learning-based architectures, where energy consumption is model-dependent. UIE-SNN optimizes the loss function in latent space representation to reconstruct clear underwater images. Our algorithm performs on par with its non-spiking counterpart methods in terms of PSNR and structural similarity index (SSIM) at reduced timesteps ($T=5$) and energy consumption of $85\%$. The algorithm is trained on two publicly available benchmark datasets, UIEB and EUVP, and tested on unseen images from UIEB, EUVP, LSUI, U45, and our custom UIE dataset. The UIE-SNN algorithm achieves PSNR of \(17.7801~dB\) and SSIM of \(0.7454\) on UIEB, and PSNR of \(23.1725~dB\) and SSIM of \(0.7890\) on EUVP. UIE-SNN achieves this algorithmic performance with fewer operators (\(147.49\) GSOPs) and energy (\(0.1327~J\)) compared to its non-spiking counterpart (GFLOPs = \(218.88\) and Energy=\(1.0068~J\)). Compared with existing SOTA UIE methods, UIE-SNN achieves an average of \(6.5\times\) improvement in energy efficiency. The source code is available at \href{this https URL}{UIE-SNN}.
Abstract (translated)
水下图像增强（UIE）对于海洋应用至关重要，包括自主视觉导航。使用卷积神经网络（CNN）和视觉变换器的深度学习方法提升了UIE性能。近期，脉冲神经网络（SNN）因其轻量级设计、能源效率及可扩展性而备受关注。本文介绍了UIE-SNN，一种基于SNN的首个水下图像增强算法，旨在提高水下图像的可见度。UIE-SNN是一个包含19层卷积脉冲编码器-解码器框架，并通过跳连结构直接使用基于替代梯度的时间反向传播（BPTT）策略进行训练。我们探讨并验证了训练数据集对能耗减少的影响，这突显了UIE-SNN架构的独特优势，在传统学习架构中，能源消耗依赖于模型本身。UIE-SNN在潜在空间表示中优化损失函数以重构清晰的水下图像。 我们的算法在PSNR（峰值信噪比）和结构相似性指数（SSIM）方面与非脉冲版本的方法相当，但在减少时间步长（$T=5$）和能耗（85%）的情况下表现良好。该算法使用两个公开基准数据集UIEB和EUVP进行训练，并在来自UIEB、EUVP、LSUI、U45以及我们自定义的UIE数据集中未见过的图像上进行了测试。UIE-SNN在UIEB上的PSNR为$17.7801\,dB$，SSIM为$0.7454$；而在EUVP上的PSNR为$23.1725\,dB$，SSIM为$0.7890$。与非脉冲版本相比，UIE-SNN使用更少的操作（GSOPs = $147.49$）和能量消耗（$0.1327~J$），而非脉冲版本的GFLOPs = $218.88$且能耗为$1.0068~J$。相较于现有的UIE先进方法，UIE-SNN实现了平均$6.5\times$的能效提升。 该算法的源代码可在[UIE-SNN](this https URL)获取。
URL
https://arxiv.org/abs/2503.20485
PDF
-
LENVIZ: A High-Resolution Low-Exposure Night Vision Benchmark Dataset
2025-03-25 16:12:28Manjushree Aithal, Rosaura G. VidalMata, Manikandtan Kartha, Gong Chen, Eashan Adhikarla, Lucas N. Kirsten, Zhicheng Fu, Nikhil A. Madhusudhana, Joe NastiAbstract
Low-light image enhancement is crucial for a myriad of applications, from night vision and surveillance, to autonomous driving. However, due to the inherent limitations that come in hand with capturing images in low-illumination environments, the task of enhancing such scenes still presents a formidable challenge. To advance research in this field, we introduce our Low Exposure Night Vision (LENVIZ) Dataset, a comprehensive multi-exposure benchmark dataset for low-light image enhancement comprising of over 230K frames showcasing 24K real-world indoor and outdoor, with-and without human, scenes. Captured using 3 different camera sensors, LENVIZ offers a wide range of lighting conditions, noise levels, and scene complexities, making it the largest publicly available up-to 4K resolution benchmark in the field. LENVIZ includes high quality human-generated ground truth, for which each multi-exposure low-light scene has been meticulously curated and edited by expert photographers to ensure optimal image quality. Furthermore, we also conduct a comprehensive analysis of current state-of-the-art low-light image enhancement techniques on our dataset and highlight potential areas of improvement.
Abstract (translated)
低光图像增强对于夜视、监控和自动驾驶等多种应用至关重要。然而，由于在低光照环境下拍摄照片时会遇到固有的局限性，提高这些场景质量的任务仍然充满挑战。为了推动该领域的研究进展，我们推出了《低曝光夜视（LENVIZ）数据集》，这是一个全面的多曝光基准数据集，专门用于低光图像增强，包含超过230,000帧的真实世界室内和室外场景，涵盖有人与无人的情况，总计展示出24,000多个场景。该数据集使用三种不同的相机传感器采集而成，并提供了广泛的不同光照条件、噪声水平及场景复杂度的样本，使其成为目前公开可用的最大规模低光图像增强基准数据集之一，最高可达4K分辨率。 LENVIZ数据集包括高质量的人工生成的真实标注，每个多曝光低光场景均由专业摄影师精心策划和编辑，以确保最佳图像质量。此外，我们还在我们的数据集上对当前最先进的低光图像增强技术进行了全面分析，并指出了潜在的改进领域。
URL
https://arxiv.org/abs/2503.19804
PDF
-
LLGS: Unsupervised Gaussian Splatting for Image Enhancement and Reconstruction in Pure Dark Environment
2025-03-24 13:05:05Haoran Wang, Jingwei Huang, Lu Yang, Tianchen Deng, Gaojing Zhang, Mingrui LiAbstract
3D Gaussian Splatting has shown remarkable capabilities in novel view rendering tasks and exhibits significant potential for multi-view this http URL, the original 3D Gaussian Splatting lacks color representation for inputs in low-light environments. Simply using enhanced images as inputs would lead to issues with multi-view consistency, and current single-view enhancement systems rely on pre-trained data, lacking scene generalization. These problems limit the application of 3D Gaussian Splatting in low-light conditions in the field of robotics, including high-fidelity modeling and feature matching. To address these challenges, we propose an unsupervised multi-view stereoscopic system based on Gaussian Splatting, called Low-Light Gaussian Splatting (LLGS). This system aims to enhance images in low-light environments while reconstructing the scene. Our method introduces a decomposable Gaussian representation called M-Color, which separately characterizes color information for targeted enhancement. Furthermore, we propose an unsupervised optimization method with zero-knowledge priors, using direction-based enhancement to ensure multi-view consistency. Experiments conducted on real-world datasets demonstrate that our system outperforms state-of-the-art methods in both low-light enhancement and 3D Gaussian Splatting.
Abstract (translated)
三维高斯点阵（3D Gaussian Splatting）在新颖视角渲染任务中展现了显著的能力，并且对于多视图立体视觉展示了巨大的潜力。然而，原始的3D高斯点阵在处理低光环境下的输入时缺乏颜色表示能力。简单地使用增强后的图像作为输入会导致多视图一致性问题，并且当前单视图增强系统依赖于预训练数据，这导致了场景泛化能力不足。这些问题限制了3D高斯点阵在包括高保真建模和特征匹配等领域的机器人技术中低光条件下的应用。 为了解决这些挑战，我们提出了一种基于高斯点阵的无监督多视图立体视觉系统，称为低光高斯点阵（Low-Light Gaussian Splatting, LLGS）。该系统旨在增强低光照环境中的图像并重建场景。我们的方法引入了一种可分解的高斯表示形式，称为M-Color，它可以单独表征颜色信息以进行有针对性的提升。此外，我们还提出了一种基于零先验知识的无监督优化方法，并使用方向导向性的增强技术来确保多视图一致性。 在真实世界数据集上的实验表明，我们的系统在低光环境下的图像增强以及三维高斯点阵任务中均优于现有的最先进技术。
URL
https://arxiv.org/abs/2503.18640
PDF
-
Cross-Domain Underwater Image Enhancement Guided by No-Reference Image Quality Assessment: A Transfer Learning Approach
2025-03-23 04:40:07Zhi Zhang, Daoyi ChenAbstract
Single underwater image enhancement (UIE) is a challenging ill-posed problem, but its development is hindered by two major issues: (1) The labels in underwater reference datasets are pseudo labels, relying on these pseudo ground truths in supervised learning leads to domain discrepancy. (2) Underwater reference datasets are scarce, making training on such small datasets prone to overfitting and distribution shift. To address these challenges, we propose Trans-UIE, a transfer learning-based UIE model that captures the fundamental paradigms of UIE through pretraining and utilizes a dataset composed of both reference and non-reference datasets for fine-tuning. However, fine-tuning the model using only reconstruction loss may introduce confirmation bias. To mitigate this, our method leverages no-reference image quality assessment (NR-IQA) metrics from above-water scenes to guide the transfer learning process across domains while generating enhanced images with the style of the above-water image domain. Additionally, to reduce the risk of overfitting during the pretraining stage, we introduce Pearson correlation loss. Experimental results on both full-reference and no-reference underwater benchmark datasets demonstrate that Trans-UIE significantly outperforms state-of-the-art methods.
Abstract (translated)
单一的水下图像增强（UIE）是一个具有挑战性的、条件不明确的问题，其发展受到两个主要问题的影响：(1) 水下参考数据集中的标签是伪标签，依赖这些伪真实标签进行监督学习会导致领域差异。(2) 水下参考数据集稀缺，使得在这样的小规模数据集上训练容易导致过拟合和分布偏移。为了解决这些问题，我们提出了Trans-UIE，这是一种基于迁移学习的UIE模型，通过预训练捕捉到UIE的基本范式，并使用包含参考和非参考数据集组成的混合数据集进行微调。然而，仅依靠重构损失对模型进行微调可能会引入确认偏差（即只加强已有的假设）。为了解决这个问题，我们的方法利用了来自水面场景的无参考图像质量评估（NR-IQA）指标来指导跨领域的迁移学习过程，并生成具有水面图像领域风格的增强图像。此外，在预训练阶段为了降低过拟合的风险，我们引入了皮尔逊相关损失。 在包含全参考和无参考水下基准数据集上的实验结果表明，Trans-UIE显著优于最先进的方法。
URL
https://arxiv.org/abs/2503.17937
PDF
-
DPF-Net: Physical Imaging Model Embedded Data-Driven Underwater Image Enhancement
2025-03-16 11:53:18Han Mei, Kunqian Li, Shuaixin Liu, Chengzhi Ma, Qianli JiangAbstract
Due to the complex interplay of light absorption and scattering in the underwater environment, underwater images experience significant degradation. This research presents a two-stage underwater image enhancement network called the Data-Driven and Physical Parameters Fusion Network (DPF-Net), which harnesses the robustness of physical imaging models alongside the generality and efficiency of data-driven methods. We first train a physical parameter estimate module using synthetic datasets to guarantee the trustworthiness of the physical parameters, rather than solely learning the fitting relationship between raw and reference images by the application of the imaging equation, as is common in prior studies. This module is subsequently trained in conjunction with an enhancement network, where the estimated physical parameters are integrated into a data-driven model within the embedding space. To maintain the uniformity of the restoration process amid underwater imaging degradation, we propose a physics-based degradation consistency loss. Additionally, we suggest an innovative weak reference loss term utilizing the entire dataset, which alleviates our model's reliance on the quality of individual reference images. Our proposed DPF-Net demonstrates superior performance compared to other benchmark methods across multiple test sets, achieving state-of-the-art results. The source code and pre-trained models are available on the project home page: this https URL.
Abstract (translated)
由于水下环境中光的吸收和散射的复杂相互作用，水下图像会经历显著的质量退化。这项研究提出了一种两阶段的水下图像增强网络——数据驱动与物理参数融合网络（DPF-Net），该网络结合了物理成像模型的稳健性和基于数据的方法的通用性及高效性。 我们首先使用合成数据集训练一个物理参数估计模块，以确保所获得物理参数的真实有效性，而不是仅仅通过应用成像方程来学习原始图像和参考图像之间的拟合关系（如先前研究中常见的做法）。随后，在增强网络与该物理参数估计模块共同训练的过程中，将估算出的物理参数整合到嵌入空间中的数据驱动模型内。 为在水下成像退化过程中保持恢复过程的一致性，我们提出了一种基于物理学的降解一致性损失。此外，我们还建议利用整个数据集创新地引入一个弱参考损失项，以减轻我们的模型对单个参考图像质量依赖的问题。 我们在多个测试集合上验证了所提出的DPF-Net方法，其性能优于其他基准方法，并且达到了最先进的水平。该项目的源代码和预训练模型可在项目主页：[此链接]（请将URL替换为实际提供的链接）获取。
URL
https://arxiv.org/abs/2503.12470
PDF
