Image_Enhancement
-
Dream-IF: Dynamic Relative EnhAnceMent for Image Fusion
2025-03-13 07:08:35Xingxin Xu, Bing Cao, Yinan Xia, Pengfei Zhu, Qinghua HuAbstract
Image fusion aims to integrate comprehensive information from images acquired through multiple sources. However, images captured by diverse sensors often encounter various degradations that can negatively affect fusion quality. Traditional fusion methods generally treat image enhancement and fusion as separate processes, overlooking the inherent correlation between them; notably, the dominant regions in one modality of a fused image often indicate areas where the other modality might benefit from enhancement. Inspired by this observation, we introduce the concept of dominant regions for image enhancement and present a Dynamic Relative EnhAnceMent framework for Image Fusion (Dream-IF). This framework quantifies the relative dominance of each modality across different layers and leverages this information to facilitate reciprocal cross-modal enhancement. By integrating the relative dominance derived from image fusion, our approach supports not only image restoration but also a broader range of image enhancement applications. Furthermore, we employ prompt-based encoding to capture degradation-specific details, which dynamically steer the restoration process and promote coordinated enhancement in both multi-modal image fusion and image enhancement scenarios. Extensive experimental results demonstrate that Dream-IF consistently outperforms its counterparts.
Abstract (translated)
图像融合的目标是整合通过多种来源获取的图像中的全面信息。然而，不同传感器捕获的图像经常会遇到各种退化问题，这些问题可能会影响融合质量。传统的方法通常将图像增强和融合视为两个独立的过程，忽视了它们之间的内在关联；特别地，在一幅融合图像中某一模式下的主导区域往往表明其他模式下该区域可能存在需要提升的地方。基于这一观察，我们引入了用于图像增强的主导区域的概念，并提出了一种名为动态相对增强框架（Dynamic Relative EnhAnceMent framework for Image Fusion，简称Dream-IF）的方法。此框架量化了不同层次上每一种模式相对于另一种模式的优势，并利用这些信息来促进相互之间的跨模态增强。通过将从图像融合中得出的相对优势融入其中，我们的方法不仅支持图像恢复，还能够广泛应用于其他图像增强应用场合。 此外，我们采用基于提示编码的方法捕捉特定退化的细节，这可以动态地引导恢复过程并推动多模式图像融合和图像增强场景中的协调性提升。广泛的实验结果表明，Dream-IF在各种对比测试中始终表现出色。
URL
https://arxiv.org/abs/2503.10109
PDF
-
QUIET-SR: Quantum Image Enhancement Transformer for Single Image Super-Resolution
2025-03-11 16:06:16Siddhant Dutta, Nouhaila Innan, Khadijeh Najafi, Sadok Ben Yahia, Muhammad ShafiqueAbstract
Recent advancements in Single-Image Super-Resolution (SISR) using deep learning have significantly improved image restoration quality. However, the high computational cost of processing high-resolution images due to the large number of parameters in classical models, along with the scalability challenges of quantum algorithms for image processing, remains a major obstacle. In this paper, we propose the Quantum Image Enhancement Transformer for Super-Resolution (QUIET-SR), a hybrid framework that extends the Swin transformer architecture with a novel shifted quantum window attention mechanism, built upon variational quantum neural networks. QUIET-SR effectively captures complex residual mappings between low-resolution and high-resolution images, leveraging quantum attention mechanisms to enhance feature extraction and image restoration while requiring a minimal number of qubits, making it suitable for the Noisy Intermediate-Scale Quantum (NISQ) era. We evaluate our framework in MNIST (30.24 PSNR, 0.989 SSIM), FashionMNIST (29.76 PSNR, 0.976 SSIM) and the MedMNIST dataset collection, demonstrating that QUIET-SR achieves PSNR and SSIM scores comparable to state-of-the-art methods while using fewer parameters. These findings highlight the potential of scalable variational quantum machine learning models for SISR, marking a step toward practical quantum-enhanced image super-resolution.
Abstract (translated)
最近，基于深度学习的单图像超分辨率（SISR）技术取得了显著进展，大幅提升了图像恢复质量。然而，由于经典模型中参数数量庞大而导致处理高分辨率图像时计算成本高昂，以及量子算法在图像处理中的可扩展性挑战，仍然是一个主要障碍。为此，本文提出了名为“用于超分辨率的量子图像增强变换器”（QUIET-SR），这是一个混合框架，它通过引入一种新颖的移位量子窗口注意机制来扩展Swin变压器架构，并基于变分量子神经网络构建而成。QUIET-SR能够有效捕捉低分辨率和高分辨率图像之间的复杂残差映射关系，在利用量子注意机制增强特征提取和图像恢复的同时，只需要少量的量子比特（qubits），使其适合于嘈杂中等规模量子（NISQ）时代。 我们通过在MNIST数据集（峰值信噪比30.24，结构相似性指数0.989）、FashionMNIST数据集（峰值信噪比29.76，结构相似性指数0.976）和MedMNIST数据集集合上评估我们的框架，证明了QUIET-SR在使用较少参数的情况下取得了与现有最佳方法相当的峰值信噪比（PSNR）和结构相似性指标（SSIM）。这些发现表明可扩展变分量子机器学习模型在SISR中的潜力，并标志着向实用量子增强图像超分辨率技术迈进的重要一步。
URL
https://arxiv.org/abs/2503.08759
PDF
-
PromptLNet: Region-Adaptive Aesthetic Enhancement via Prompt Guidance in Low-Light Enhancement Net
2025-03-11 10:45:08Jun Yin, Yangfan He, Miao Zhang, Pengyu Zeng, Tianyi Wang, Shuai Lu, Xueqian WangAbstract
Learning and improving large language models through human preference feedback has become a mainstream approach, but it has rarely been applied to the field of low-light image enhancement. Existing low-light enhancement evaluations typically rely on objective metrics (such as FID, PSNR, etc.), which often result in models that perform well objectively but lack aesthetic quality. Moreover, most low-light enhancement models are primarily designed for global brightening, lacking detailed refinement. Therefore, the generated images often require additional local adjustments, leading to research gaps in practical applications. To bridge this gap, we propose the following innovations: 1) We collect human aesthetic evaluation text pairs and aesthetic scores from multiple low-light image datasets (e.g., LOL, LOL2, LOM, DCIM, MEF, etc.) to train a low-light image aesthetic evaluation model, supplemented by an optimization algorithm designed to fine-tune the diffusion model. 2) We propose a prompt-driven brightness adjustment module capable of performing fine-grained brightness and aesthetic adjustments for specific instances or regions. 3) We evaluate our method alongside existing state-of-the-art algorithms on mainstream benchmarks. Experimental results show that our method not only outperforms traditional methods in terms of visual quality but also provides greater flexibility and controllability, paving the way for improved aesthetic quality.
Abstract (translated)
通过人类偏好反馈来学习和改进大型语言模型已成为主流方法，但这种方法很少被应用于低光图像增强领域。现有的低光照增强评估通常依赖于客观指标（如FID、PSNR等），这往往导致在客观性能上表现良好但实际上缺乏美学质量的模型出现。此外，大多数低光照增强模型主要设计用于全局增亮，缺少细节上的精细处理。因此，生成的图像常常需要额外的局部调整，从而导致实际应用中的研究空白。 为了填补这一空白，我们提出了以下创新： 1) 我们从多个低光图像数据集（如LOL、LOL2、LOM、DCIM、MEF等）中收集了人类美学评价文本对和美学评分，以训练一个专门用于评估低光照图像美学的模型，并补充了一个旨在微调扩散模型的优化算法。 2) 我们提出了一种由提示驱动的亮度调整模块，该模块能够针对特定实例或区域执行精细的亮度和美学调整。 3) 我们在主流基准测试中与现有的最先进算法进行了比较评估。实验结果表明，我们的方法不仅在视觉质量上超越了传统方法，还提供了更大的灵活性和可控性，为提高图像美学质量铺平了道路。
URL
https://arxiv.org/abs/2503.08276
PDF
-
TSCnet: A Text-driven Semantic-level Controllable Framework for Customized Low-Light Image Enhancement
2025-03-11 08:30:50Miao Zhang, Jun Yin, Pengyu Zeng, Yiqing Shen, Shuai Lu, Xueqian WangAbstract
Deep learning-based image enhancement methods show significant advantages in reducing noise and improving visibility in low-light conditions. These methods are typically based on one-to-one mapping, where the model learns a direct transformation from low light to specific enhanced images. Therefore, these methods are inflexible as they do not allow highly personalized mapping, even though an individual's lighting preferences are inherently personalized. To overcome these limitations, we propose a new light enhancement task and a new framework that provides customized lighting control through prompt-driven, semantic-level, and quantitative brightness adjustments. The framework begins by leveraging a Large Language Model (LLM) to understand natural language prompts, enabling it to identify target objects for brightness adjustments. To localize these target objects, the Retinex-based Reasoning Segment (RRS) module generates precise target localization masks using reflection images. Subsequently, the Text-based Brightness Controllable (TBC) module adjusts brightness levels based on the generated illumination map. Finally, an Adaptive Contextual Compensation (ACC) module integrates multi-modal inputs and controls a conditional diffusion model to adjust the lighting, ensuring seamless and precise enhancements accurately. Experimental results on benchmark datasets demonstrate our framework's superior performance at increasing visibility, maintaining natural color balance, and amplifying fine details without creating artifacts. Furthermore, its robust generalization capabilities enable complex semantic-level lighting adjustments in diverse open-world environments through natural language interactions.
Abstract (translated)
基于深度学习的图像增强方法在减少噪声和提高低光条件下的可视性方面表现出显著优势。这些方法通常依赖于一对一映射，即模型从低光照直接转换到特定的增强图像上进行学习。因此，这类方法不够灵活，无法提供高度个性化的映射调整，尽管每个人的照明偏好是独特的。为了克服这一限制，我们提出了一项新的光增强任务和一个新框架，该框架能够通过提示驱动、语义级别以及亮度量化的自定义光照控制来实现个性化。 该框架首先利用大型语言模型（LLM）理解自然语言指令，并识别需要进行亮度调整的目标对象。为了精确定位这些目标对象，基于Retinex的推理分割(RRS)模块使用反射图像生成精确的目标定位掩码。接着，文本驱动亮度可控(TBC)模块根据生成的照明图来调节亮度水平。最后，自适应上下文补偿(ACC)模块结合多模态输入，并控制条件扩散模型以调整光照，从而确保无缝且精准的增强效果。 在基准数据集上的实验结果显示，我们的框架在提高可见性、保持自然色彩平衡以及放大细节点方面优于现有技术，同时不产生伪影。此外，其强大的泛化能力使其能够通过自然语言互动，在多样的开放世界环境中实现复杂的语义级光照调整。
URL
https://arxiv.org/abs/2503.08168
PDF
-
Deep Perceptual Enhancement for Medical Image Analysis
2025-03-11 04:20:16S M A Sharif, Rizwan Ali Naqvi, Mithun Biswas, Woong-Kee LohAbstract
Due to numerous hardware shortcomings, medical image acquisition devices are susceptible to producing low-quality (i.e., low contrast, inappropriate brightness, noisy, etc.) images. Regrettably, perceptually degraded images directly impact the diagnosis process and make the decision-making manoeuvre of medical practitioners notably complicated. This study proposes to enhance such low-quality images by incorporating end-to-end learning strategies for accelerating medical image analysis tasks. To the best concern, this is the first work in medical imaging which comprehensively tackles perceptual enhancement, including contrast correction, luminance correction, denoising, etc., with a fully convolutional deep network. The proposed network leverages residual blocks and a residual gating mechanism for diminishing visual artefacts and is guided by a multi-term objective function to perceive the perceptually plausible enhanced images. The practicability of the deep medical image enhancement method has been extensively investigated with sophisticated experiments. The experimental outcomes illustrate that the proposed method could outperform the existing enhancement methods for different medical image modalities by 5.00 to 7.00 dB in peak signal-to-noise ratio (PSNR) metrics and 4.00 to 6.00 in DeltaE metrics. Additionally, the proposed method can drastically improve the medical image analysis tasks' performance and reveal the potentiality of such an enhancement method in real-world applications. Code Available: this https URL
Abstract (translated)
由于硬件方面的诸多不足，医疗图像采集设备容易产生低质量的图像（例如，对比度低、亮度不适当、噪点多等）。这些感知上降级的图像会直接对诊断过程造成影响，并使医疗从业者的决策操作变得复杂。本研究提出了一种通过引入端到端学习策略来增强此类低质量图像的方法，以加速医学影像分析任务的处理速度。据我们所知，这是首次在医学成像领域综合应对感知增强问题的研究，包括对比度校正、亮度校正和去噪等，并使用了一个完全卷积的深度网络来实现这些功能。该提出的网络利用了残差块和一个残差门控机制以减少视觉伪影，并通过一个多项式目标函数引导其操作，从而能够感知并生成感知上合理的增强图像。通过对复杂实验进行广泛调查，证明了这种深度医学影像增强方法的实际应用性。实验结果显示，相较于现有的增强方法，在不同的医学图像模式下，所提出的方法在峰值信噪比（PSNR）指标上可以提高5.00到7.00分贝，在DeltaE度量上提高了4.00至6.00。此外，该提出的方案能够极大地提升医学影像分析任务的表现，并揭示了此类增强方法在实际应用中的潜力。 代码可用：[此链接](this%20https%20URL)
URL
https://arxiv.org/abs/2503.08027
PDF
-
GM-MoE: Low-Light Enhancement with Gated-Mechanism Mixture-of-Experts
2025-03-10 15:05:50Minwen Liao, Hao Bo Dong, Xinyi Wang, Ziyang Yan, Yihua ShaoAbstract
Low-light enhancement has wide applications in autonomous driving, 3D reconstruction, remote sensing, surveillance, and so on, which can significantly improve information utilization. However, most existing methods lack generalization and are limited to specific tasks such as image recovery. To address these issues, we propose \textbf{Gated-Mechanism Mixture-of-Experts (GM-MoE)}, the first framework to introduce a mixture-of-experts network for low-light image enhancement. GM-MoE comprises a dynamic gated weight conditioning network and three sub-expert networks, each specializing in a distinct enhancement task. Combining a self-designed gated mechanism that dynamically adjusts the weights of the sub-expert networks for different data domains. Additionally, we integrate local and global feature fusion within sub-expert networks to enhance image quality by capturing multi-scale features. Experimental results demonstrate that the GM-MoE achieves superior generalization with respect to 25 compared approaches, reaching state-of-the-art performance on PSNR on 5 benchmarks and SSIM on 4 benchmarks, respectively.
Abstract (translated)
低光增强技术在自动驾驶、三维重建、遥感、监控等领域有广泛的应用，可以显著提高信息利用效率。然而，现有的大多数方法缺乏泛化能力，并且局限于特定任务如图像恢复。为解决这些问题，我们提出了**门控机制专家混合网络（GM-MoE）**，这是首个引入专家混合网络以进行低光图像增强的框架。GM-MoE包含一个动态门控权重调节网络和三个子专家网络，每个子专家网络专注于不同的增强任务。通过结合自设计的门控机制来动态调整针对不同数据域的子专家网络权重。此外，我们还在子专家网络中集成了局部与全局特征融合功能，以捕捉多尺度特征从而提升图像质量。实验结果表明，GM-MoE在25种比较方法中表现出色，并且在五个基准测试上达到了PSNR指标上的最新性能，在四个基准测试上达到SSIM指标的领先表现。
URL
https://arxiv.org/abs/2503.07417
PDF
-
MERLION: Marine ExploRation with Language guIded Online iNformative Visual Sampling and Enhancement
2025-03-10 05:59:01Shrutika Vishal Thengane, Marcel Bartholomeus Prasetyo, Yu Xiang Tan, Malika MeghjaniAbstract
Autonomous and targeted underwater visual monitoring and exploration using Autonomous Underwater Vehicles (AUVs) can be a challenging task due to both online and offline constraints. The online constraints comprise limited onboard storage capacity and communication bandwidth to the surface, whereas the offline constraints entail the time and effort required for the selection of desired key frames from the video data. An example use case of targeted underwater visual monitoring is finding the most interesting visual frames of fish in a long sequence of an AUV's visual experience. This challenge of targeted informative sampling is further aggravated in murky waters with poor visibility. In this paper, we present MERLION, a novel framework that provides semantically aligned and visually enhanced summaries for murky underwater marine environment monitoring and exploration. Specifically, our framework integrates (a) an image-text model for semantically aligning the visual samples to the users' needs, (b) an image enhancement model for murky water visual data and (c) an informative sampler for summarizing the monitoring experience. We validate our proposed MERLION framework on real-world data with user studies and present qualitative and quantitative results using our evaluation metric and show improved results compared to the state-of-the-art approaches. We have open-sourced the code for MERLION at the following link this https URL.
Abstract (translated)
使用自主水下航行器（AUV）进行自主和目标化的水下视觉监测与探索是一项充满挑战的任务，这些挑战既包括在线约束也包括离线约束。在线约束主要包括有限的机载存储容量以及与水面通信时的带宽限制；而离线约束则涉及从视频数据中选择所需关键帧所花费的时间和精力。例如，在目标化水下视觉监测的一个应用场景中，研究者需要在一段长时间的AUV视觉体验序列中找到最有趣的鱼的画面。这一信息采样挑战在浑浊、能见度差的水域里会更加严重。 本文介绍了一种名为MERLION的新框架，它为浑浊的水下海洋环境监测和探索提供了语义对齐且视觉增强的摘要。具体来说，我们的框架集成了以下三个部分：(a) 一个图像-文本模型，用于将视觉样本与用户需求进行语义对齐；(b) 一个适用于浑浊水域视觉数据的图像增强模型；以及 (c) 一种信息采样器，用于总结监测体验。我们通过真实世界的数据和用户研究验证了MERLION框架的有效性，并使用我们的评估指标展示了定性和定量的结果，结果显示相比现有最佳方法，我们的方法有显著改进。 MERLION框架的代码已在以下链接开源：[此处插入实际链接]。
URL
https://arxiv.org/abs/2503.06953
PDF
-
An Adaptive Underwater Image Enhancement Framework via Multi-Domain Fusion and Color Compensation
2025-03-05 16:19:56Yuezhe Tian, Kangchen Yao, Xiaoyang YuAbstract
Underwater optical imaging is severely degraded by light absorption, scattering, and color distortion, hindering visibility and accurate image analysis. This paper presents an adaptive enhancement framework integrating illumination compensation, multi-domain filtering, and dynamic color correction. A hybrid illumination compensation strategy combining CLAHE, Gamma correction, and Retinex enhances visibility. A two-stage filtering process, including spatial-domain (Gaussian, Bilateral, Guided) and frequency-domain (Fourier, Wavelet) methods, effectively reduces noise while preserving details. To correct color distortion, an adaptive color compensation (ACC) model estimates spectral attenuation and water type to combine RCP, DCP, and MUDCP dynamically. Finally, a perceptually guided color balance mechanism ensures natural color restoration. Experimental results on benchmark datasets demonstrate superior performance over state-of-the-art methods in contrast enhancement, color correction, and structural preservation, making the framework robust for underwater imaging applications.
Abstract (translated)
水下光学成像受到光吸收、散射和色彩失真的严重影响，这阻碍了可见度和准确的图像分析。本文提出了一种自适应增强框架，该框架集成了光照补偿、多域过滤和动态颜色校正。一种结合CLAHE（对比限制自适应直方图均衡）、Gamma矫正和Retinex技术的混合光照补偿策略增强了可见性。两阶段滤波过程包括空间领域（高斯、双边、导向）和频率领域（傅里叶、小波）方法，有效地减少噪声同时保留细节。为了校正颜色失真，自适应色彩补偿(ACC)模型估计光谱衰减并确定水体类型，从而动态结合RCP（反射色偏置）、DCP（直接颜色偏置）和MUDCP（多域统一动态色彩处理）。最后，感知导向的色彩平衡机制确保了自然的颜色恢复。在基准数据集上的实验结果表明，该框架在对比度增强、颜色校正和结构保留方面优于现有的最先进的方法，使其适用于水下成像应用，并表现出色的鲁棒性。
URL
https://arxiv.org/abs/2503.03640
PDF
-
ERetinex: Event Camera Meets Retinex Theory for Low-Light Image Enhancement
2025-03-04 10:48:44Xuejian Guo, Zhiqiang Tian, Yuehang Wang, Siqi Li, Yu Jiang, Shaoyi Du, Yue GaoAbstract
Low-light image enhancement aims to restore the under-exposure image captured in dark scenarios. Under such scenarios, traditional frame-based cameras may fail to capture the structure and color information due to the exposure time limitation. Event cameras are bio-inspired vision sensors that respond to pixel-wise brightness changes asynchronously. Event cameras' high dynamic range is pivotal for visual perception in extreme low-light scenarios, surpassing traditional cameras and enabling applications in challenging dark environments. In this paper, inspired by the success of the retinex theory for traditional frame-based low-light image restoration, we introduce the first methods that combine the retinex theory with event cameras and propose a novel retinex-based low-light image restoration framework named ERetinex. Among our contributions, the first is developing a new approach that leverages the high temporal resolution data from event cameras with traditional image information to estimate scene illumination accurately. This method outperforms traditional image-only techniques, especially in low-light environments, by providing more precise lighting information. Additionally, we propose an effective fusion strategy that combines the high dynamic range data from event cameras with the color information of traditional images to enhance image quality. Through this fusion, we can generate clearer and more detail-rich images, maintaining the integrity of visual information even under extreme lighting conditions. The experimental results indicate that our proposed method outperforms state-of-the-art (SOTA) methods, achieving a gain of 1.0613 dB in PSNR while reducing FLOPS by \textbf{84.28}\%.
Abstract (translated)
低光图像增强旨在恢复在黑暗场景中拍摄的曝光不足的图像。在这种情况下，传统的帧基摄像头由于受限于曝光时间而可能无法捕捉到结构和颜色信息。事件相机是一种仿生视觉传感器，能异步响应像素亮度变化，并具有高动态范围，在极端低光照条件下对于视觉感知至关重要，超过了传统相机的表现能力，从而能够应用于挑战性的黑暗环境。 在本文中，受Retinex理论成功用于传统帧基低光图像恢复的启发，我们首次将Retinex理论与事件相机结合，并提出了一种名为ERetinex的新框架。我们的贡献之一是开发了一种新方法，利用来自事件相机的高时间分辨率数据和传统的图像信息来准确估计场景照明。这种方法在低光照环境下超越了传统仅基于图像的技术，提供了更精确的照明信息。 此外，我们还提出了一种有效的融合策略，将事件相机提供的高动态范围数据与传统图像的颜色信息相结合以增强图像质量。通过这种融合方法，我们可以生成更为清晰和细节丰富的图像，在极端光线条件下也能保持视觉信息的完整性。 实验结果表明，我们的方法超越了现有最先进的（SOTA）技术，在PSNR上提高了1.0613分，同时将FLOPS减少了84.28%。
URL
https://arxiv.org/abs/2503.02484
PDF
-
Self-supervision via Controlled Transformation and Unpaired Self-conditioning for Low-light Image Enhancement
2025-03-01 22:25:49Aupendu Kar, Sobhan K. Dhara, Debashis Sen, Prabir K. BiswasAbstract
Real-world low-light images captured by imaging devices suffer from poor visibility and require a domain-specific enhancement to produce artifact-free outputs that reveal details. In this paper, we propose an unpaired low-light image enhancement network leveraging novel controlled transformation-based self-supervision and unpaired self-conditioning strategies. The model determines the required degrees of enhancement at the input image pixels, which are learned from the unpaired low-lit and well-lit images without any direct supervision. The self-supervision is based on a controlled transformation of the input image and subsequent maintenance of its enhancement in spite of the transformation. The self-conditioning performs training of the model on unpaired images such that it does not enhance an already-enhanced image or a well-lit input image. The inherent noise in the input low-light images is handled by employing low gradient magnitude suppression in a detail-preserving manner. In addition, our noise handling is self-conditioned by preventing the denoising of noise-free well-lit images. The training based on low-light image enhancement-specific attributes allows our model to avoid paired supervision without compromising significantly in performance. While our proposed self-supervision aids consistent enhancement, our novel self-conditioning facilitates adequate enhancement. Extensive experiments on multiple standard datasets demonstrate that our model, in general, outperforms the state-of-the-art both quantitatively and subjectively. Ablation studies show the effectiveness of our self-supervision and self-conditioning strategies, and the related loss functions.
Abstract (translated)
现实世界中，成像设备捕获的低光图像由于可见度差而需要特定领域的增强处理，以生成无伪影且揭示细节的输出。本文提出了一种利用新颖的基于受控变换的自监督和未配对自我条件策略的低光图像增强网络。该模型根据从未经配对的昏暗图像与良好照明图像中学习到的信息来确定输入图像像素所需的增强程度，而无需任何直接监督。这种自监督基于输入图像的受控转换，并随后在转换后保持其增强效果。自我条件化策略则通过训练模型以不增强已经过增强或光照良好的输入图像的方式，在未配对图像上进行训练。 输入低光图像中的固有噪声被采用细节保留方式的低梯度幅度抑制来处理，同时我们的噪声处理是自适应的，可以防止去噪无噪声的良好照明图像。基于低光图像增强特定属性的训练使得我们的模型能够在不显著降低性能的情况下避免配对监督。虽然我们提出的自监督有助于持续一致地进行增强，但新颖的自我条件化策略则促进了适当的增强。 在多个标准数据集上的广泛实验表明，总体而言，我们的模型无论是定量还是定性评估都优于现有技术。消融研究表明了我们自监督和自我条件化策略以及相关损失函数的有效性。
URL
https://arxiv.org/abs/2503.00642
PDF
-
Flow Matching for Medical Image Synthesis: Bridging the Gap Between Speed and Quality
2025-03-01 00:49:47Milad Yazdani, Yasamin Medghalchi, Pooria Ashrafian, Ilker Hacihaliloglu, Dena ShahriariAbstract
Deep learning models have emerged as a powerful tool for various medical applications. However, their success depends on large, high-quality datasets that are challenging to obtain due to privacy concerns and costly annotation. Generative models, such as diffusion models, offer a potential solution by synthesizing medical images, but their practical adoption is hindered by long inference times. In this paper, we propose the use of an optimal transport flow matching approach to accelerate image generation. By introducing a straighter mapping between the source and target distribution, our method significantly reduces inference time while preserving and further enhancing the quality of the outputs. Furthermore, this approach is highly adaptable, supporting various medical imaging modalities, conditioning mechanisms (such as class labels and masks), and different spatial dimensions, including 2D and 3D. Beyond image generation, it can also be applied to related tasks such as image enhancement. Our results demonstrate the efficiency and versatility of this framework, making it a promising advancement for medical imaging applications. Code with checkpoints and a synthetic dataset (beneficial for classification and segmentation) is now available on: this https URL.
Abstract (translated)
深度学习模型已成为各种医学应用的强大工具。然而，它们的成功依赖于大规模、高质量的数据集，由于隐私问题和昂贵的标注成本，这类数据集难以获得。生成模型（如扩散模型）通过合成医学图像提供了一种潜在解决方案，但其实际采用受到长推理时间的限制。在本文中，我们提出使用最优传输流匹配方法来加速图像生成过程。通过引入源分布与目标分布之间更直接的映射关系，我们的方法显著减少了推理时间，同时保持并进一步增强了输出的质量。 此外，该方法具有高度的适应性，支持各种医学成像模式、条件机制（如类别标签和掩码）以及不同空间维度（包括2D和3D）。除了图像生成之外，它还可以应用于诸如图像增强等相关任务。我们的结果展示了此框架在效率和多功能性方面的优势，使其成为医学影像应用中的一个有前景的发展方向。 代码及相关检查点与合成数据集（有利于分类和分割）现已公开于：[这个链接](this https URL)。
URL
https://arxiv.org/abs/2503.00266
PDF
-
SEE: See Everything Every Time -- Adaptive Brightness Adjustment for Broad Light Range Images via Events
2025-02-28 14:55:37Yunfan Lu, Xiaogang Xu, Hao Lu, Yanlin Qian, Pengteng Li, Huizai Yao, Bin Yang, Junyi Li, Qianyi Cai, Weiyu Guo, Hui XiongAbstract
Event cameras, with a high dynamic range exceeding $120dB$, significantly outperform traditional embedded cameras, robustly recording detailed changing information under various lighting conditions, including both low- and high-light situations. However, recent research on utilizing event data has primarily focused on low-light image enhancement, neglecting image enhancement and brightness adjustment across a broader range of lighting conditions, such as normal or high illumination. Based on this, we propose a novel research question: how to employ events to enhance and adaptively adjust the brightness of images captured under broad lighting conditions? To investigate this question, we first collected a new dataset, SEE-600K, consisting of 610,126 images and corresponding events across 202 scenarios, each featuring an average of four lighting conditions with over a 1000-fold variation in illumination. Subsequently, we propose a framework that effectively utilizes events to smoothly adjust image brightness through the use of prompts. Our framework captures color through sensor patterns, uses cross-attention to model events as a brightness dictionary, and adjusts the image's dynamic range to form a broad light-range representation (BLR), which is then decoded at the pixel level based on the brightness prompt. Experimental results demonstrate that our method not only performs well on the low-light enhancement dataset but also shows robust performance on broader light-range image enhancement using the SEE-600K dataset. Additionally, our approach enables pixel-level brightness adjustment, providing flexibility for post-processing and inspiring more imaging applications. The dataset and source code are publicly available at:this https URL.
Abstract (translated)
事件相机，其动态范围超过$120dB$，在各种光照条件下（包括低光和高光情况）记录详细变化信息方面远超传统嵌入式摄像头。然而，最近利用事件数据的研究主要集中在低光图像增强上，忽略了在整个更广泛的照明条件下的图像增强与亮度调节，如正常或高强度光线环境。基于此，我们提出一个新的研究问题：如何使用事件来增强并自适应地调整不同光照条件下捕捉到的图像亮度？为了解决这个问题，我们首先收集了一个新的数据集SEE-600K，该数据集中包含610,126张图像及其对应事件，覆盖了202个场景，每个场景平均包括四种照明条件，且光照变化范围超过千倍。随后，我们提出了一种框架，通过使用提示词有效利用事件来平滑地调整图像亮度。我们的框架通过传感器模式捕捉颜色，使用交叉注意力机制将事件建模为一个亮度字典，并调节图像的动态范围以形成广域光级表示（BLR），然后根据亮度提示在像素级别解码该表示。 实验结果显示，我们提出的方法不仅在低光照增强数据集上表现出色，在利用SEE-600K数据集进行广泛照明条件下的图像增强时也展现了强大的性能。此外，我们的方法还提供了像素级别的亮度调节功能，为后期处理提供灵活性，并激发了更多的成像应用可能性。该数据集和源代码可在此处公开获取：[此链接指向发布数据集和代码的网页]。
URL
https://arxiv.org/abs/2502.21120
PDF
-
HVI: A New color space for Low-light Image Enhancement
2025-02-27 16:59:51Qingsen Yan, Yixu Feng, Cheng Zhang, Guansong Pang, Kangbiao Shi, Peng Wu, Wei Dong, Jinqiu Sun, Yanning ZhangAbstract
Low-Light Image Enhancement (LLIE) is a crucial computer vision task that aims to restore detailed visual information from corrupted low-light images. Many existing LLIE methods are based on standard RGB (sRGB) space, which often produce color bias and brightness artifacts due to inherent high color sensitivity in sRGB. While converting the images using Hue, Saturation and Value (HSV) color space helps resolve the brightness issue, it introduces significant red and black noise artifacts. To address this issue, we propose a new color space for LLIE, namely Horizontal/Vertical-Intensity (HVI), defined by polarized HS maps and learnable intensity. The former enforces small distances for red coordinates to remove the red artifacts, while the latter compresses the low-light regions to remove the black artifacts. To fully leverage the chromatic and intensity information, a novel Color and Intensity Decoupling Network (CIDNet) is further introduced to learn accurate photometric mapping function under different lighting conditions in the HVI space. Comprehensive results from benchmark and ablation experiments show that the proposed HVI color space with CIDNet outperforms the state-of-the-art methods on 10 datasets. The code is available at this https URL.
Abstract (translated)
低光图像增强（LLIE）是一项重要的计算机视觉任务，旨在从受污染的低光照图像中恢复详细的视觉信息。许多现有的LLIE方法基于标准RGB（sRGB）空间，由于sRGB固有的高色彩敏感性，在此空间内生成的颜色偏差和亮度伪影较为常见。尽管使用色调、饱和度和值（HSV）颜色空间转换图像有助于解决亮度问题，但会引入显著的红色和黑色噪点伪影。为了解决这个问题，我们提出了一种新的LLIE专用颜色空间——水平/垂直强度（HVI），它由极化HS图和可学习强度定义。前者通过确保红色坐标的短距离来消除红色伪影，而后者则通过压缩低光照区域来移除黑色噪点。 为了充分利用色度和强度信息，我们进一步引入了一种新的网络——颜色与强度解耦网络（CIDNet），用于在HVI空间下学习不同照明条件下的准确光度映射函数。基准测试和消融实验的全面结果显示，在10个数据集上，所提出的带有CIDNet的HVI颜色空间优于当前最先进的方法。代码可在提供的链接中获取。
URL
https://arxiv.org/abs/2502.20272
PDF
-
Night-Voyager: Consistent and Efficient Nocturnal Vision-Aided State Estimation in Object Maps
2025-02-27 12:53:58Tianxiao Gao, Mingle Zhao, Chengzhong Xu, Hui KongAbstract
Accurate and robust state estimation at nighttime is essential for autonomous robotic navigation to achieve nocturnal or round-the-clock tasks. An intuitive question arises: Can low-cost standard cameras be exploited for nocturnal state estimation? Regrettably, most existing visual methods may fail under adverse illumination conditions, even with active lighting or image enhancement. A pivotal insight, however, is that streetlights in most urban scenarios act as stable and salient prior visual cues at night, reminiscent of stars in deep space aiding spacecraft voyage in interstellar navigation. Inspired by this, we propose Night-Voyager, an object-level nocturnal vision-aided state estimation framework that leverages prior object maps and keypoints for versatile localization. We also find that the primary limitation of conventional visual methods under poor lighting conditions stems from the reliance on pixel-level metrics. In contrast, metric-agnostic, non-pixel-level object detection serves as a bridge between pixel-level and object-level spaces, enabling effective propagation and utilization of object map information within the system. Night-Voyager begins with a fast initialization to solve the global localization problem. By employing an effective two-stage cross-modal data association, the system delivers globally consistent state updates using map-based observations. To address the challenge of significant uncertainties in visual observations at night, a novel matrix Lie group formulation and a feature-decoupled multi-state invariant filter are introduced, ensuring consistent and efficient estimation. Through comprehensive experiments in both simulation and diverse real-world scenarios (spanning approximately 12.3 km), Night-Voyager showcases its efficacy, robustness, and efficiency, filling a critical gap in nocturnal vision-aided state estimation.
Abstract (translated)
夜间精确且稳健的状态估计对于自主机器人导航完成夜行或全天候任务至关重要。一个直观的问题随之而来：低成本的标准摄像头能否用于夜间状态估算？遗憾的是，大多数现有的视觉方法在不利的照明条件下可能会失效，即使使用主动照明或图像增强也不例外。然而，一项关键见解是，在多数城市场景中，街灯作为稳定的、显著的先验视觉线索，在夜晚的作用类似于深空中的星星帮助航天器进行星际导航。受此启发，我们提出了Night-Voyager框架——一种基于对象级别的夜间辅助视觉状态估计框架，利用了先验物体地图和关键点以实现多样化的定位功能。我们还发现，传统的视觉方法在照明条件较差时的主要限制在于依赖像素级的度量标准。相比之下，非像素级、无度量感知的对象检测充当了像素级与对象级空间之间的桥梁，使得系统能够有效传播并利用物体地图信息。 Night-Voyager框架以快速初始化开始，用于解决全局定位问题。通过采用有效的双阶段跨模态数据关联技术，该系统能够在基于地图的观测下提供一致的状态更新。为了解决夜间视觉观察中存在显著不确定性的挑战，引入了一种新颖的矩阵李群形式化和特征解耦的多状态不变滤波器，确保了估计的一致性和效率。 通过在模拟和多样化的真实场景（总计约12.3公里）中的全面实验，Night-Voyager展示了其在夜间视觉辅助状态估算方面的有效性、稳健性及高效性，填补了该领域的一个重要空白。
URL
https://arxiv.org/abs/2502.20054
PDF
-
Striving for Faster and Better: A One-Layer Architecture with Auto Re-parameterization for Low-Light Image Enhancement
2025-02-27 08:20:03Nan An, Long Ma, Guangchao Han, Xin Fan, RIsheng LiuAbstract
Deep learning-based low-light image enhancers have made significant progress in recent years, with a trend towards achieving satisfactory visual quality while gradually reducing the number of parameters and improving computational efficiency. In this work, we aim to delving into the limits of image enhancers both from visual quality and computational efficiency, while striving for both better performance and faster processing. To be concrete, by rethinking the task demands, we build an explicit connection, i.e., visual quality and computational efficiency are corresponding to model learning and structure design, respectively. Around this connection, we enlarge parameter space by introducing the re-parameterization for ample model learning of a pre-defined minimalist network (e.g., just one layer), to avoid falling into a local solution. To strengthen the structural representation, we define a hierarchical search scheme for discovering a task-oriented re-parameterized structure, which also provides powerful support for efficiency. Ultimately, this achieves efficient low-light image enhancement using only a single convolutional layer, while maintaining excellent visual quality. Experimental results show our sensible superiority both in quality and efficiency against recently-proposed methods. Especially, our running time on various platforms (e.g., CPU, GPU, NPU, DSP) consistently moves beyond the existing fastest scheme. The source code will be released at this https URL.
Abstract (translated)
基于深度学习的低光图像增强技术近年来取得了显著进展，尤其是在实现令人满意的视觉质量的同时逐渐减少参数数量并提高计算效率方面。在本项工作中，我们旨在深入探究图像增强器在视觉质量和计算效率方面的极限，并努力同时提升性能和加快处理速度。具体而言，通过重新思考任务需求，我们建立了明确的联系：即视觉质量和计算效率分别对应于模型学习和结构设计。围绕这一联系，我们通过引入重参数化来扩大预定义极简网络（例如仅一层）的学习空间，以避免陷入局部最优解，并为充分利用该预定义网络提供条件。为了加强结构表示能力，我们定义了分层搜索方案，用于发现面向任务的重新参数化结构，这同时也为效率提供了强大支持。最终，在保持出色视觉质量的前提下，使用单一卷积层实现了高效的低光图像增强。 实验结果表明，在质量和效率方面，我们的方法相对于最近提出的其他方法具有明显的优越性。特别是在各种平台（如CPU、GPU、NPU和DSP）上的运行时间上，我们超越了现有的最快方案。源代码将在此网址发布：[此链接为示例占位符，请替换为实际的URL]。 请注意，在提供具体链接时，确保使用有效的URL，并且根据实际情况填写正确的网站或GitHub仓库地址。
URL
https://arxiv.org/abs/2502.19867
PDF
-
CLIP-Optimized Multimodal Image Enhancement via ISP-CNN Fusion for Coal Mine IoVT under Uneven Illumination
2025-02-26 05:09:40Shuai Wang, Shihao Zhang, Jiaqi Wu, Zijian Tian, Wei Chen, Tongzhu Jin, Miaomiao Xue, Zehua Wang, Fei Richard Yu, Victor C. M. LeungAbstract
Clear monitoring images are crucial for the safe operation of coal mine Internet of Video Things (IoVT) systems. However, low illumination and uneven brightness in underground environments significantly degrade image quality, posing challenges for enhancement methods that often rely on difficult-to-obtain paired reference images. Additionally, there is a trade-off between enhancement performance and computational efficiency on edge devices within IoVT this http URL address these issues, we propose a multimodal image enhancement method tailored for coal mine IoVT, utilizing an ISP-CNN fusion architecture optimized for uneven illumination. This two-stage strategy combines global enhancement with detail optimization, effectively improving image quality, especially in poorly lit areas. A CLIP-based multimodal iterative optimization allows for unsupervised training of the enhancement algorithm. By integrating traditional image signal processing (ISP) with convolutional neural networks (CNN), our approach reduces computational complexity while maintaining high performance, making it suitable for real-time deployment on edge this http URL results demonstrate that our method effectively mitigates uneven brightness and enhances key image quality metrics, with PSNR improvements of 2.9%-4.9%, SSIM by 4.3%-11.4%, and VIF by 4.9%-17.8% compared to seven state-of-the-art algorithms. Simulated coal mine monitoring scenarios validate our method's ability to balance performance and computational demands, facilitating real-time enhancement and supporting safer mining operations.
Abstract (translated)
清晰的监控图像对于煤矿物联网（IoVT）系统的安全运行至关重要。然而，地下环境中光线不足和亮度不均会显著降低图像质量，这对通常依赖于难以获取的配对参考图像进行增强的方法提出了挑战。此外，在IoVT设备中还存在着性能提升与计算效率之间的权衡问题。为了解决这些问题，我们提出了一种专门针对煤矿IoVT的多模态图像增强方法，并采用了一个优化了不均匀照明条件下的ISP-CNN融合架构。这一两阶段策略结合了全局增强和细节优化，有效提升了图像质量，尤其是在光线不足的情况下。 基于CLIP（Contrastive Language–Image Pre-training）的多模态迭代优化使我们能够在没有监督训练数据的情况下对增强算法进行培训。通过将传统的图像信号处理技术（ISP）与卷积神经网络（CNN）相结合，我们的方法在保持高性能的同时减少了计算复杂性，使其适用于边缘设备上的实时部署。 实验结果表明，与七种最先进的算法相比，我们的方法能够有效缓解亮度不均，并提高了关键的图像质量指标：PSNR（峰值信噪比）提升了2.9%-4.9%，SSIM（结构相似度指数）提升了4.3%-11.4%，VIF（视觉信息量）提升了4.9%-17.8%。模拟煤矿监控场景验证了我们的方法能够在性能和计算需求之间找到平衡，从而支持实时增强并促进更安全的采矿作业。
URL
https://arxiv.org/abs/2502.19450
PDF
-
Improved Partial Differential Equation and Fast Approximation Algorithm for Hazy/Underwater/Dust Storm Image Enhancement
2025-02-21 22:44:12Uche A. NnolimAbstract
This paper presents an improved and modified partial differential equation (PDE)-based de-hazing algorithm. The proposed method combines logarithmic image processing models in a PDE formulation refined with linear filter-based operators in either spatial or frequency domain. Additionally, a fast, simplified de-hazing function approximation of the hazy image formation model is developed in combination with fuzzy homomorphic refinement. The proposed algorithm solves the problem of image darkening and over-enhancement of edges in addition to enhancement of dark image regions encountered in previous formulations. This is in addition to avoiding enhancement of sky regions in de-hazed images while avoiding halo effect. Furthermore, the proposed algorithm is utilized for underwater and dust storm image enhancement with the incorporation of a modified global contrast enhancement algorithm. Experimental comparisons indicate that the proposed approach surpasses a majority of the algorithms from the literature based on quantitative image quality metrics.
Abstract (translated)
本文提出了一种改进和修改的基于偏微分方程（PDE）的去雾算法。所提方法结合了对数图像处理模型，并在空间或频域中使用线性滤波器操作符进行了优化，以形成PDE公式。此外，还开发了一种快速简化的模糊同态精炼组合式去雾函数近似，用于模拟有雾图像的生成过程。该算法解决了以前方法中存在的图像变暗和边缘过度增强的问题，并且在去除雾霾的同时避免了天空区域的过度增强以及晕轮效应。另外，在结合改进的全局对比度增强算法后，所提算法也被应用于水下和沙尘暴图像的优化处理中。实验比较表明，根据定量图像质量指标，该方法优于大多数文献中的算法。
URL
https://arxiv.org/abs/2502.15986
PDF
-
LUMINA-Net: Low-light Upgrade through Multi-stage Illumination and Noise Adaptation Network for Image Enhancement
2025-02-21 03:37:58Namrah Siddiqua, Kim SuneungAbstract
Low-light image enhancement (LLIE) is a crucial task in computer vision aimed to enhance the visual fidelity of images captured under low-illumination conditions. Conventional methods frequently struggle to mitigate pervasive shortcomings such as noise, over-exposure, and color distortion thereby precipitating a pronounced degradation in image quality. To address these challenges, we propose LUMINA-Net an advanced deep learning framework designed specifically by integrating multi-stage illumination and reflectance modules. First, the illumination module intelligently adjusts brightness and contrast levels while meticulously preserving intricate textural details. Second, the reflectance module incorporates a noise reduction mechanism that leverages spatial attention and channel-wise feature refinement to mitigate noise contamination. Through a comprehensive suite of experiments conducted on LOL and SICE datasets using PSNR, SSIM and LPIPS metrics, surpassing state-of-the-art methodologies and showcasing its efficacy in low-light image enhancement.
Abstract (translated)
低光图像增强（LLIE）是计算机视觉中的一个关键任务，旨在提升在低光照条件下捕获的图像的视觉保真度。传统方法经常难以解决诸如噪声、过度曝光和色彩失真的普遍问题，导致图像质量显著下降。为了解决这些挑战，我们提出了LUMINA-Net，这是一种先进的深度学习框架，通过整合多阶段照明模块与反射模块而专门设计。 首先，照明模块智能地调整亮度和对比度水平，同时精细保留复杂的纹理细节。其次，反射模块引入了一种噪声减少机制，该机制利用空间注意力和通道级特征精炼来减轻噪声污染。通过对LOL和SICE数据集进行一系列全面的实验，并使用PSNR、SSIM和LPIPS指标评估，我们的方法超越了现有的最佳方法，展示了在低光图像增强中的有效性。 此段落介绍了LUMINA-Net框架如何通过综合运用照明与反射模块来改进传统LLIE技术，同时强调了其在多项标准测试中表现出的优越性能。
URL
https://arxiv.org/abs/2502.15186
PDF
-
Optimized Pap Smear Image Enhancement: Hybrid PMD Filter-CLAHE Using Spider Monkey Optimization
2025-02-21 02:34:57Ach Khozaimi, Isnani Darti, Syaiful Anam, Wuryansari Muharini KusumawinahyuAbstract
Pap smear image quality is crucial for cervical cancer detection. This study introduces an optimized hybrid approach that combines the Perona-Malik Diffusion (PMD) filter with contrast-limited adaptive histogram equalization (CLAHE) to enhance Pap smear image quality. The PMD filter reduces the image noise, whereas CLAHE improves the image contrast. The hybrid method was optimized using spider monkey optimization (SMO PMD-CLAHE). BRISQUE and CEIQ are the new objective functions for the PMD filter and CLAHE optimization, respectively. The simulations were conducted using the SIPaKMeD dataset. The results indicate that SMO outperforms state-of-the-art methods in optimizing the PMD filter and CLAHE. The proposed method achieved an average effective measure of enhancement (EME) of 5.45, root mean square (RMS) contrast of 60.45, Michelson's contrast (MC) of 0.995, and entropy of 6.80. This approach offers a new perspective for improving Pap smear image quality.
Abstract (translated)
宫颈癌检测中，巴氏涂片图像的质量至关重要。本研究提出了一种优化的混合方法，结合了佩罗纳-马尔基（Perona-Malik Diffusion, PMD）滤波器和对比度受限自适应直方图均衡化（Contrast-Limited Adaptive Histogram Equalization, CLAHE），以提高巴氏涂片图像的质量。PMD滤波器减少图像噪声，而CLAHE则增强图像对比度。该混合方法使用蜘蛛猴优化算法（SMO PMD-CLAHE）进行了优化。BRISQUE和CEIQ分别是用于PMD滤波器和CLAHE优化的新目标函数。实验数据集采用的是SIPaKMeD数据集。 结果显示，SMO在优化PMD滤波器和CLAHE方面优于现有的先进技术。所提出的方法实现了平均有效增强量（Effective Measure of Enhancement, EME）为5.45，均方根对比度（Root Mean Square Contrast, RMS）为60.45，米氏对比度（Michelson's Contrast, MC）为0.995，熵值为6.80。这种方法为提高巴氏涂片图像质量提供了一种新的视角。
URL
https://arxiv.org/abs/2502.15156
PDF
-
EyeBench: A Call for More Rigorous Evaluation of Retinal Image Enhancement
2025-02-20 04:56:03Wenhui Zhu, Xuanzhao Dong, Xin Li, Yujian Xiong, Xiwen Chen, Peijie Qiu, Vamsi Krishna Vasa, Zhangsihao Yang, Yi Su, Oana Dumitrascu, Yalin WangAbstract
Over the past decade, generative models have achieved significant success in enhancement fundus this http URL, the evaluation of these models still presents a considerable challenge. A comprehensive evaluation benchmark for fundus image enhancement is indispensable for three main reasons: 1) The existing denoising metrics (e.g., PSNR, SSIM) are hardly to extend to downstream real-world clinical research (e.g., Vessel morphology consistency). 2) There is a lack of comprehensive evaluation for both paired and unpaired enhancement methods, along with the need for expert protocols to accurately assess clinical value. 3) An ideal evaluation system should provide insights to inform future developments of fundus image enhancement. To this end, we propose a novel comprehensive benchmark, EyeBench, to provide insights that align enhancement models with clinical needs, offering a foundation for future work to improve the clinical relevance and applicability of generative models for fundus image enhancement. EyeBench has three appealing properties: 1) multi-dimensional clinical alignment downstream evaluation: In addition to evaluating the enhancement task, we provide several clinically significant downstream tasks for fundus images, including vessel segmentation, DR grading, denoising generalization, and lesion segmentation. 2) Medical expert-guided evaluation design: We introduce a novel dataset that promote comprehensive and fair comparisons between paired and unpaired methods and includes a manual evaluation protocol by medical experts. 3) Valuable insights: Our benchmark study provides a comprehensive and rigorous evaluation of existing methods across different downstream tasks, assisting medical experts in making informed choices. Additionally, we offer further analysis of the challenges faced by existing methods. The code is available at \url{this https URL}
Abstract (translated)
过去十年里，生成模型在眼底图像增强方面取得了显著的成功。然而，在评估这些模型时仍然面临着重大挑战。一个全面的基准测试对于眼底图像增强来说是必不可少的，原因有三：1）现有的去噪指标（如PSNR、SSIM等）难以扩展到下游的实际临床研究中（例如血管形态的一致性）。2）目前缺乏对配对和非配对增强方法进行全面评估的方法，并且需要专家制定准确衡量临床价值的标准。3）理想的评估系统应该能为未来的眼底图像增强生成模型的发展提供见解。 为此，我们提出了一个新型的全面基准测试EyeBench，旨在使增强模型更好地与临床需求相匹配，为将来改进眼底图像增强生成模型的临床相关性和应用性奠定基础。EyeBench具有三大优势： 1. 多维度下游任务的临床对齐评估：除了对增强任务进行评价外，我们还提供了多个重要的下游任务，包括血管分割、糖尿病视网膜病变分级、去噪泛化和病灶分割等。 2. 医学专家引导下的评估设计：引入了一个新的数据集以促进配对与非配对方法之间的全面且公平比较，并包括由医学专家制定的手动评价协议。 3. 有价值的见解：我们的基准测试研究提供了现有方法在不同下游任务中的综合和严格的评估，帮助医学专家做出知情选择。此外，我们还分析了当前方法面临的挑战。 代码可以在\url{此链接}获得。
URL
https://arxiv.org/abs/2502.14260
PDF
