Polarization images facilitate image enhancement and 3D reconstruction tasks, but the limited accessibility of polarization cameras hinders their broader application. This gap drives the need for synthesizing photorealistic polarization this http URL existing polarization simulator Mitsuba relies on a parametric polarization image formation model and requires extensive 3D assets covering shape and PBR materials, preventing it from generating large-scale photorealistic images. To address this problem, we propose PolarAnything, capable of synthesizing polarization images from a single RGB input with both photorealism and physical accuracy, eliminating the dependency on 3D asset collections. Drawing inspiration from the zero-shot performance of pretrained diffusion models, we introduce a diffusion-based generative framework with an effective representation strategy that preserves the fidelity of polarization properties. Experiments show that our model generates high-quality polarization images and supports downstream tasks like shape from polarization.
偏振图像促进了图像增强和三维重建等任务,但极化相机的有限可访问性阻碍了它们更广泛的应用。这种差距推动了合成逼真的极化图像的需求。现有的极化模拟器Mitsuba依赖于一个参数化的极化图像生成模型,并需要大量的3D资产(包括形状和物理基础渲染材料),这使其无法生成大规模的逼真图像。为了解决这个问题,我们提出了PolarAnything,它可以从单个RGB输入合成具有逼真度和物理准确性的偏振图像,从而消除对3D资源集的需求。 受到预训练扩散模型零样本性能的启发,我们引入了一个基于扩散的生成框架,并采用了一种有效的表示策略来保持极化属性的保真度。实验表明,我们的模型可以生成高质量的偏振图像并支持诸如从偏振中获取形状等下游任务。
https://arxiv.org/abs/2507.17268
Existing image contrast enhancement methods are typically designed for specific tasks such as under-/over-exposure correction, low-light and backlit image enhancement, etc. The learned models, however, exhibit poor generalization performance across different tasks, even across different datasets of a specific task. It is important to explore whether we can learn a universal and generalized model for various contrast enhancement tasks. In this work, we observe that the common key factor of these tasks lies in the need of exposure and contrast adjustment, which can be well-addressed if high-dynamic range (HDR) inputs are available. We hence collect 46,928 HDR raw images from public sources, and render 328,496 sRGB images to build multi-exposure sequences (MES) and the corresponding pseudo sRGB ground-truths via multi-exposure fusion. Consequently, we train a network to generate an MES from a single sRGB image, followed by training another network to fuse the generated MES into an enhanced image. Our proposed method, namely UNiversal Image Contrast Enhancer (UNICE), is free of costly human labeling. However, it demonstrates significantly stronger generalization performance than existing image contrast enhancement methods across and within different tasks, even outperforming manually created ground-truths in multiple no-reference image quality metrics. The dataset, code and model are available at this https URL.
现有的图像对比度增强方法通常针对特定任务(如欠曝/过曝校正、低光和逆光图像增强等)设计。然而,这些通过学习得到的模型在不同任务之间以及同一任务的不同数据集上的泛化性能较差。探索能否学习一个适用于各种对比度增强任务的通用且泛化的模型变得尤为重要。 在这项工作中,我们观察到这些任务中的共同关键因素在于需要调整曝光和对比度,如果能够使用高动态范围(HDR)输入,则可以很好地解决这一问题。因此,我们从公开来源收集了46,928张HDR原始图像,并生成了328,496张sRGB图像来构建多曝光序列(MES),并借助多曝光融合技术生成相应的伪sRGB地面实况数据。 接着,我们训练了一个网络以从单个sRGB图像中生成MES,随后又训练另一个网络将生成的MES融合成增强后的图像。我们的方法名为UNiversal Image Contrast Enhancer (UNICE),无需昂贵的人工标签,并且在不同任务之间以及同一任务的不同场景下,其泛化性能显著优于现有图像对比度增强方法,在多个无参考图像质量指标上甚至超过了手动创建的真实地面实况数据。 该数据集、代码和模型可在提供的URL处获取。
https://arxiv.org/abs/2507.17157
Recent Transformer-based low-light enhancement methods have made promising progress in recovering global illumination. However, they still struggle with non-uniform lighting scenarios, such as backlit and shadow, appearing as over-exposure or inadequate brightness restoration. To address this challenge, we present a Spatially-Adaptive Illumination-Guided Transformer (SAIGFormer) framework that enables accurate illumination restoration. Specifically, we propose a dynamic integral image representation to model the spatially-varying illumination, and further construct a novel Spatially-Adaptive Integral Illumination Estimator ($\text{SAI}^2\text{E}$). Moreover, we introduce an Illumination-Guided Multi-head Self-Attention (IG-MSA) mechanism, which leverages the illumination to calibrate the lightness-relevant features toward visual-pleased illumination enhancement. Extensive experiments on five standard low-light datasets and a cross-domain benchmark (LOL-Blur) demonstrate that our SAIGFormer significantly outperforms state-of-the-art methods in both quantitative and qualitative metrics. In particular, our method achieves superior performance in non-uniform illumination enhancement while exhibiting strong generalization capabilities across multiple datasets. Code is available at this https URL.
最近基于Transformer的低光增强方法在恢复全局照明方面取得了显著进展。然而,它们仍然难以应对非均匀光照场景(如背光和阴影),这些问题会导致过度曝光或亮度恢复不足。为了应对这一挑战,我们提出了一种空间自适应照明引导变换器(SAIGFormer)框架,以实现准确的光照恢复。具体而言,我们提出了动态积分图像表示来建模空间变化的光照,并进一步构建了一个新颖的空间自适应积分光照估计器($\text{SAI}^2\text{E}$)。此外,我们引入了一种照明引导多头自我注意机制(IG-MSA),该机制利用光照校准与亮度相关的特征以实现视觉愉悦的光照增强。在五个标准低光数据集和一个跨域基准测试(LOL-Blur)上的广泛实验表明,我们的SAIGFormer方法在量化指标和定性评价中都显著优于最先进的方法。特别地,我们提出的方法在非均匀照明增强方面表现出色,并且在多个数据集中展示了强大的泛化能力。代码可在该链接获取。
https://arxiv.org/abs/2507.15520
Underwater image enhancement is vital for marine conservation, particularly coral reef monitoring. However, AI-based enhancement models often face dataset bias, high computational costs, and lack of transparency, leading to potential misinterpretations. This paper introduces EBA-AI, an ethics-guided bias-aware AI framework to address these challenges. EBA-AI leverages CLIP embeddings to detect and mitigate dataset bias, ensuring balanced representation across varied underwater environments. It also integrates adaptive processing to optimize energy efficiency, significantly reducing GPU usage while maintaining competitive enhancement quality. Experiments on LSUI400, Oceanex, and UIEB100 show that while PSNR drops by a controlled 1.0 dB, computational savings enable real-time feasibility for large-scale marine monitoring. Additionally, uncertainty estimation and explainability techniques enhance trust in AI-driven environmental decisions. Comparisons with CycleGAN, FunIEGAN, RAUNENet, WaterNet, UGAN, PUGAN, and UTUIE validate EBA-AI's effectiveness in balancing efficiency, fairness, and interpretability in underwater image processing. By addressing key limitations of AI-driven enhancement, this work contributes to sustainable, bias-aware, and computationally efficient marine conservation efforts. For interactive visualizations, animations, source code, and access to the preprint, visit: this https URL
水下图像增强对于海洋保护,尤其是珊瑚礁监测至关重要。然而,基于AI的增强模型经常面临数据集偏差、高昂的计算成本和缺乏透明度的问题,这些问题可能导致误解。本文介绍了EBA-AI(伦理导向的认识偏差意识型人工智能框架),用于解决这些挑战。EBA-AI利用CLIP嵌入来检测和减轻数据集偏差,确保在各种水下环境中实现平衡表示。此外,它还集成了自适应处理以优化能源效率,在保持竞争力的增强质量的同时显著减少GPU使用量。在LSUI400、Oceanex和UIEB100上的实验表明,尽管峰值信噪比(PSNR)下降了可控的1.0分贝,但计算节省使得大规模海洋监测中的实时可行性成为可能。此外,不确定性估计和可解释性技术增强了对AI驱动环境决策的信任。与CycleGAN、FunIEGAN、RAUNENet、WaterNet、UGAN、PUGAN和UTUIE相比,EBA-AI在水下图像处理的效率、公平性和可解释性方面证明了其有效性。通过解决AI驱动增强的关键限制,这项工作为可持续、有意识偏差和计算高效的海洋保护努力做出了贡献。 对于交互式可视化、动画、源代码及预印本访问,请参见:[此链接](https://thislink.com)
https://arxiv.org/abs/2507.15036
Low-light images suffer from complex degradation, and existing enhancement methods often encode all degradation factors within a single latent space. This leads to highly entangled features and strong black-box characteristics, making the model prone to shortcut learning. To mitigate the above issues, this paper proposes a wavelet-based low-light stereo image enhancement method with feature space decoupling. Our insight comes from the following findings: (1) Wavelet transform enables the independent processing of low-frequency and high-frequency information. (2) Illumination adjustment can be achieved by adjusting the low-frequency component of a low-light image, extracted through multi-level wavelet decomposition. Thus, by using wavelet transform the feature space is decomposed into a low-frequency branch for illumination adjustment and multiple high-frequency branches for texture enhancement. Additionally, stereo low-light image enhancement can extract useful cues from another view to improve enhancement. To this end, we propose a novel high-frequency guided cross-view interaction module (HF-CIM) that operates within high-frequency branches rather than across the entire feature space, effectively extracting valuable image details from the other view. Furthermore, to enhance the high-frequency information, a detail and texture enhancement module (DTEM) is proposed based on cross-attention mechanism. The model is trained on a dataset consisting of images with uniform illumination and images with non-uniform illumination. Experimental results on both real and synthetic images indicate that our algorithm offers significant advantages in light adjustment while effectively recovering high-frequency information. The code and dataset are publicly available at: this https URL.
低光图像会遭受复杂的退化影响,而现有的增强方法通常在一个单一的潜在空间中编码所有退化因素。这导致了高度纠缠的功能特性及强烈的黑箱特征,使得模型容易陷入捷径学习。为了缓解上述问题,本文提出了一种基于小波变换的低光照立体图像增强方法,并通过解耦特征空间来解决这些问题。我们的见解来自以下发现: 1. 小波变换能够独立处理低频和高频信息。 2. 通过对多级小波分解提取出的低光图像中的低频分量进行调整,可以实现照明调节。 因此,利用小波变换将特征空间分解为一个用于照明调整的低频分支和多个用于纹理增强的高频分支。此外,立体低光照图像增强可以从另一个视角中提取有用的线索来改进增强效果。为此,我们提出了一种新颖的高频引导跨视图交互模块(HF-CIM),该模块在高频分支内而不是在整个特征空间上运行,有效地从另一视角中提取有价值的图像细节。 为了增强高频信息,还基于交叉注意力机制提出了一个细节和纹理增强模块(DTEM)。模型是在包含均匀照明和非均匀照明的图像数据集上进行训练的。实验证明,在真实和合成图像上的实验结果表明,我们的算法在光线调整方面具有显著优势,同时有效地恢复了高频信息。 代码和数据集公开获取链接:[此URL](this https URL)。
https://arxiv.org/abs/2507.12188
Deep learning-based bilateral grid processing has emerged as a promising solution for image enhancement, inherently encoding spatial and intensity information while enabling efficient full-resolution processing through slicing operations. However, existing approaches are limited to linear affine transformations, hindering their ability to model complex color relationships. Meanwhile, while multi-layer perceptrons (MLPs) excel at non-linear mappings, traditional MLP-based methods employ globally shared parameters, which is hard to deal with localized variations. To overcome these dual challenges, we propose a Bilateral Grid-based Pixel-Adaptive Multi-layer Perceptron (BPAM) framework. Our approach synergizes the spatial modeling of bilateral grids with the non-linear capabilities of MLPs. Specifically, we generate bilateral grids containing MLP parameters, where each pixel dynamically retrieves its unique transformation parameters and obtain a distinct MLP for color mapping based on spatial coordinates and intensity values. In addition, we propose a novel grid decomposition strategy that categorizes MLP parameters into distinct types stored in separate subgrids. Multi-channel guidance maps are used to extract category-specific parameters from corresponding subgrids, ensuring effective utilization of color information during slicing while guiding precise parameter generation. Extensive experiments on public datasets demonstrate that our method outperforms state-of-the-art methods in performance while maintaining real-time processing capabilities.
基于深度学习的双边网格处理已经成为图像增强的一个有前途的解决方案,它能够内在地编码空间和强度信息,并通过切片操作实现高效的全分辨率处理。然而,现有的方法仅限于线性仿射变换,这限制了它们建模复杂色彩关系的能力。同时,尽管多层感知机(MLPs)在非线性映射方面表现出色,但传统基于MLP的方法采用全局共享参数的方式难以应对局部变化。为克服这些双重挑战,我们提出了一种基于双边网格的像素自适应多层感知机(BPAM)框架。我们的方法将双边网格的空间建模能力和MLPs的非线性能力结合起来。具体来说,我们在生成的双边网格中包含MLP参数,每个像素根据其空间坐标和强度值动态检索独特的变换参数,并获得用于色彩映射的不同MLP。此外,我们提出了一种新颖的网格分解策略,将MLP参数分类为不同的类型并存储在单独的子网格中。多通道引导图被用来从相应的子网格中提取特定类别的参数,在切片过程中有效利用颜色信息的同时指导精确参数生成。在公共数据集上的广泛实验表明,我们的方法不仅在性能上超越了现有的最佳方法,还能保持实时处理能力。
https://arxiv.org/abs/2507.12135
The development of advanced packaging is essential in the semiconductor manufacturing industry. However, non-destructive testing (NDT) of advanced packaging becomes increasingly challenging due to the depth and complexity of the layers involved. In such a scenario, Magnetic field imaging (MFI) enables the imaging of magnetic fields generated by currents. For MFI to be effective in NDT, the magnetic fields must be converted into current density. This conversion has typically relied solely on a Fast Fourier Transform (FFT) for magnetic field inversion; however, the existing approach does not consider eddy current effects or image misalignment in the test setup. In this paper, we present a spatial-physics informed model (SPIM) designed for a 3D spiral sample scanned using Superconducting QUantum Interference Device (SQUID) microscopy. The SPIM encompasses three key components: i) magnetic image enhancement by aligning all the "sharp" wire field signals to mitigate the eddy current effect using both in-phase (I-channel) and quadrature-phase (Q-channel) images; (ii) magnetic image alignment that addresses skew effects caused by any misalignment of the scanning SQUID microscope relative to the wire segments; and (iii) an inversion method for converting magnetic fields to magnetic currents by integrating the Biot-Savart Law with FFT. The results show that the SPIM improves I-channel sharpness by 0.3% and reduces Q-channel sharpness by 25%. Also, we were able to remove rotational and skew misalignments of 0.30 in a real image. Overall, SPIM highlights the potential of combining spatial analysis with physics-driven models in practical applications.
先进封装技术在半导体制造业中至关重要,然而由于其层层深入且复杂度高,对先进封装进行无损检测(NDT)变得愈发困难。在这种情况下,磁场成像(MFI)能够实现由电流产生的磁场的成像。为了使MFI在NDT中有效,必须将这些磁场转换为电流密度。这一过程通常仅依赖于快速傅里叶变换(FFT)来进行磁场反演;然而,现有的方法并未考虑到涡流效应或测试装置中的图像错位问题。 本文介绍了一种专为使用超导量子干涉仪(SQUID)显微镜扫描三维螺旋样品设计的空间物理信息模型(SPIM)。该模型包括三个关键组成部分: 1. 通过同步对相位(I通道)和正交相位(Q通道)的“尖锐”线场信号进行校准,来增强磁场图像,以减轻涡流效应的影响。 2. 磁场图像校准,用于解决由于SQUID显微镜相对于导电线段位置错位所引起的偏斜问题。 3. 将毕奥-萨法尔定律与FFT结合使用,将磁场转换为磁电流的反演方法。 实验结果表明,SPIM使I通道的“尖锐度”提高了0.3%,同时减少了Q通道的“尖锐度”25%。此外,在真实图像中成功消除了0.30度的旋转和偏斜错位。总的来说,SPIM展示了结合空间分析与物理驱动模型在实际应用中的巨大潜力。
https://arxiv.org/abs/2507.11853
Traditional Low-Light Image Enhancement (LLIE) methods primarily focus on uniform brightness adjustment, often neglecting instance-level semantic information and the inherent characteristics of different features. To address these limitations, we propose CWNet (Causal Wavelet Network), a novel architecture that leverages wavelet transforms for causal reasoning. Specifically, our approach comprises two key components: 1) Inspired by the concept of intervention in causality, we adopt a causal reasoning perspective to reveal the underlying causal relationships in low-light enhancement. From a global perspective, we employ a metric learning strategy to ensure causal embeddings adhere to causal principles, separating them from non-causal confounding factors while focusing on the invariance of causal factors. At the local level, we introduce an instance-level CLIP semantic loss to precisely maintain causal factor consistency. 2) Based on our causal analysis, we present a wavelet transform-based backbone network that effectively optimizes the recovery of frequency information, ensuring precise enhancement tailored to the specific attributes of wavelet transforms. Extensive experiments demonstrate that CWNet significantly outperforms current state-of-the-art methods across multiple datasets, showcasing its robust performance across diverse scenes. Code is available at this https URL.
传统低光图像增强(LLIE)方法主要集中在均匀亮度调整上,往往忽视了实例级别的语义信息以及不同特征的固有特性。为了克服这些限制,我们提出了CWNet(因果小波网络),这是一种利用小波变换进行因果推理的新架构。 具体来说,我们的方法包含两个关键组件: 1. **因果推理解析**:借鉴因果关系中的干预概念,我们采用一种因果推理视角来揭示低光增强中潜在的因果关系。从全局角度来看,我们采用了度量学习策略以确保因果嵌入符合理论因果原则,并且能够将它们与非因果混淆因素区分开来,同时关注于因果因素的不变性。在局部层面,我们引入了一种实例级别的CLIP语义损失,用于精确保持因果因素的一致性。 2. **基于小波变换的骨干网络**:根据我们的因果分析结果,我们提出了一种基于小波变换的骨干网络,该网络能够有效地优化频率信息的恢复过程,并确保增强效果精准地符合特定的小波变换属性。 广泛的实验表明,在多个数据集上CWNet显著优于当前最先进的方法,展示了其在各种场景下的稳健性能。代码可在以下链接获取:[此URL](https://this-url.com)(请注意将"[此URL]"替换为实际的GitHub或项目页面地址)。
https://arxiv.org/abs/2507.10689
Robust Visual SLAM (vSLAM) is essential for autonomous systems operating in real-world environments, where challenges such as dynamic objects, low texture, and critically, varying illumination conditions often degrade performance. Existing feature-based SLAM systems rely on fixed front-end parameters, making them vulnerable to sudden lighting changes and unstable feature tracking. To address these challenges, we propose ``IRAF-SLAM'', an Illumination-Robust and Adaptive Feature-Culling front-end designed to enhance vSLAM resilience in complex and challenging environments. Our approach introduces: (1) an image enhancement scheme to preprocess and adjust image quality under varying lighting conditions; (2) an adaptive feature extraction mechanism that dynamically adjusts detection sensitivity based on image entropy, pixel intensity, and gradient analysis; and (3) a feature culling strategy that filters out unreliable feature points using density distribution analysis and a lighting impact factor. Comprehensive evaluations on the TUM-VI and European Robotics Challenge (EuRoC) datasets demonstrate that IRAF-SLAM significantly reduces tracking failures and achieves superior trajectory accuracy compared to state-of-the-art vSLAM methods under adverse illumination conditions. These results highlight the effectiveness of adaptive front-end strategies in improving vSLAM robustness without incurring significant computational overhead. The implementation of IRAF-SLAM is publicly available at https://thanhnguyencanh. this http URL.
鲁棒的视觉同步定位与地图构建(vSLAM)对于在真实世界环境中运行的自主系统至关重要。动态物体、低纹理以及最关键的是光照条件的变化,常常会降低其性能。现有的基于特征的SLAM系统依赖于固定的前端参数设置,这使得它们对突然的照明变化和不稳定的特征跟踪变得脆弱。 为了解决这些问题,我们提出了“IRAF-SLAM”,这是一种设计用于提升vSLAM在复杂且挑战性环境中稳健性的光照鲁棒性和自适应特征剔除前端。我们的方法包括: 1. 一种图像增强方案,用于预处理并根据不同的光照条件调整图像质量; 2. 自适应的特征提取机制,该机制可以根据图像熵、像素强度和梯度分析动态地调整检测敏感性; 3. 基于密度分布分析和照明影响因子过滤不可靠特征点的特征剔除策略。 我们在TUM-VI和欧洲机器人挑战赛(EuRoC)数据集上进行了全面评估,结果表明IRAF-SLAM在不良光照条件下显著减少了跟踪失败,并且其轨迹准确性优于最先进的vSLAM方法。这些结果突显了自适应前端策略在不增加显著计算开销的情况下提升vSLAM鲁棒性方面的有效性。 IRAF-SLAM的实现代码公开可用,网址为https://thanhnguyencanh.github.io/iraf-slam/。
https://arxiv.org/abs/2507.07752
Low-Light Image Enhancement (LLIE) aims to restore vivid content and details from corrupted low-light images. However, existing standard RGB (sRGB) color space-based LLIE methods often produce color bias and brightness artifacts due to the inherent high color sensitivity. While Hue, Saturation, and Value (HSV) color space can decouple brightness and color, it introduces significant red and black noise artifacts. To address this problem, we propose a new color space for LLIE, namely Horizontal/Vertical-Intensity (HVI), defined by the HV color map and learnable intensity. The HV color map enforces small distances for the red coordinates to remove red noise artifacts, while the learnable intensity compresses the low-light regions to remove black noise artifacts. Additionally, we introduce the Color and Intensity Decoupling Network+ (HVI-CIDNet+), built upon the HVI color space, to restore damaged content and mitigate color distortion in extremely dark regions. Specifically, HVI-CIDNet+ leverages abundant contextual and degraded knowledge extracted from low-light images using pre-trained vision-language models, integrated via a novel Prior-guided Attention Block (PAB). Within the PAB, latent semantic priors can promote content restoration, while degraded representations guide precise color correction, both particularly in extremely dark regions through the meticulously designed cross-attention fusion mechanism. Furthermore, we construct a Region Refinement Block that employs convolution for information-rich regions and self-attention for information-scarce regions, ensuring accurate brightness adjustments. Comprehensive results from benchmark experiments demonstrate that the proposed HVI-CIDNet+ outperforms the state-of-the-art methods on 10 datasets.
低光图像增强(LLIE)的目标是从受损的低光图像中恢复生动的内容和细节。然而,现有的基于标准RGB(sRGB)色彩空间的LLIE方法常常会产生色偏和亮度伪影,因为这种颜色空间对颜色变化非常敏感。尽管HSV色彩空间可以分离亮度和颜色,但它会引入显著的红色和黑色噪点伪影。为了解决这个问题,我们提出了一种新的用于LLIE的颜色空间,即水平/垂直强度(HVI),由HV色图和可学习的强度定义。HV色图强制执行红色坐标的短距离以移除红色噪点伪影,而可学习的强度则压缩低光区域以去除黑色噪点伪影。 此外,我们引入了一种基于HVI颜色空间构建的颜色和强度解耦网络+(HVI-CIDNet+),用于恢复受损内容并减轻极暗区域中的色彩失真。具体来说,HVI-CIDNet+利用预训练的视觉-语言模型从低光图像中提取丰富的上下文和降级知识,并通过一种新颖的先验引导注意模块(PAB)集成这些知识。在PAB内部,潜在语义先验可以促进内容恢复,而退化的表示则指导精确的颜色校正,在极暗区域尤其如此,这是通过精心设计的交叉注意力融合机制实现的。 此外,我们构建了一个区域精炼块,该块采用卷积处理信息丰富的区域,并使用自注意处理信息匮乏的区域,确保准确的亮度调整。基准实验的全面结果表明,所提出的HVI-CIDNet+在10个数据集上优于当前最先进的方法。
https://arxiv.org/abs/2507.06814
Real-time low-light image enhancement on mobile and embedded devices requires models that balance visual quality and computational efficiency. Existing deep learning methods often rely on large networks and labeled datasets, limiting their deployment on resource-constrained platforms. In this paper, we propose LiteIE, an ultra-lightweight unsupervised enhancement framework that eliminates dependence on large-scale supervision and generalizes well across diverse conditions. We design a backbone-agnostic feature extractor with only two convolutional layers to produce compact image features enhancement tensors. In addition, we develop a parameter-free Iterative Restoration Module, which reuses the extracted features to progressively recover fine details lost in earlier enhancement steps, without introducing any additional learnable parameters. We further propose an unsupervised training objective that integrates exposure control, edge-aware smoothness, and multi-scale color consistency losses. Experiments on the LOL dataset, LiteIE achieves 19.04 dB PSNR, surpassing SOTA by 1.4 dB while using only 0.07\% of its parameters. On a Snapdragon 8 Gen 3 mobile processor, LiteIE runs at 30 FPS for 4K images with just 58 parameters, enabling real-time deployment on edge devices. These results establish LiteIE as an efficient and practical solution for low-light enhancement on resource-limited platforms.
在移动和嵌入式设备上进行实时低光图像增强需要平衡视觉质量和计算效率的模型。现有的深度学习方法通常依赖于大型网络和标注数据集,这限制了它们在资源受限平台上的部署。本文提出了LiteIE,一个超轻量级无监督增强框架,该框架消除了对大规模监督信息的依赖,并且能够在各种条件下很好地泛化。我们设计了一个与主干网络无关的功能提取器,它仅包含两个卷积层,用于生成紧凑的图像特征增强张量。此外,我们开发了一个无需参数的迭代恢复模块(Iterative Restoration Module),该模块利用提取的功能逐步恢复早期增强步骤中丢失的细节点,而不需要引入任何可学习的额外参数。我们还提出了一种无监督训练目标,集成了曝光控制、边缘感知平滑度和多尺度颜色一致性损失。 在LOL数据集上的实验表明,LiteIE实现了19.04 dB PSNR(峰值信噪比),超越了现有最佳解决方案1.4 dB,同时仅使用其参数的0.07%。在Snapdragon 8 Gen 3移动处理器上,LiteIE以每秒30帧的速度处理4K图像,且只需58个参数,这使得它能够在边缘设备上实现实时部署。这些结果使LiteIE成为低光增强在资源受限平台上的高效和实用解决方案。
https://arxiv.org/abs/2507.04277
Speckle patterns in ultrasound images often obscure anatomical details, leading to diagnostic uncertainty. Recently, various deep learning (DL)-based techniques have been introduced to effectively suppress speckle; however, their high computational costs pose challenges for low-resource devices, such as portable ultrasound systems. To address this issue, EdgeSRIE, which is a lightweight hybrid DL framework for real-time speckle reduction and image enhancement in portable ultrasound imaging, is introduced. The proposed framework consists of two main branches: an unsupervised despeckling branch, which is trained by minimizing a loss function between speckled images, and a deblurring branch, which restores blurred images to sharp images. For hardware implementation, the trained network is quantized to 8-bit integer precision and deployed on a low-resource system-on-chip (SoC) with limited power consumption. In the performance evaluation with phantom and in vivo analyses, EdgeSRIE achieved the highest contrast-to-noise ratio (CNR) and average gradient magnitude (AGM) compared with the other baselines (different 2-rule-based methods and other 4-DL-based methods). Furthermore, EdgeSRIE enabled real-time inference at over 60 frames per second while satisfying computational requirements (< 20K parameters) on actual portable ultrasound hardware. These results demonstrated the feasibility of EdgeSRIE for real-time, high-quality ultrasound imaging in resource-limited environments.
超声图像中的斑点模式常常掩盖解剖细节,导致诊断不确定性。近年来,各种基于深度学习(DL)的技术被引入以有效抑制斑点噪声;然而,这些技术的高计算成本给资源有限的设备带来了挑战,如便携式超声系统。为了解决这一问题,提出了EdgeSRIE,这是一种轻量级混合DL框架,用于在便携式超声成像中实现实时斑点减少和图像增强。该框架主要由两个分支组成:一个无监督去斑点分支,通过最小化斑点图像之间的损失函数进行训练;以及一个去模糊分支,将模糊的图像恢复为清晰的图像。 为了硬件实现,经过训练的网络被量化到8位整数精度,并部署在低资源系统级芯片(SoC)上,该芯片具有有限的功耗。在使用仿体和体内分析的性能评估中,EdgeSRIE与其它基准方法相比,在对比噪声比(CNR)和平均梯度幅度(AGM)方面取得了最高成绩。这些其他基准方法包括不同的基于规则的方法和其他四种基于DL的方法。此外,EdgeSRIE能够在实际便携式超声设备上满足计算要求(参数数量少于20K)的同时实现每秒超过60帧的实时推断。 这些结果证明了在资源有限环境中进行实时高质量超声成像时,EdgeSRIE的可行性。
https://arxiv.org/abs/2507.03937
This paper introduces a novel deep learning framework for low-light image enhancement, named the Encoder-Decoder Network with Illumination Guidance (EDNIG). Building upon the U-Net architecture, EDNIG integrates an illumination map, derived from Bright Channel Prior (BCP), as a guidance input. This illumination guidance helps the network focus on underexposed regions, effectively steering the enhancement process. To further improve the model's representational power, a Spatial Pyramid Pooling (SPP) module is incorporated to extract multi-scale contextual features, enabling better handling of diverse lighting conditions. Additionally, the Swish activation function is employed to ensure smoother gradient propagation during training. EDNIG is optimized within a Generative Adversarial Network (GAN) framework using a composite loss function that combines adversarial loss, pixel-wise mean squared error (MSE), and perceptual loss. Experimental results show that EDNIG achieves competitive performance compared to state-of-the-art methods in quantitative metrics and visual quality, while maintaining lower model complexity, demonstrating its suitability for real-world applications. The source code for this work is available at this https URL.
本文介绍了一种新的深度学习框架,用于低光图像增强,名为带有光照指导的编码器-解码器网络(EDNIG)。该框架基于U-Net架构构建,并集成了由亮通道先验(BCP)衍生出的光照图作为指导输入。这种光照引导帮助网络聚焦于曝光不足的区域,从而更有效地控制图像增强过程。 为了进一步提高模型的表现能力,文中还加入了一个空间金字塔池化(SPP)模块以提取多尺度上下文特征,使得模型能够更好地处理不同的照明条件。此外,在训练过程中使用Swish激活函数来确保平滑的梯度传播。 EDNIG在生成对抗网络(GAN)框架中进行了优化,并采用了一种结合了对抗损失、像素级均方误差(MSE)和感知损失的复合损失函数进行训练。 实验结果表明,与现有最先进的方法相比,EDNIG在量化指标和视觉质量方面表现出了竞争力,同时保持较低的模型复杂度,证明其适合于实际应用。该工作的源代码可以在[提供的URL]获取。
https://arxiv.org/abs/2507.13360
Current methods for restoring underexposed images typically rely on supervised learning with paired underexposed and well-illuminated images. However, collecting such datasets is often impractical in real-world scenarios. Moreover, these methods can lead to over-enhancement, distorting well-illuminated regions. To address these issues, we propose IGDNet, a Zero-Shot enhancement method that operates solely on a single test image, without requiring guiding priors or training data. IGDNet exhibits strong generalization ability and effectively suppresses noise while restoring illumination. The framework comprises a decomposition module and a denoising module. The former separates the image into illumination and reflection components via a dense connection network, while the latter enhances non-uniformly illuminated regions using an illumination-guided pixel adaptive correction method. A noise pair is generated through downsampling and refined iteratively to produce the final result. Extensive experiments on four public datasets demonstrate that IGDNet significantly improves visual quality under complex lighting conditions. Quantitative results on metrics like PSNR (20.41dB) and SSIM (0.860dB) show that it outperforms 14 state-of-the-art unsupervised methods. The code will be released soon.
当前用于恢复曝光不足图像的方法通常依赖于监督学习,需要成对的曝光不足和光照良好的图像数据。然而,在实际场景中收集这样的数据集往往是不切实际的。此外,这些方法还可能导致过度增强,从而扭曲了原本光照良好的区域。为了解决这些问题,我们提出了IGDNet(Illumination Guided Decomposition Network),这是一种零样本增强方法,仅基于单个测试图像进行操作,并且不需要引导先验或训练数据。IGDNet展示了强大的泛化能力和在恢复照明的同时有效抑制噪声的能力。该框架包括分解模块和去噪模块:前者通过密集连接网络将图像分离为光照和反射组件;后者则使用光照引导的像素自适应校正方法来增强非均匀光照区域。通过对降采样生成的噪声对进行迭代细化以产生最终结果。在四个公开数据集上进行了广泛的实验,结果显示IGDNet在复杂照明条件下显著提高了视觉质量。基于PSNR(20.41dB)和SSIM(0.860)等指标的定量结果表明,它优于现有的14种最先进的无监督方法。代码将很快发布。
https://arxiv.org/abs/2507.02445
Enhancing underwater images is crucial for exploration. These images face visibility and color issues due to light changes, water turbidity, and bubbles. Traditional prior-based methods and pixel-based methods often fail, while deep learning lacks sufficient high-quality datasets. We introduce the Multi-Axis Conditional Lookup (MAC-Lookup) model, which enhances visual quality by improving color accuracy, sharpness, and contrast. It includes Conditional 3D Lookup Table Color Correction (CLTCC) for preliminary color and quality correction and Multi-Axis Adaptive Enhancement (MAAE) for detail refinement. This model prevents over-enhancement and saturation while handling underwater challenges. Extensive experiments show that MAC-Lookup excels in enhancing underwater images by restoring details and colors better than existing methods. The code is this https URL.
改善水下图像对于探索至关重要。这些图像由于光线变化、水质浑浊和气泡的影响,往往会面临能见度低和色彩失真的问题。传统的基于先验的方法和像素级方法往往效果不佳,而深度学习又缺乏足够高质量的数据集。我们引入了多轴条件查找(MAC-Lookup)模型,该模型通过提高颜色准确性、清晰度和对比度来提升视觉质量。它包括用于初步颜色和质量校正的条件3D查找表色彩校正(CLTCC),以及用于细节微调的多轴自适应增强(MAAE)。此模型能够防止过度增强和饱和,并且可以有效处理水下环境中的各种挑战。广泛的实验表明,MAC-Lookup在恢复图像细节和颜色方面优于现有的方法,在提升水下图像质量上表现出色。该代码位于此链接:[https URL]。
https://arxiv.org/abs/2507.02270
Precise surgical interventions are vital to patient safety, and advanced enhancement algorithms have been developed to assist surgeons in decision-making. Despite significant progress, these algorithms are typically designed for single tasks in specific scenarios, limiting their effectiveness in complex real-world situations. To address this limitation, we propose SurgVisAgent, an end-to-end intelligent surgical vision agent built on multimodal large language models (MLLMs). SurgVisAgent dynamically identifies distortion categories and severity levels in endoscopic images, enabling it to perform a variety of enhancement tasks such as low-light enhancement, overexposure correction, motion blur elimination, and smoke removal. Specifically, to achieve superior surgical scenario understanding, we design a prior model that provides domain-specific knowledge. Additionally, through in-context few-shot learning and chain-of-thought (CoT) reasoning, SurgVisAgent delivers customized image enhancements tailored to a wide range of distortion types and severity levels, thereby addressing the diverse requirements of surgeons. Furthermore, we construct a comprehensive benchmark simulating real-world surgical distortions, on which extensive experiments demonstrate that SurgVisAgent surpasses traditional single-task models, highlighting its potential as a unified solution for surgical assistance.
精确的外科手术干预对于患者安全至关重要,为此已经开发出先进的增强算法来帮助外科医生做出决策。尽管取得了显著进展,但这些算法通常仅针对特定场景下的单一任务而设计,因此在复杂的现实情况中其有效性受到限制。为了解决这一局限性,我们提出了SurgVisAgent,这是一个基于多模态大型语言模型(MLLM)构建的端到端智能外科手术视觉代理。SurgVisAgent能够动态识别内窥镜图像中的失真类型和严重程度,并执行多种增强任务,例如低光增强、过曝校正、运动模糊消除以及烟雾去除。 为了实现对复杂外科场景的卓越理解,我们设计了一个提供领域特定知识的先验模型。此外,通过上下文少量样本学习(in-context few-shot learning)和链式思维推理(CoT),SurgVisAgent能够根据各种失真类型和严重程度定制图像增强方案,从而满足外科医生多样化的实际需求。 为了验证这一方法的有效性,我们构建了一个全面的基准测试集,该集合模拟了真实世界中的外科手术失真。实验结果表明,与传统的单一任务模型相比,SurgVisAgent表现更佳,突显了其作为统一外科辅助解决方案的巨大潜力。
https://arxiv.org/abs/2507.02252
Document shadow removal is a crucial task in the field of document image enhancement. However, existing methods tend to remove shadows with constant color background and ignore color shadows. In this paper, we first design a diffusion model in latent space for document image shadow removal, called DocShaDiffusion. It translates shadow images from pixel space to latent space, enabling the model to more easily capture essential features. To address the issue of color shadows, we design a shadow soft-mask generation module (SSGM). It is able to produce accurate shadow mask and add noise into shadow regions specially. Guided by the shadow mask, a shadow mask-aware guided diffusion module (SMGDM) is proposed to remove shadows from document images by supervising the diffusion and denoising process. We also propose a shadow-robust perceptual feature loss to preserve details and structures in document images. Moreover, we develop a large-scale synthetic document color shadow removal dataset (SDCSRD). It simulates the distribution of realistic color shadows and provides powerful supports for the training of models. Experiments on three public datasets validate the proposed method's superiority over state-of-the-art. Our code and dataset will be publicly available.
文档阴影去除是文档图像增强领域中的一个重要任务。然而,现有的方法倾向于移除具有恒定颜色背景的阴影,并忽视了彩色阴影的存在。在本文中,我们首次设计了一个用于文档图像去阴影处理的潜在空间扩散模型,命名为DocShaDiffusion。该模型能够将含有阴影的图像从像素空间转换到潜在空间,从而使得模型更容易捕捉到关键特征。为了解决彩色阴影的问题,我们设计了一种名为影子软掩码生成模块(SSGM)的方法,它能够准确地生成影子掩码,并在影子区域特别添加噪声。通过影子掩码的引导,我们提出了一个影子掩码感知指导扩散模块(SMGDM),该模块可以在文档图像中去除阴影并通过监督扩散和去噪过程实现这一目标。此外,为了保持文档图像中的细节和结构信息不受损失,我们还提出了一种抗阴影的感知特征损失函数。同时,我们开发了一个大规模合成的文档彩色阴影去除数据集(SDCSRD),该数据集模拟了实际颜色阴影的分布情况,并为模型训练提供了强有力的支持。在三个公开的数据集上的实验验证了所提方法优于当前最先进的技术。我们的代码和数据集将公开发布。
https://arxiv.org/abs/2507.01422
Capturing high-quality visual content under low-light conditions remains a challenging problem due to severe noise, motion blur, and underexposure, which degrade the performance of downstream applications. Traditional frame-based low-light enhancement methods often amplify noise or fail to preserve structural details, especially in real-world scenarios. Event cameras, offering high dynamic range and microsecond temporal resolution by asynchronously capturing brightness changes, emerge as promising alternatives for low-light imaging. However, existing event-image fusion methods suffer from simplistic fusion strategies and inadequate handling of spatial-temporal misalignment and noise. To address these challenges, we propose EvRWKV, a novel framework that enables continuous cross-modal interaction through dual-domain processing. Our approach incorporates a Cross-RWKV module, leveraging the Receptance Weighted Key Value (RWKV) architecture for fine-grained temporal and cross-modal fusion, and an Event Image Spectral Fusion Enhancer (EISFE) module, which jointly performs adaptive frequency-domain noise suppression and spatial-domain deformable convolution alignment. Extensive qualitative and quantitative evaluations on real-world low-light datasets(SDE, SDSD, RELED) demonstrate that EvRWKV achieves state-of-the-art performance, effectively enhancing image quality by suppressing noise, restoring structural details, and improving visual clarity in challenging low-light conditions.
在低光条件下捕捉高质量的视觉内容仍然是一项具有挑战性的任务,因为严重的噪声、运动模糊和曝光不足会降低下游应用的表现。传统的帧基础低光增强方法往往会放大噪声或无法保留结构细节,特别是在真实世界的场景中。事件相机通过异步捕获亮度变化提供高动态范围和微秒级的时间分辨率,成为低光成像的有前景的选择之一。然而,现有的事件图像融合方法面临着简单的融合策略以及对时空错位和噪声处理不足的问题。为了应对这些挑战,我们提出了EvRWKV,这是一个新的框架,通过双域处理实现持续跨模态交互。 我们的方法包括一个Cross-RWKV模块,利用接收权重值(Receptance Weighted Key Value, RWKV)架构进行细粒度的时间融合和跨模态融合,以及一个事件图像频谱融合增强器(EISFE)模块,该模块同时执行自适应频率域噪声抑制和空间域可变形卷积对齐。在真实世界的低光数据集(SDE、SDSD、RELED)上进行广泛的定性和定量评估表明,EvRWKV达到了最先进的性能,通过抑制噪声、恢复结构细节以及改善视觉清晰度,在具有挑战性的低光条件下有效提高了图像质量。
https://arxiv.org/abs/2507.03184
The first algorithm, called Oneta, for a novel task of multi-style image enhancement is proposed in this work. Oneta uses two point operators sequentially: intensity enhancement with a transformation function (TF) and color correction with a color correction matrix (CCM). This two-step enhancement model, though simple, achieves a high performance upper bound. Also, we introduce eigentransformation function (eigenTF) to represent TF compactly. The Oneta network comprises Y-Net and C-Net to predict eigenTF and CCM parameters, respectively. To support $K$ styles, Oneta employs $K$ learnable tokens. During training, each style token is learned using image pairs from the corresponding dataset. In testing, Oneta selects one of the $K$ style tokens to enhance an image accordingly. Extensive experiments show that the single Oneta network can effectively undertake six enhancement tasks -- retouching, image signal processing, low-light image enhancement, dehazing, underwater image enhancement, and white balancing -- across 30 datasets.
本文提出了一种名为Oneta的新颖的多风格图像增强任务算法。Oneta依次使用两个点操作器:一个是强度增强,通过转换函数(TF)进行;另一个是颜色校正,采用色彩校正矩阵(CCM)。尽管这种两步增强模型很简单,但它能够达到很高的性能上限。此外,我们引入了特征变换函数(eigenTF)来紧凑地表示TF。Oneta网络包括Y-Net和C-Net两个部分,分别用于预测eigenTF和CCM参数。为了支持K种风格,Oneta使用了K个可学习的标记(token)。在训练过程中,每个风格标签通过来自相应数据集的图像对进行学习。而在测试阶段,Oneta会选择一个K种风格标记中的其中一个来增强相应的图像。广泛的实验表明,单一的Oneta网络能够有效地完成六项增强任务——润饰、图像信号处理、低光环境下的图片增强、去雾、水下图片增强和白平衡调整,在30个数据集中表现出色。
https://arxiv.org/abs/2506.23547
Infrared image helps improve the perception capabilities of autonomous driving in complex weather conditions such as fog, rain, and low light. However, infrared image often suffers from low contrast, especially in non-heat-emitting targets like bicycles, which significantly affects the performance of downstream high-level vision tasks. Furthermore, achieving contrast enhancement without amplifying noise and losing important information remains a challenge. To address these challenges, we propose a task-oriented infrared image enhancement method. Our approach consists of two key components: layer decomposition and saliency information extraction. First, we design an layer decomposition method for infrared images, which enhances scene details while preserving dark region features, providing more features for subsequent saliency information extraction. Then, we propose a morphological reconstruction-based saliency extraction method that effectively extracts and enhances target information without amplifying noise. Our method improves the image quality for object detection and semantic segmentation tasks. Extensive experiments demonstrate that our approach outperforms state-of-the-art methods.
红外图像有助于提高自动驾驶在复杂天气条件(如雾、雨和低光环境)下的感知能力。然而,红外图像常常存在对比度较低的问题,尤其是在非热辐射目标(如自行车)上,这严重影响了下游高级视觉任务的性能。此外,在不放大噪声并丢失重要信息的情况下实现对比度增强仍然是一项挑战。为了解决这些挑战,我们提出了一种面向任务的红外图像增强方法。我们的方法包括两个关键组成部分:层分解和显著性信息提取。 首先,我们设计了一种针对红外图像的分层分解方法,该方法在增强场景细节的同时保持暗区特征不变,从而为后续的显著性信息提取提供更多特征。然后,我们提出了一种基于形态重构的显著性抽取方法,可以有效地提取并增强目标信息而不放大噪声。我们的方法改善了物体检测和语义分割任务中的图像质量。 广泛的实验表明,与现有最先进的方法相比,我们的方法在性能上具有优势。
https://arxiv.org/abs/2506.23353