Abstract
Extracting useful visual cues for the downstream tasks is especially challenging under low-light vision. Prior works create enhanced representations by either correlating visual quality with machine perception or designing illumination-degrading transformation methods that require pre-training on synthetic datasets. We argue that optimizing enhanced image representation pertaining to the loss of the downstream task can result in more expressive representations. Therefore, in this work, we propose a novel module, FeatEnHancer, that hierarchically combines multiscale features using multiheaded attention guided by task-related loss function to create suitable representations. Furthermore, our intra-scale enhancement improves the quality of features extracted at each scale or level, as well as combines features from different scales in a way that reflects their relative importance for the task at hand. FeatEnHancer is a general-purpose plug-and-play module and can be incorporated into any low-light vision pipeline. We show with extensive experimentation that the enhanced representation produced with FeatEnHancer significantly and consistently improves results in several low-light vision tasks, including dark object detection (+5.7 mAP on ExDark), face detection (+1.5 mAPon DARK FACE), nighttime semantic segmentation (+5.1 mIoU on ACDC ), and video object detection (+1.8 mAP on DarkVision), highlighting the effectiveness of enhancing hierarchical features under low-light vision.
Abstract (translated)
在低光视觉环境下提取后续任务有用的视觉提示是非常具有挑战性的。以前的研究可以通过与机器感知相关的比较视觉质量或者设计照明削弱变换方法来增强表示。我们认为优化增强图像表示与后续任务的损失相关度可以产生更加表现力强的表示。因此,在本文中,我们提出了一个全新的模块,FeatEnHancer,它通过引导任务相关损失函数的多重头注意力Hierarchically combines multiscale features,以构建适当的表示。此外,我们的内部尺度增强改善了每个尺度或层次的提取特征的质量,并同时结合来自不同尺度的特征,以反映它们对于当前任务的重要性。FeatEnHancer是一个通用插件模块,可以添加到任何低光视觉流程中。我们通过广泛的实验表明,使用FeatEnHancer产生的增强表示在多个低光视觉任务中显著且一致性地提高了结果,包括暗物体检测(+5.7 mAP on ExDark)、人脸检测(+1.5 mAPon DARK FACE)、夜晚语义分割(+5.1 mIoU on ACDC)和视频物体检测(+1.8 mAP on DarkVision),强调了在低光视觉环境下增强Hierarchical features的有效性。
URL
https://arxiv.org/abs/2308.03594
