Abstract
A significant challenge in the field of object detection lies in the system's performance under non-ideal imaging conditions, such as rain, fog, low illumination, or raw Bayer images that lack ISP processing. Our study introduces "Feature Corrective Transfer Learning", a novel approach that leverages transfer learning and a bespoke loss function to facilitate the end-to-end detection of objects in these challenging scenarios without the need to convert non-ideal images into their RGB counterparts. In our methodology, we initially train a comprehensive model on a pristine RGB image dataset. Subsequently, non-ideal images are processed by comparing their feature maps against those from the initial ideal RGB model. This comparison employs the Extended Area Novel Structural Discrepancy Loss (EANSDL), a novel loss function designed to quantify similarities and integrate them into the detection loss. This approach refines the model's ability to perform object detection across varying conditions through direct feature map correction, encapsulating the essence of Feature Corrective Transfer Learning. Experimental validation on variants of the KITTI dataset demonstrates a significant improvement in mean Average Precision (mAP), resulting in a 3.8-8.1% relative enhancement in detection under non-ideal conditions compared to the baseline model, and a less marginal performance difference within 1.3% of the mAP@[0.5:0.95] achieved under ideal conditions by the standard Faster RCNN algorithm.
Abstract (translated)
在物体检测领域，一个重要的挑战是在非理想成像条件下，例如雨、雾、低照度或原始Bayer图像上，系统的性能。我们的研究引入了一种名为“特征纠正传输学习”的新方法，利用传输学习和自定义损失函数来促进在不需要将非理想图像转换为RGB同义品的情况下，端到端检测这些具有挑战性的场景中的物体。在我们的方法中，我们首先在一个纯净的RGB图像数据集上训练一个全面的模型。随后，通过将非理想图像的特征图与初始理想RGB模型的特征图进行比较来进行处理。这个比较采用了一种名为扩展区域新颖结构差异损失（EANSDL）的新损失函数，这是一种专门用于衡量相似性并将其集成到检测损失中的新损失函数。通过直接特征图纠正来优化模型的能力，包容了特征纠正传输学习的精髓。在KITTI数据集中的变体实验验证了这种方法在非理想条件下检测性能的显著提高，与基线模型相比，非理想条件的检测性能提高了3.8-8.1%，而在理想条件下，标准Faster RCNN算法的mAP@[0.5:0.95]的相对增强只有1.3%。
URL
https://arxiv.org/abs/2404.11214
