Abstract
Unmanned surface vehicles can encounter a number of varied visual circumstances during operation, some of which can be very difficult to interpret. While most cases can be solved only using color camera images, some weather and lighting conditions require additional information. To expand the available maritime data, we present a novel multimodal maritime dataset MULTIAQUA (Multimodal Aquatic Dataset). Our dataset contains synchronized, calibrated and annotated data captured by sensors of different modalities, such as RGB, thermal, IR, LIDAR, etc. The dataset is aimed at developing supervised methods that can extract useful information from these modalities in order to provide a high quality of scene interpretation regardless of potentially poor visibility conditions. To illustrate the benefits of the proposed dataset, we evaluate several multimodal methods on our difficult nighttime test set. We present training approaches that enable multimodal methods to be trained in a more robust way, thus enabling them to retain reliable performance even in near-complete darkness. Our approach allows for training a robust deep neural network only using daytime images, thus significantly simplifying data acquisition, annotation, and the training process.
Abstract (translated)
无人水面车辆在运行过程中会遇到多种多样的视觉情况，其中一些情况下很难进行解读。虽然大多数案例仅使用彩色相机图像就可以解决，但某些天气和光照条件则需要额外的信息。为了扩充可用的海洋数据，我们提出了一种新的多模态海事数据集MULTIAQUA（多模态水下数据集）。我们的数据集中包含了由不同类型的传感器同步捕获、校准并标注的数据，如RGB、热成像、红外、激光雷达等。该数据集旨在开发能够从这些模式中提取有用信息的监督方法，从而在可能存在的低能见度条件下也能提供高质量的场景解读。 为了展示所提出数据集的优势，我们评估了几种多模态方法在我们的夜间测试集中难以处理的情况下的表现。我们还提出了训练方法，使多模态方法能够以更稳健的方式进行训练，从而使它们即使在近乎完全黑暗的情况下也能保持可靠的性能。我们的方法仅通过使用白天的图像就可以训练出一个鲁棒的深度神经网络，从而大大简化了数据采集、标注和训练过程。
URL
https://arxiv.org/abs/2512.17450
