Abstract
The combined use of event-based vision and Spiking Neural Networks (SNNs) is expected to significantly impact robotics, particularly in tasks like visual odometry and obstacle avoidance. While existing real-world event-based datasets for optical flow prediction, typically captured with Unmanned Aerial Vehicles (UAVs), offer valuable insights, they are limited in diversity, scalability, and are challenging to collect. Moreover, there is a notable lack of labelled datasets for underwater applications, which hinders the integration of event-based vision with Autonomous Underwater Vehicles (AUVs). To address this, synthetic datasets could provide a scalable solution while bridging the gap between simulation and reality. In this work, we introduce eStonefish-scenes, a synthetic event-based optical flow dataset based on the Stonefish simulator. Along with the dataset, we present a data generation pipeline that enables the creation of customizable underwater environments. This pipeline allows for simulating dynamic scenarios, such as biologically inspired schools of fish exhibiting realistic motion patterns, including obstacle avoidance and reactive navigation around corals. Additionally, we introduce a scene generator that can build realistic reef seabeds by randomly distributing coral across the terrain. To streamline data accessibility, we present eWiz, a comprehensive library designed for processing event-based data, offering tools for data loading, augmentation, visualization, encoding, and training data generation, along with loss functions and performance metrics.
Abstract (translated)
基于事件的视觉与脉冲神经网络（SNNs）的结合预计将显著影响机器人技术，特别是在如视觉测距和障碍物规避等任务中。尽管现有的用于光流预测的真实世界事件数据集（通常由无人驾驶飞机系统[UAV]捕获），提供了宝贵的见解，但这些数据集在多样性、可扩展性和收集难度上存在局限性。此外，缺乏水下应用的标记化数据集限制了基于事件的视觉与自主水下航行器(AUVs)的集成。为解决这些问题，合成数据集可以提供一种可扩展的解决方案，并弥合仿真和现实之间的差距。在本工作中，我们引入了eStonefish-scenes，这是一个基于石鱼模拟器的合成事件驱动光流数据集。除了该数据集外，我们还介绍了一个数据生成管道，能够创建定制化的水下环境。这一管道支持动态场景的模拟，例如受生物启发的鱼类群体表现出包括障碍物规避和珊瑚周围反应导航在内的现实运动模式。此外，我们引入了一种场景生成器，它可以构建真实的礁石海底地形，通过在地面上随机分布珊瑚来实现。 为了简化数据访问性，我们推出了eWiz，这是一个全面处理基于事件的数据的库，提供了用于数据加载、增强、可视化、编码和训练数据生成的工具，以及损失函数和性能指标。
URL
https://arxiv.org/abs/2505.13309
