Abstract
Dynamic obstacle avoidance is a popular research topic for autonomous systems, such as micro aerial vehicles and service robots. Accurately evaluating the performance of dynamic obstacle avoidance methods necessitates the establishment of a metric to quantify the environment's difficulty, a crucial aspect that remains unexplored. In this paper, we propose four metrics to measure the difficulty of dynamic environments. These metrics aim to comprehensively capture the influence of obstacles' number, size, velocity, and other factors on the difficulty. We compare the proposed metrics with existing static environment difficulty metrics and validate them through over 1.5 million trials in a customized simulator. This simulator excludes the effects of perception and control errors and supports different motion and gaze planners for obstacle avoidance. The results indicate that the survivability metric outperforms and establishes a monotonic relationship between the success rate, with a Spearman's Rank Correlation Coefficient (SRCC) of over 0.9. Specifically, for every planner, lower survivability leads to a higher success rate. This metric not only facilitates fair and comprehensive benchmarking but also provides insights for refining collision avoidance methods, thereby furthering the evolution of autonomous systems in dynamic environments.
Abstract (translated)
动态避障是自动驾驶系统和服务机器人的一个热门研究课题。准确评估动态避障方法的性能需要建立一个指标来量化环境的难度，这是的一个重要方面，但尚未被探索。在本文中，我们提出了四个指标来衡量动态环境的难度。这些指标旨在全面捕捉障碍物数量、大小、速度和其他因素对难度的影响。我们将所提出的指标与现有的静态环境难度指标进行比较，并通过在定制仿真器上进行超过150000次试验来验证它们。这个仿真器排除了感知和控制误差的影响，支持不同避障规划的运动和视觉计划。结果表明，生存能力指标超过了传统的避障方法，并建立了成功率与幸存能力之间的单调关系，相关系数（SRCC）超过0.9。具体来说，对于每个规划器，较低的生存能力会导致更高的成功率。这个指标不仅促进了公平和全面的基准测试，还为改进避障方法提供了洞察，从而进一步推动自动驾驶系统在动态环境中的发展。
URL
https://arxiv.org/abs/2404.14848
