Abstract
Live tracking of wildlife via high-resolution video processing directly onboard drones is widely unexplored and most existing solutions rely on streaming video to ground stations to support navigation. Yet, both autonomous animal-reactive flight control beyond visual line of sight and/or mission-specific individual and behaviour recognition tasks rely to some degree on this capability. In response, we introduce WildLive -- a near real-time animal detection and tracking framework for high-resolution imagery running directly onboard uncrewed aerial vehicles (UAVs). The system performs multi-animal detection and tracking at 17fps+ for HD and 7fps+ on 4K video streams suitable for operation during higher altitude flights to minimise animal disturbance. Our system is optimised for Jetson Orin AGX onboard hardware. It integrates the efficiency of sparse optical flow tracking and mission-specific sampling with device-optimised and proven YOLO-driven object detection and segmentation techniques. Essentially, computational resource is focused onto spatio-temporal regions of high uncertainty to significantly improve UAV processing speeds without domain-specific loss of accuracy. Alongside, we introduce our WildLive dataset, which comprises 200k+ annotated animal instances across 19k+ frames from 4K UAV videos collected at the Ol Pejeta Conservancy in Kenya. All frames contain ground truth bounding boxes, segmentation masks, as well as individual tracklets and tracking point trajectories. We compare our system against current object tracking approaches including OC-SORT, ByteTrack, and SORT. Our multi-animal tracking experiments with onboard hardware confirm that near real-time high-resolution wildlife tracking is possible on UAVs whilst maintaining high accuracy levels as needed for future navigational and mission-specific animal-centric operational autonomy.
Abstract (translated)
通过高分辨率视频处理在无人机上实时追踪野生动物的方法目前还很少被探索，而大多数现有的解决方案依赖于将视频流传输到地面站来支持导航。然而，无论是自主的动物反应式飞行控制（超越视线范围）还是特定任务下的个体和行为识别任务，在某种程度上都依赖于此能力。为此，我们介绍了WildLive——这是一个近实时野生动物检测与追踪框架，适用于在无人驾驶航空器(UAV)上直接运行高分辨率图像处理。 该系统能够在高清视频流中实现多动物检测和追踪速度超过17帧/秒，并且在4K视频流中的速度超过7帧/秒。这些性能对于高空飞行操作而言非常合适，能够尽量减少对野生动物的干扰。我们的系统优化了Jetson Orin AGX车载硬件，并结合稀疏光流跟踪、任务特定采样与设备优化和经过验证的YOLO驱动对象检测及分割技术的有效性。简言之，计算资源集中在时空区域中的高不确定性部分上，从而显著提升无人机处理速度，同时保持对领域的准确度不降低。 此外，我们还推出了WildLive数据集，该数据集包括超过20万个标注动物实例，这些实例来自在肯尼亚奥尔佩杰塔保护区收集的4K无人机视频中的19,000多帧。所有帧都包含地面真值边界框、分割掩码以及个体跟踪轨迹和跟踪点路径。 我们还将WildLive系统与现有对象追踪方法（如OC-SORT，ByteTrack 和SORT）进行了比较。通过车载硬件进行的多动物追踪实验表明，在无人驾驶航空器上实现近实时高分辨率野生动物追踪是可能的，并且在所需的未来导航及特定任务下的动物中心操作自主性方面仍能保持高水平的准确度。
URL
https://arxiv.org/abs/2504.10165
