Abstract
Recent advances in machine learning and computer vision are revolutionizing the field of animal behavior by enabling researchers to track the poses and locations of freely moving animals without any marker attachment. However, large datasets of annotated images of animals for markerless pose tracking, especially high-resolution images taken from multiple angles with accurate 3D annotations, are still scant. Here, we propose a method that uses a motion capture (mo-cap) system to obtain a large amount of annotated data on animal movement and posture (2D and 3D) in a semi-automatic manner. Our method is novel in that it extracts the 3D positions of morphological keypoints (e.g eyes, beak, tail) in reference to the positions of markers attached to the animals. Using this method, we obtained, and offer here, a new dataset - 3D-POP with approximately 300k annotated frames (4 million instances) in the form of videos having groups of one to ten freely moving birds from 4 different camera views in a 3.6m x 4.2m area. 3D-POP is the first dataset of flocking birds with accurate keypoint annotations in 2D and 3D along with bounding box and individual identities and will facilitate the development of solutions for problems of 2D to 3D markerless pose, trajectory tracking, and identification in birds.
Abstract (translated)
近年来在机器学习和计算机视觉方面的进展正在动物行为领域带来革命性的变革，使研究人员能够追踪自由移动动物的 pose 和位置，而无需附加标记。然而，用于标记less pose 跟踪的大型标注图像动物数据集仍然稀缺，特别是从多个角度拍摄且高精度3D标注的高清视频数据集。在这里，我们提出了一种方法，使用运动捕捉(mo-cap)系统以半自动方式获取大量关于动物运动和姿态(2D 和 3D)的标注数据。我们的方法是新颖的，它从参考动物附加标记的位置中提取形态学关键帧的 3D 位置，这种方法能够提取关键帧的位置，包括眼睛、喙和尾巴等。通过这种方法，我们获取了并在这里提供了一个新的数据集 - 3D-POP，其中包括约 300 万帧标注帧(4 百万实例)，以视频形式，其中包含由四个不同相机视图组成的群体，每个视频片段包含 one 到 ten 个自由移动的鸟类。3D-POP是第一个包含2D和3D高精度关键帧标注并与边界框和个人身份一起考虑的鸟类群体数据集，这将促进解决2D到3D标记less 姿态、轨迹跟踪和识别鸟类问题的解决方案。
URL
https://arxiv.org/abs/2303.13174
