Abstract
The new trend in multi-object tracking task is to track objects of interest using natural language. However, the scarcity of paired prompt-instance data hinders its progress. To address this challenge, we propose a high-quality yet low-cost data generation method base on Unreal Engine 5 and construct a brand-new benchmark dataset, named Refer-UE-City, which primarily includes scenes from intersection surveillance videos, detailing the appearance and actions of people and vehicles. Specifically, it provides 14 videos with a total of 714 expressions, and is comparable in scale to the Refer-KITTI dataset. Additionally, we propose a multi-level semantic-guided multi-object framework called MLS-Track, where the interaction between the model and text is enhanced layer by layer through the introduction of Semantic Guidance Module (SGM) and Semantic Correlation Branch (SCB). Extensive experiments on Refer-UE-City and Refer-KITTI datasets demonstrate the effectiveness of our proposed framework and it achieves state-of-the-art performance. Code and datatsets will be available.
Abstract (translated)
跨对象跟踪任务的新趋势是使用自然语言跟踪感兴趣的对象。然而，缺乏成对提示实例数据会阻碍其进展。为解决这个问题，我们提出了一个高质量但成本低的数据生成方法，基于Unreal Engine 5，并构建了一个名为Refer-UE-City的新基准数据集，主要包括路口监视视频的场景，详细描述了人和车辆的外观和行为。具体来说，它提供了14个视频，总共有714个表情，与Refer-KITTI数据集的规模相当。此外，我们提出了一个多层语义引导多对象框架MLS-Track，通过引入语义引导模块（SGM）和语义相关分支（SCB）来增强模型和文本之间的交互。对Refer-UE-City和Refer-KITTI数据集的实验表明，我们提出的框架的有效性得到了证明，并且实现了最先进的性能。代码和数据集将可用。
URL
https://arxiv.org/abs/2404.12031
