Abstract
Unmanned aerial vehicles (UAVs) have become powerful platforms for real-time, high-resolution data collection, producing massive volumes of aerial videos. Efficient retrieval of relevant content from these videos is crucial for applications in urban management, emergency response, security, and disaster relief. While text-video retrieval has advanced in natural video domains, the UAV domain remains underexplored due to limitations in existing datasets, such as coarse and redundant captions. Thus, in this work, we construct the Drone Video-Text Match Dataset (DVTMD), which contains 2,864 videos and 14,320 fine-grained, semantically diverse captions. The annotations capture multiple complementary aspects, including human actions, objects, background settings, environmental conditions, and visual style, thereby enhancing text-video correspondence and reducing redundancy. Building on this dataset, we propose the Text-Conditioned Multi-granularity Alignment (TCMA) framework, which integrates global video-sentence alignment, sentence-guided frame aggregation, and word-guided patch alignment. To further refine local alignment, we design a Word and Patch Selection module that filters irrelevant content, as well as a Text-Adaptive Dynamic Temperature Mechanism that adapts attention sharpness to text type. Extensive experiments on DVTMD and CapERA establish the first complete benchmark for drone text-video retrieval. Our TCMA achieves state-of-the-art performance, including 45.5% R@1 in text-to-video and 42.8% R@1 in video-to-text retrieval, demonstrating the effectiveness of our dataset and method. The code and dataset will be released.
Abstract (translated)
无人飞行器（UAVs）已成为实时、高分辨率数据采集的强大平台，生成了大量的空中视频。从这些视频中高效检索相关内容对于城市管理、应急响应、安全和灾害救援等应用至关重要。尽管在自然视频领域内文本-视频检索技术有所进步，但由于现有数据集存在粗略且冗余的描述等问题，UAV领域的研究尚处于探索阶段。 因此，在这项工作中，我们构建了无人机视频-文本匹配数据集（DVTMD），该数据集中包含了2,864个视频和14,320条精细、语义多样化的描述。这些标注涵盖了人类行为、物体、背景设置、环境条件及视觉风格等多个互补方面，从而增强了文本与视频之间的对应关系并减少了冗余性。 基于此数据集，我们提出了受文本调控的多粒度对齐框架（TCMA），该框架整合了全局视频-句子对齐、以句子为引导的帧聚合以及以单词为指导的补丁对齐。为了进一步优化局部对齐，我们设计了一个词语和补丁选择模块，用于过滤无关内容，并提出了一种基于文本适应性的动态温度机制，根据文本类型调整注意力强度。 在DVTMD和CapERA数据集上进行的广泛实验确立了首个完整的无人机文本-视频检索基准测试。我们的TCMA框架取得了最先进的性能，在文本到视频检索中达到45.5%的R@1，在视频到文本检索中达到42.8%的R@1，证明了我们所建立的数据集和方法的有效性。 代码与数据集将在后续发布。
URL
https://arxiv.org/abs/2510.10180
