Abstract
Traditional object detection models are typically trained on a fixed set of classes, limiting their flexibility and making it costly to incorporate new categories. Open-vocabulary object detection addresses this limitation by enabling models to identify unseen classes without explicit training. Leveraging pretrained models contrastively trained on abundantly available ground-view image-text classification pairs provides a strong foundation for open-vocabulary object detection in aerial imagery. Domain shifts, viewpoint variations, and extreme scale differences make direct knowledge transfer across domains ineffective, requiring specialized adaptation strategies. In this paper, we propose a novel framework for adapting open-vocabulary representations from ground-view images to solve object detection in aerial imagery through structured domain alignment. The method introduces contrastive image-to-image alignment to enhance the similarity between aerial and ground-view embeddings and employs multi-instance vocabulary associations to align aerial images with text embeddings. Extensive experiments on the xView, DOTAv2, VisDrone, DIOR, and HRRSD datasets are used to validate our approach. Our open-vocabulary model achieves improvements of +6.32 mAP on DOTAv2, +4.16 mAP on VisDrone (Images), and +3.46 mAP on HRRSD in the zero-shot setting when compared to finetuned closed-vocabulary dataset-specific model performance, thus paving the way for more flexible and scalable object detection systems in aerial applications.
Abstract (translated)
传统的对象检测模型通常在固定的一组类别上进行训练，这限制了其灵活性，并且增加了将新类别纳入系统的成本。开放词汇对象检测通过使模型能够在没有显式训练的情况下识别未见过的类来解决这一限制。利用预先训练好的模型，这些模型是在大量可用的地平视图像-文本分类对上对比性训练得到的，为在航拍图像中进行开放词汇对象检测提供了坚实的基础。 然而，领域变化、视角差异和极端尺度不同使得直接跨域知识迁移无效，需要专门的适应策略。在这篇论文中，我们提出了一种新颖的框架，用于将地平视图图像中的开放词汇表示适应到解决航拍图像中的对象检测问题，通过结构化的领域对齐来实现这一目标。该方法引入了对比性的图像对齐以增强航拍与地平视图嵌入之间的相似性，并采用多实例词汇关联来使航拍图像与文本嵌入对齐。 我们在xView、DOTAv2、VisDrone、DIOR和HRRSD数据集上进行了广泛的实验，用以验证我们的方法。在零样本设置中，我们的开放词汇模型相比经过微调的封闭词汇特定数据集模型，在DOTAv2上的mAP提升了+6.32%，在VisDrone（图像）上的mAP提升了+4.16%，在HRRSD上提升为+3.46%。这表明了更灵活和可扩展的对象检测系统在航拍应用中的潜力。
URL
https://arxiv.org/abs/2510.03858
