Abstract
This paper introduces Grounding DINO 1.5, a suite of advanced open-set object detection models developed by IDEA Research, which aims to advance the "Edge" of open-set object detection. The suite encompasses two models: Grounding DINO 1.5 Pro, a high-performance model designed for stronger generalization capability across a wide range of scenarios, and Grounding DINO 1.5 Edge, an efficient model optimized for faster speed demanded in many applications requiring edge deployment. The Grounding DINO 1.5 Pro model advances its predecessor by scaling up the model architecture, integrating an enhanced vision backbone, and expanding the training dataset to over 20 million images with grounding annotations, thereby achieving a richer semantic understanding. The Grounding DINO 1.5 Edge model, while designed for efficiency with reduced feature scales, maintains robust detection capabilities by being trained on the same comprehensive dataset. Empirical results demonstrate the effectiveness of Grounding DINO 1.5, with the Grounding DINO 1.5 Pro model attaining a 54.3 AP on the COCO detection benchmark and a 55.7 AP on the LVIS-minival zero-shot transfer benchmark, setting new records for open-set object detection. Furthermore, the Grounding DINO 1.5 Edge model, when optimized with TensorRT, achieves a speed of 75.2 FPS while attaining a zero-shot performance of 36.2 AP on the LVIS-minival benchmark, making it more suitable for edge computing scenarios. Model examples and demos with API will be released at this https URL
Abstract (translated)
本文介绍了IDEA研究开发的高级开放集物体检测模型Grounding DINO 1.5，该模型的目标是提高开放集物体检测的“边缘”。该系列包括两个模型：Grounding DINO 1.5 Pro，一种高性能模型，旨在在广泛的场景中提高泛化能力，以及Grounding DINO 1.5 Edge，一种专注于更快速度要求的许多需要边缘部署的应用程序的低延迟模型。Grounding DINO 1.5 Pro模型通过扩展模型架构、集成增强的视觉骨架和扩展训练数据集（带有 grounding 注释的超过2000万图像）来超越其前辈，从而实现更丰富的语义理解。Grounding DINO 1.5 Edge模型，虽然设计为具有较低特征缩放的高效模型，但在全面的数据集上训练仍具有稳健的检测能力。实验结果证明了Grounding DINO 1.5的有效性，Grounding DINO 1.5 Pro模型在COCO检测基准上获得了54.3的AP，在LVIS-minival零散转移基准上获得了55.7的AP，创造了新的开放集物体检测纪录。此外，当使用TensorRT优化Grounding DINO 1.5 Edge模型时，其速度达到75.2 FPS，同时取得了LVIS-minival基准上的零散转移性能为36.2 AP，使其更适合边缘计算场景。模型示例和API演示将会在这个链接上发布。
URL
https://arxiv.org/abs/2405.10300
