Abstract
Designing high-performance object detection architectures is a complex task, where traditional manual design is time-consuming and labor-intensive, and Neural Architecture Search (NAS) is computationally prohibitive. While recent approaches using Large Language Models (LLMs) show promise, they often function as iterative optimizers within a search loop, rather than generating architectures directly from a holistic understanding of the data. To address this gap, we propose Cognitive-YOLO, a novel framework for LLM-driven architecture synthesis that generates network configurations directly from the intrinsic characteristics of the dataset. Our method consists of three stages: first, an analysis module extracts key meta-features (e.g., object scale distribution and scene density) from the target dataset; second, the LLM reasons upon these features, augmented with state-of-the-art components retrieved via Retrieval-Augmented Generation (RAG), to synthesize the architecture into a structured Neural Architecture Description Language (NADL); finally, a compiler instantiates this description into a deployable model. Extensive experiments on five diverse object detection datasets demonstrate that our proposed Cognitive-YOLO consistently generates superior architectures, achieving highly competitive performance and demonstrating a superior performance-per-parameter trade-off compared to strong baseline models across multiple benchmarks. Crucially, our ablation studies prove that the LLM's data-driven reasoning is the primary driver of performance, demonstrating that a deep understanding of data "first principles" is more critical for achieving a superior architecture than simply retrieving SOTA components.
Abstract (translated)
设计高性能的目标检测架构是一项复杂的任务，传统的手动设计耗时且劳动密集型，而神经架构搜索（NAS）则计算成本高昂。尽管最近使用大型语言模型（LLMs）的方法显示出前景，但它们通常作为搜索循环中的迭代优化器工作，而不是直接从数据的全面理解生成架构。为了解决这一差距，我们提出了Cognitive-YOLO，这是一种新的框架，通过该框架利用大型语言模型驱动的架构合成来直接从数据集的基本特征生成网络配置。 我们的方法分为三个阶段：首先，分析模块从目标数据集中提取关键元特征（例如对象尺度分布和场景密度）；其次，LLM在这些特征的基础上进行推理，并借助检索增强生成（RAG）获取最先进的组件，以将架构合成为结构化的神经架构描述语言（NADL）；最后，一个编译器将这种描述实例化为可部署的模型。 我们在五个不同的目标检测数据集上进行了广泛的实验，结果表明我们提出的Cognitive-YOLO能够一致地生成优于基准的架构，在多个基准测试中实现了具有竞争力的表现，并且在性能与参数之间的权衡方面表现更优。尤为重要的是，我们的消融研究证明了LLM的数据驱动推理是性能的主要驱动力，显示出对数据“基本原则”的深入了解比单纯检索最先进的组件对于实现优越架构更为关键。
URL
https://arxiv.org/abs/2512.12281
