Abstract
Neural architecture search (NAS) traditionally requires significant human expertise or automated trial-and-error to design deep learning models. We present NN-Caption, an LLM-guided neural architecture search pipeline that generates runnable image-captioning models by composing CNN encoders from LEMUR's classification backbones with sequence decoders (LSTM/GRU/Transformer) under a strict Net API. Using DeepSeek-R1-0528-Qwen3-8B as the primary generator, we present the prompt template and examples of generated architectures. We evaluate on MS COCO with BLEU-4. The LLM generated dozens of captioning models, with over half successfully trained and producing meaningful captions. We analyse the outcomes of using different numbers of input model snippets (5 vs. 10) in the prompt, finding a slight drop in success rate when providing more candidate components. We also report training dynamics (caption accuracy vs. epochs) and the highest BLEU-4 attained. Our results highlight the promise of LLM-guided NAS: the LLM not only proposes architectures but also suggests hyperparameters and training practices. We identify the challenges encountered (e.g., code hallucinations or API compliance issues) and detail how prompt rules and iterative code fixes addressed them. This work presents a pipeline that integrates prompt-based code generation with automatic evaluation, and adds dozens of novel captioning models to the open LEMUR dataset to facilitate reproducible benchmarking and downstream AutoML research.
Abstract (translated)
神经架构搜索（NAS）传统上需要大量的专业知识或自动化试错来设计深度学习模型。我们提出了一个由大语言模型（LLM）引导的神经架构搜索流水线——NN-Caption，它通过组合从LEMUR分类骨干中提取的CNN编码器和序列解码器（LSTM/GRU/Transformer），在严格的Net API下生成可运行的图像描述模型。使用DeepSeek-R1-0528-Qwen3-8B作为主要生成器，我们展示了提示模板以及生成架构的例子。我们在MS COCO数据集上进行了BLEU-4评估，结果显示LLM生成了数十个描述模型，其中一半以上成功训练并产生了有意义的描述。我们分析了在提示中使用不同数量的输入模型片段（5与10）对成功率的影响，发现提供更多候选组件会导致成功率略有下降。 此外，我们报告了训练动态（随时间变化的描述准确性）和最高获得的BLEU-4分数。我们的研究结果强调了由LLM引导NAS的潜力：LLM不仅提出架构建议，还提出了超参数设置和训练实践方面的建议。我们识别了遇到的一些挑战（如代码幻觉或API兼容性问题），并详细说明了如何通过提示规则以及迭代代码修复来解决这些问题。 这项工作提供了一个集成基于提示的代码生成与自动评估的流水线，并向开放的LEMUR数据集增加了数十个新颖的描述模型，以促进可重复性的基准测试和下游自动化机器学习研究。
URL
https://arxiv.org/abs/2512.14706
