Abstract
Neural Architecture Search (NAS) automates network design, but conventional methods demand substantial computational resources. We propose a closed-loop pipeline leveraging large language models (LLMs) to iteratively generate, evaluate, and refine convolutional neural network architectures for image classification on a single consumer-grade GPU without LLM fine-tuning. Central to our approach is a historical feedback memory inspired by Markov chains: a sliding window of $K{=}5$ recent improvement attempts keeps context size constant while providing sufficient signal for iterative learning. Unlike prior LLM optimizers that discard failure trajectories, each history entry is a structured diagnostic triple -- recording the identified problem, suggested modification, and resulting outcome -- treating code execution failures as first-class learning signals. A dual-LLM specialization reduces per-call cognitive load: a Code Generator produces executable PyTorch architectures while a Prompt Improver handles diagnostic reasoning. Since both the LLM and architecture training share limited VRAM, the search implicitly favors compact, hardware-efficient models suited to edge deployment. We evaluate three frozen instruction-tuned LLMs (${\leq}7$B parameters) across up to 2000 iterations in an unconstrained open code space, using one-epoch proxy accuracy on CIFAR-10, CIFAR-100, and ImageNette as a fast ranking signal. On CIFAR-10, DeepSeek-Coder-6.7B improves from 28.2% to 69.2%, Qwen2.5-7B from 50.0% to 71.5%, and GLM-5 from 43.2% to 62.0%. A full 2000-iteration search completes in ${\approx}18$ GPU hours on a single RTX~4090, establishing a low-budget, reproducible, and hardware-aware paradigm for LLM-driven NAS without cloud infrastructure.
Abstract (translated)
神经架构搜索（NAS）自动化了网络设计，但传统方法需要大量的计算资源。我们提出了一种闭环流水线，利用大型语言模型（LLMs）在单个消费级GPU上无需对LLM进行微调的情况下，迭代生成、评估和优化卷积神经网络架构以进行图像分类。我们的方法的核心是一种受马尔可夫链启发的历史反馈记忆：一个宽度为$K=5$的滑动窗口保持上下文大小恒定，同时提供足够的信号来进行迭代学习。与之前的LLM优化器丢弃失败路径不同，每个历史条目都是一个结构化的诊断三元组——记录识别的问题、建议的修改以及结果——将代码执行失败视为首要的学习信号。双LLM专业化减少了每次调用的认知负荷：一个代码生成器生产可执行的PyTorch架构，而提示改进者处理诊断推理。由于LLM和架构训练共享有限的VRAM资源，搜索过程隐式地倾向于紧凑、硬件高效的模型，适合边缘部署。 我们在约束最少的开放代码空间中评估了三个冻结指令调优的LLMs（≤70亿参数），最多进行2000次迭代，并使用CIFAR-10、CIFAR-100和ImageNette上一个epoch的代理准确度作为快速排名信号。在CIFAR-10数据集上，DeepSeek-Coder-6.7B从28.2%提高到了69.2%，Qwen2.5-7B从50.0%提升到71.5%，而GLM-5从43.2%升至62.0%。一个完整的2000次迭代搜索在单个RTX 4090 GPU上大约需要18小时，从而建立了低成本、可重复和硬件感知的LLM驱动NAS范式，无需云基础设施。 这种方法提供了显著的成本效益，并且为在边缘设备上部署高效的神经网络架构开辟了新的可能性。
URL
https://arxiv.org/abs/2603.12091
