Abstract
Building efficient neural network architectures can be a time-consuming task requiring extensive expert knowledge. This task becomes particularly challenging for edge devices because one has to consider parameters such as power consumption during inferencing, model size, inferencing speed, and CO2 emissions. In this article, we introduce a novel framework designed to automatically discover new neural network architectures based on user-defined parameters, an expert system, and an LLM trained on a large amount of open-domain knowledge. The introduced framework (LeMo-NADe) is tailored to be used by non-AI experts, does not require a predetermined neural architecture search space, and considers a large set of edge device-specific parameters. We implement and validate this proposed neural architecture discovery framework using CIFAR-10, CIFAR-100, and ImageNet16-120 datasets while using GPT-4 Turbo and Gemini as the LLM component. We observe that the proposed framework can rapidly (within hours) discover intricate neural network models that perform extremely well across a diverse set of application settings defined by the user.
Abstract (translated)
建立高效的神经网络架构可能是一个耗时且需要广泛专家知识的任务。对于边缘设备来说，这个任务变得尤为具有挑战性，因为需要考虑诸如功耗、模型大小、推理速度和二氧化碳排放等参数。在本文中，我们介绍了一个新框架，该框架可以根据用户定义的参数、专家系统和基于大量开放领域知识的大型语言模型（LLM）自动发现新的神经网络架构。所提出的框架（LeMo-NADe）专门针对非AI专家设计，不需要预先确定的神经架构搜索空间，并考虑了一个大型的边缘设备特定参数集。我们使用CIFAR-10、CIFAR-100和ImageNet16-120数据集来实施和验证所提出的神经网络架构发现框架，同时使用GPT-4 Turbo和Gemini作为LLM组件。我们观察到，与现有的方法相比，所提出的框架可以在几小时内迅速发现用户定义的复杂神经网络模型，这些模型在各种应用场景中表现出色。
URL
https://arxiv.org/abs/2402.18443
