NAS
-
TAIGen: Training-Free Adversarial Image Generation via Diffusion Models
2025-08-20 19:21:51Susim Roy, Anubhooti Jain, Mayank Vatsa, Richa SinghAbstract
Adversarial attacks from generative models often produce low-quality images and require substantial computational resources. Diffusion models, though capable of high-quality generation, typically need hundreds of sampling steps for adversarial generation. This paper introduces TAIGen, a training-free black-box method for efficient adversarial image generation. TAIGen produces adversarial examples using only 3-20 sampling steps from unconditional diffusion models. Our key finding is that perturbations injected during the mixing step interval achieve comparable attack effectiveness without processing all timesteps. We develop a selective RGB channel strategy that applies attention maps to the red channel while using GradCAM-guided perturbations on green and blue channels. This design preserves image structure while maximizing misclassification in target models. TAIGen maintains visual quality with PSNR above 30 dB across all tested datasets. On ImageNet with VGGNet as source, TAIGen achieves 70.6% success against ResNet, 80.8% against MNASNet, and 97.8% against ShuffleNet. The method generates adversarial examples 10x faster than existing diffusion-based attacks. Our method achieves the lowest robust accuracy, indicating it is the most impactful attack as the defense mechanism is least successful in purifying the images generated by TAIGen.
Abstract (translated)
来自生成模型的对抗性攻击通常会产生质量低下的图像，并且需要大量的计算资源。扩散模型虽然能够生成高质量的图片，但在进行对抗性生成时通常需要几百个采样步骤。本文介绍了TAIGen，一种无需训练的黑盒方法，用于高效的对抗性图像生成。TAIGen仅使用3到20步无条件扩散模型中的采样步骤就能产生对抗样本。 我们的主要发现是，在混合步骤间隔期间注入扰动可以实现相当有效的攻击效果，而不需要处理所有时间步长。我们开发了一种选择性的RGB通道策略，该策略在红色通道上应用注意力图，同时在绿色通道和蓝色通道使用GradCAM引导的扰动。这种设计保留了图像结构的同时最大化目标模型中的分类错误。 TAIGen在所有测试数据集上保持视觉质量，PSNR值超过30dB。在以ImageNet为基准、VGGNet作为源的情况下，TAIGen对ResNet的成功率为70.6%，对MNASNet的成功率为80.8%，对ShuffleNet的成功率则高达97.8%。 该方法生成对抗样本的速度是现有基于扩散的攻击方法的10倍。我们的方法实现了最低的鲁棒准确性，表明它是最具影响力的攻击方式，因为防御机制在净化由TAIGen产生的图像方面最不成功。
URL
https://arxiv.org/abs/2508.15020
PDF
-
Quantum Long Short-term Memory with Differentiable Architecture Search
2025-08-20 16:15:00Samuel Yen-Chi Chen, Prayag TiwariAbstract
Recent advances in quantum computing and machine learning have given rise to quantum machine learning (QML), with growing interest in learning from sequential data. Quantum recurrent models like QLSTM are promising for time-series prediction, NLP, and reinforcement learning. However, designing effective variational quantum circuits (VQCs) remains challenging and often task-specific. To address this, we propose DiffQAS-QLSTM, an end-to-end differentiable framework that optimizes both VQC parameters and architecture selection during training. Our results show that DiffQAS-QLSTM consistently outperforms handcrafted baselines, achieving lower loss across diverse test settings. This approach opens the door to scalable and adaptive quantum sequence learning.
Abstract (translated)
最近在量子计算和机器学习领域的进展催生了量子机器学习（QML），特别是在处理序列数据方面引起了越来越多的兴趣。例如，像QLSTM这样的量子递归模型对于时间序列预测、自然语言处理以及强化学习都表现出潜力。然而，设计有效的变分量子电路（VQC）仍然具有挑战性，并且往往需要针对具体任务进行定制。为了解决这个问题，我们提出了DiffQAS-QLSTM，这是一个端到端可微框架，它在训练过程中同时优化VQC参数和架构选择。我们的实验结果显示，与人工设计的基准模型相比，DiffQAS-QLSTM能够持续获得更低的损失值，在各种测试设置中表现出色。这种方法为规模化的、适应性强的量子序列学习打开了大门。
URL
https://arxiv.org/abs/2508.14955
PDF
-
MorphNAS: Differentiable Architecture Search for Morphologically-Aware Multilingual NER
2025-08-19 04:48:51Prathamesh Devadiga, Omkaar Jayadev Shetty, Hiya Nachnani, Prema RAbstract
Morphologically complex languages, particularly multiscript Indian languages, present significant challenges for Natural Language Processing (NLP). This work introduces MorphNAS, a novel differentiable neural architecture search framework designed to address these challenges. MorphNAS enhances Differentiable Architecture Search (DARTS) by incorporating linguistic meta-features such as script type and morphological complexity to optimize neural architectures for Named Entity Recognition (NER). It automatically identifies optimal micro-architectural elements tailored to language-specific morphology. By automating this search, MorphNAS aims to maximize the proficiency of multilingual NLP models, leading to improved comprehension and processing of these complex languages.
Abstract (translated)
形态复杂的语言，尤其是多文种的印度语系语言，在自然语言处理（NLP）中面临着重大挑战。本文介绍了MorphNAS，这是一种新颖的可微神经架构搜索框架，旨在解决这些挑战。MorphNAS通过引入诸如书写系统类型和形态复杂性等语言元特征来增强可微架构搜索（DARTS），以优化针对命名实体识别（NER）任务的神经网络结构。它能够自动识别出最适合特定语言词形变化特点的小规模架构元素。通过自动化这一搜索过程，MorphNAS旨在最大化多语种NLP模型的专业能力，从而提高对这些复杂语言的理解和处理效率。
URL
https://arxiv.org/abs/2508.15836
PDF
-
Dextr: Zero-Shot Neural Architecture Search with Singular Value Decomposition and Extrinsic Curvature
2025-08-18 14:52:14Rohan Asthana, Joschua Conrad, Maurits Ortmanns, Vasileios BelagiannisAbstract
Zero-shot Neural Architecture Search (NAS) typically optimises the architecture search process by exploiting the network or gradient properties at initialisation through zero-cost proxies. The existing proxies often rely on labelled data, which is usually unavailable in real-world settings. Furthermore, the majority of the current methods focus either on optimising the convergence and generalisation attributes or solely on the expressivity of the network architectures. To address both limitations, we first demonstrate how channel collinearity affects the convergence and generalisation properties of a neural network. Then, by incorporating the convergence, generalisation and expressivity in one approach, we propose a zero-cost proxy that omits the requirement of labelled data for its computation. In particular, we leverage the Singular Value Decomposition (SVD) of the neural network layer features and the extrinsic curvature of the network output to design our proxy. %As a result, the proposed proxy is formulated as the simplified harmonic mean of the logarithms of two key components: the sum of the inverse of the feature condition number and the extrinsic curvature of the network output. Our approach enables accurate prediction of network performance on test data using only a single label-free data sample. Our extensive evaluation includes a total of six experiments, including the Convolutional Neural Network (CNN) search space, i.e. DARTS and the Transformer search space, i.e. AutoFormer. The proposed proxy demonstrates a superior performance on multiple correlation benchmarks, including NAS-Bench-101, NAS-Bench-201, and TransNAS-Bench-101-micro; as well as on the NAS task within the DARTS and the AutoFormer search space, all while being notably efficient. The code is available at this https URL.
Abstract (translated)
零样本神经架构搜索（NAS）通常通过在初始阶段利用网络或梯度属性的无成本代理来优化架构搜索过程。现有的许多代理依赖于有标签的数据，而在现实场景中这些数据往往不可用。此外，目前大多数方法要么侧重于优化收敛性和泛化属性，要么仅关注网络架构的表达能力。为了解决这两个限制，我们首先展示了通道共线性如何影响神经网络的收敛性和泛化特性。然后，通过将收敛性、泛化和表达能力结合在一个方法中，我们提出了一种无成本代理，该代理在计算时不依赖有标签的数据需求。具体来说，我们利用了神经网络层特征的奇异值分解（SVD）以及网络输出的外在曲率来设计我们的代理。 所提出的代理被公式化为两个关键组件对数的简化调和平均值：特征条件数之逆的总和与网络输出的外在曲率。这种方法仅使用一个无标签的数据样本就能准确预测网络在测试数据上的性能。我们进行了广泛的评估，包括六个实验，涉及卷积神经网络（CNN）搜索空间（例如DARTS）和Transformer搜索空间（如AutoFormer）。所提出的代理在多个相关基准上展示了优越的性能，包括NAS-Bench-101、NAS-Bench-201以及TransNAS-Bench-101-micro；同时，在DARTS和AutoFormer搜索空间内的NAS任务中表现同样出色，并且效率显著。代码可在该链接获得：[此处应填写URL]。
URL
https://arxiv.org/abs/2508.12977
PDF
-
LangVision-LoRA-NAS: Neural Architecture Search for Variable LoRA Rank in Vision Language Models
2025-08-17 22:19:02Krishna Teja Chitty-Venkata, Murali Emani, Venkatram VishwanathAbstract
Vision Language Models (VLMs) integrate visual and text modalities to enable multimodal understanding and generation. These models typically combine a Vision Transformer (ViT) as an image encoder and a Large Language Model (LLM) for text generation. LoRA (Low-Rank Adaptation) is an efficient fine-tuning method to adapt pre-trained models to new tasks by introducing low-rank updates to their weights. While LoRA has emerged as a powerful technique for fine-tuning large models by introducing low-rank updates, current implementations assume a fixed rank, potentially limiting flexibility and efficiency across diverse tasks. This paper introduces \textit{LangVision-LoRA-NAS}, a novel framework that integrates Neural Architecture Search (NAS) with LoRA to optimize VLMs for variable-rank adaptation. Our approach leverages NAS to dynamically search for the optimal LoRA rank configuration tailored to specific multimodal tasks, balancing performance and computational efficiency. Through extensive experiments using the LLaMA-3.2-11B model on several datasets, LangVision-LoRA-NAS demonstrates notable improvement in model performance while reducing fine-tuning costs. Our Base and searched fine-tuned models on LLaMA-3.2-11B-Vision-Instruct can be found \href{this https URL}{\textcolor{blue}{here}} and the code for LangVision-LoRA-NAS can be found \href{this https URL}{\textcolor{blue}{here}}.
Abstract (translated)
视觉语言模型（VLMs）结合了视觉和文本模态，以实现多模态理解和生成。这些模型通常将视觉变压器（ViT）用作图像编码器，并使用大型语言模型（LLM）进行文本生成。低秩适应（LoRA）是一种高效的微调方法，通过向预训练模型的权重引入低秩更新来使其适应新任务。虽然LoRA已成为一种强大的技术，用于通过引入低秩更新来微调大规模模型，但目前的实现假设固定的秩，这可能限制了在各种任务中的灵活性和效率。本文介绍了**LangVision-LoRA-NAS**，这是一种新颖的框架，它将神经架构搜索（NAS）与LoRA结合使用，以优化视觉语言模型进行变秩适应。我们的方法利用NAS动态地为特定多模态任务寻找最优的LoRA秩配置，在性能和计算效率之间取得平衡。通过在LLaMA-3.2-11B模型上对多个数据集进行广泛的实验，LangVision-LoRA-NAS展示了显著提高模型性能的同时降低了微调成本。 我们的基础和搜索微调后的LLaMA-3.2-11B-Vision-Instruct模型可在此[链接]处获取，而LangVision-LoRA-NAS的代码可在另一链接处找到。请注意，上述链接应替换为实际提供的URL地址。
URL
https://arxiv.org/abs/2508.12512
PDF
-
RegimeNAS: Regime-Aware Differentiable Architecture Search With Theoretical Guarantees for Financial Trading
2025-08-15 09:09:54Prathamesh Devadiga, Yashmitha ShaileshAbstract
We introduce RegimeNAS, a novel differentiable architecture search framework specifically designed to enhance cryptocurrency trading performance by explicitly integrating market regime awareness. Addressing the limitations of static deep learning models in highly dynamic financial environments, RegimeNAS features three core innovations: (1) a theoretically grounded Bayesian search space optimizing architectures with provable convergence properties; (2) specialized, dynamically activated neural modules (Volatility, Trend, and Range blocks) tailored for distinct market conditions; and (3) a multi-objective loss function incorporating market-specific penalties (e.g., volatility matching, transition smoothness) alongside mathematically enforced Lipschitz stability constraints. Regime identification leverages multi-head attention across multiple timeframes for improved accuracy and uncertainty estimation. Rigorous empirical evaluation on extensive real-world cryptocurrency data demonstrates that RegimeNAS significantly outperforms state-of-the-art benchmarks, achieving an 80.3% Mean Absolute Error reduction compared to the best traditional recurrent baseline and converging substantially faster (9 vs. 50+ epochs). Ablation studies and regime-specific analysis confirm the critical contribution of each component, particularly the regime-aware adaptation mechanism. This work underscores the imperative of embedding domain-specific knowledge, such as market regimes, directly within the NAS process to develop robust and adaptive models for challenging financial applications.
Abstract (translated)
我们介绍了一种新颖的可微架构搜索框架RegimeNAS，该框架专门设计用于通过明确地整合市场制度意识来提升加密货币交易的表现。针对静态深度学习模型在高度动态金融环境中的局限性，RegimeNAS具备三大核心创新：(1) 基于贝叶斯理论并具有证明收敛特性的优化架构搜索空间；(2) 专为不同市场条件定制的、可动态激活的神经模块（波动率、趋势和区间块）；以及 (3) 结合了市场特定惩罚（如波动匹配、转换平滑度等）及数学强制性Lipschitz稳定性约束的多目标损失函数。制度识别利用跨多个时间框架的多头注意力机制，以提高准确性和不确定性估计。 在广泛的现实世界加密货币数据上的严格实证评估表明，RegimeNAS显著优于最先进的基准测试，在均方绝对误差方面相比最好的传统循环基线模型减少了80.3%，并且收敛速度明显更快（9个周期对比50多个周期）。消融研究和制度特定分析证实了每个组件对整体性能的关键贡献，尤其是制度感知适应机制。这项工作强调了在NAS过程中直接嵌入领域特定知识（如市场制度）的重要性，以开发出适用于复杂金融应用的稳健且自适应模型。
URL
https://arxiv.org/abs/2508.11338
PDF
-
eMamba: Efficient Acceleration Framework for Mamba Models in Edge Computing
2025-08-14 06:08:05Jiyong Kim, Jaeho Lee, Jiahao Lin, Alish Kanani, Miao Sun, Umit Y. Ogras, Jaehyun ParkAbstract
State Space Model (SSM)-based machine learning architectures have recently gained significant attention for processing sequential data. Mamba, a recent sequence-to-sequence SSM, offers competitive accuracy with superior computational efficiency compared to state-of-the-art transformer models. While this advantage makes Mamba particularly promising for resource-constrained edge devices, no hardware acceleration frameworks are currently optimized for deploying it in such environments. This paper presents eMamba, a comprehensive end-to-end hardware acceleration framework explicitly designed for deploying Mamba models on edge platforms. eMamba maximizes computational efficiency by replacing complex normalization layers with lightweight hardware-aware alternatives and approximating expensive operations, such as SiLU activation and exponentiation, considering the target applications. Then, it performs an approximation-aware neural architecture search (NAS) to tune the learnable parameters used during approximation. Evaluations with Fashion-MNIST, CIFAR-10, and MARS, an open-source human pose estimation dataset, show eMamba achieves comparable accuracy to state-of-the-art techniques using 1.63-19.9$\times$ fewer parameters. In addition, it generalizes well to large-scale natural language tasks, demonstrating stable perplexity across varying sequence lengths on the WikiText2 dataset. We also quantize and implement the entire eMamba pipeline on an AMD ZCU102 FPGA and ASIC using GlobalFoundries (GF) 22 nm technology. Experimental results show 4.95-5.62$\times$ lower latency and 2.22-9.95$\times$ higher throughput, with 4.77$\times$ smaller area, 9.84$\times$ lower power, and 48.6$\times$ lower energy consumption than baseline solutions while maintaining competitive accuracy.
Abstract (translated)
基于状态空间模型（SSM）的机器学习架构最近在处理序列数据方面引起了广泛关注。Mamba，一种近期推出的序列到序列SSM，在准确性上与最先进的Transformer模型相当，但在计算效率上更胜一筹。虽然这一优势使得Mamba特别有希望应用于资源受限的边缘设备中，但目前还没有专门针对此类环境部署Mamba的硬件加速框架。本文介绍了eMamba，这是一个全面的端到端硬件加速框架，专为在边缘平台上部署Mamba模型而设计。 eMamba通过用轻量级且适合硬件实现的替代层替换复杂的归一化层，并对昂贵的操作（如SiLU激活和指数运算）进行近似处理来最大化计算效率。同时，它还执行了感知近似的神经架构搜索（NAS），以调整在近似过程中使用的可学习参数。使用Fashion-MNIST、CIFAR-10以及开源人体姿态估计数据集MARS的评估显示，eMamba实现了与最新技术相当的准确性，并且所需的模型参数减少了1.63到19.9倍。 此外，它能够很好地泛化至大规模自然语言任务，在WikiText2数据集中，无论序列长度如何，都能表现出稳定的困惑度。我们还使用GlobalFoundries（GF）22nm技术在AMD ZCU102 FPGA和ASIC上实现了整个eMamba管道的量化实施。实验结果表明，与基线解决方案相比，eMamba的延迟降低了4.95到5.62倍，吞吐量提高了2.22到9.95倍，并且面积减少了4.77倍，功耗降低至原来的十分之一（9.84倍），能耗更是减少到了之前的四十八点六倍，同时保持了相当的准确性。
URL
https://arxiv.org/abs/2508.10370
PDF
-
Designing Object Detection Models for TinyML: Foundations, Comparative Analysis, Challenges, and Emerging Solutions
2025-08-11 17:28:59Christophe EL Zeinaty, Wassim Hamidouche, Glenn Herrou, Daniel MenardAbstract
Object detection (OD) has become vital for numerous computer vision applications, but deploying it on resource-constrained IoT devices presents a significant challenge. These devices, often powered by energy-efficient microcontrollers, struggle to handle the computational load of deep learning-based OD models. This issue is compounded by the rapid proliferation of IoT devices, predicted to surpass 150 billion by 2030. TinyML offers a compelling solution by enabling OD on ultra-low-power devices, paving the way for efficient and real-time processing at the edge. Although numerous survey papers have been published on this topic, they often overlook the optimization challenges associated with deploying OD models in TinyML environments. To address this gap, this survey paper provides a detailed analysis of key optimization techniques for deploying OD models on resource-constrained devices. These techniques include quantization, pruning, knowledge distillation, and neural architecture search. Furthermore, we explore both theoretical approaches and practical implementations, bridging the gap between academic research and real-world edge artificial intelligence deployment. Finally, we compare the key performance indicators (KPIs) of existing OD implementations on microcontroller devices, highlighting the achieved maturity level of these solutions in terms of both prediction accuracy and efficiency. We also provide a public repository to continually track developments in this fast-evolving field: this https URL.
Abstract (translated)
物体检测（OD）已成为众多计算机视觉应用的核心，但在资源受限的物联网设备上部署这一技术面临重大挑战。这些由节能微控制器驱动的设备难以处理基于深度学习的物体检测模型的计算负荷。此外，随着物联网设备数量的迅速增长——预计到2030年将超过150亿台——这个问题变得更加严峻。TinyML通过在超低功耗设备上启用物体检测提供了一个有吸引力的解决方案，从而为边缘端的有效实时处理铺平了道路。尽管已经发表了大量关于这一主题的研究综述文章，但它们往往忽略了与在TinyML环境中部署OD模型相关的优化挑战。为了弥补这一空白，本文对用于在资源受限设备上部署OD模型的关键优化技术进行了详细分析。这些技术包括量化、剪枝、知识蒸馏和神经架构搜索。 此外，我们还探讨了理论方法和实际应用之间的桥梁，弥合了学术研究与现实世界边缘人工智能部署之间的差距。最后，我们将现有的微控制器设备上的物体检测实现的关键性能指标（KPI）进行对比，并强调这些解决方案在预测准确性和效率方面所达到的成熟度水平。我们还提供了一个公开存储库以持续追踪这一快速发展的领域：[此链接](this https URL)。 --- 翻译： 物体检测（OD）已成为众多计算机视觉应用的核心，但在资源受限的物联网设备上部署该技术面临重大挑战。这些由节能微控制器驱动的设备难以处理基于深度学习的物体检测模型的计算负荷。随着物联网设备数量的迅速增长——预计到2030年将超过150亿台——这个问题变得更加严峻。TinyML通过在超低功耗设备上启用物体检测提供了一个有吸引力的解决方案，从而为边缘端的有效实时处理铺平了道路。 尽管已经发表了大量关于这一主题的研究综述文章，但它们往往忽略了与在TinyML环境中部署OD模型相关的优化挑战。为了弥补这一空白，本调查论文对用于在资源受限设备上部署物体检测模型的关键优化技术进行了详细分析。这些技术包括量化、剪枝、知识蒸馏和神经架构搜索。 此外，我们还探讨了理论方法和实际应用之间的桥梁，弥合了学术研究与现实世界边缘人工智能部署之间的差距。最后，我们将现有的微控制器设备上的物体检测实现的关键性能指标（KPI）进行了对比，并强调这些解决方案在预测准确性和效率方面所达到的成熟度水平。我们还提供了一个公开存储库以持续追踪这一快速发展的领域：[此链接](this https URL)。
URL
https://arxiv.org/abs/2508.08352
PDF
-
Empowering Time Series Forecasting with LLM-Agents
2025-08-06 09:14:08Chin-Chia Michael Yeh, Vivian Lai, Uday Singh Saini, Xiran Fan, Yujie Fan, Junpeng Wang, Xin Dai, Yan ZhengAbstract
Large Language Model (LLM) powered agents have emerged as effective planners for Automated Machine Learning (AutoML) systems. While most existing AutoML approaches focus on automating feature engineering and model architecture search, recent studies in time series forecasting suggest that lightweight models can often achieve state-of-the-art performance. This observation led us to explore improving data quality, rather than model architecture, as a potentially fruitful direction for AutoML on time series data. We propose DCATS, a Data-Centric Agent for Time Series. DCATS leverages metadata accompanying time series to clean data while optimizing forecasting performance. We evaluated DCATS using four time series forecasting models on a large-scale traffic volume forecasting dataset. Results demonstrate that DCATS achieves an average 6% error reduction across all tested models and time horizons, highlighting the potential of data-centric approaches in AutoML for time series forecasting.
Abstract (translated)
大型语言模型（LLM）驱动的代理已作为自动化机器学习（AutoML）系统中的有效规划者出现。尽管现有的大多数AutoML方法主要关注自动化的特征工程和模型架构搜索，但近期在时间序列预测领域的研究表明，轻量级模型往往可以达到最先进的性能水平。这一观察结果促使我们探索提升数据质量而非改进模型架构作为改善时间序列数据上的AutoML的一种潜在有效方向。为此，我们提出了DCATS（面向时间序列的数据中心代理）。DCATS利用伴随时间序列的元数据来清理数据并优化预测性能。 我们在一个大规模的交通流量预测数据集上使用四种时间序列预测模型对DCATS进行了评估。结果表明，与所有测试过的模型和时间范围相比，DCATS实现了平均6%的误差减少，这突显了在时间序列预测中采用数据为中心的方法对于AutoML的巨大潜力。
URL
https://arxiv.org/abs/2508.04231
PDF
-
FPG-NAS: FLOPs-Aware Gated Differentiable Neural Architecture Search for Efficient 6DoF Pose Estimation
2025-08-05 16:32:45Nassim Ali Ousalah, Peyman Rostami, Anis Kacem, Enjie Ghorbel, Emmanuel Koumandakis, Djamila AouadaAbstract
We introduce FPG-NAS, a FLOPs-aware Gated Differentiable Neural Architecture Search framework for efficient 6DoF object pose estimation. Estimating 3D rotation and translation from a single image has been widely investigated yet remains computationally demanding, limiting applicability in resource-constrained scenarios. FPG-NAS addresses this by proposing a specialized differentiable NAS approach for 6DoF pose estimation, featuring a task-specific search space and a differentiable gating mechanism that enables discrete multi-candidate operator selection, thus improving architectural diversity. Additionally, a FLOPs regularization term ensures a balanced trade-off between accuracy and efficiency. The framework explores a vast search space of approximately 10\textsuperscript{92} possible architectures. Experiments on the LINEMOD and SPEED+ datasets demonstrate that FPG-NAS-derived models outperform previous methods under strict FLOPs constraints. To the best of our knowledge, FPG-NAS is the first differentiable NAS framework specifically designed for 6DoF object pose estimation.
Abstract (translated)
我们介绍了FPG-NAS，这是一种基于FLOPs感知的门控可微神经架构搜索框架，旨在高效地进行六自由度（6DoF）物体姿态估计。从单张图像中估算3D旋转和位移已受到广泛研究，但仍具有较高的计算需求，在资源受限场景中的应用受到了限制。FPG-NAS通过提出一种专门针对6DoF姿态估计算法的可微神经架构搜索方法来解决这一问题，该方法包括任务特定的搜索空间以及一个可微门控机制，能够实现离散多候选操作选择，从而提高架构多样性。此外，引入了FLOPs正则化项以确保精度与效率之间的平衡。该框架探索了一个庞大的大约包含$10^{92}$种可能架构的搜索空间。在LINEMOD和SPEED+数据集上的实验表明，在严格的FLOPs约束条件下，由FPG-NAS生成的模型优于先前的方法。据我们所知，FPG-NAS是首个专门为六自由度物体姿态估计设计的可微神经架构搜索框架。
URL
https://arxiv.org/abs/2508.03618
PDF
-
Large-Scale Model Enabled Semantic Communication Based on Robust Knowledge Distillation
2025-08-04 07:47:18Kuiyuan DIng, Caili Guo, Yang Yang, Zhongtian Du, Walid SaadAbstract
Large-scale models (LSMs) can be an effective framework for semantic representation and understanding, thereby providing a suitable tool for designing semantic communication (SC) systems. However, their direct deployment is often hindered by high computational complexity and resource requirements. In this paper, a novel robust knowledge distillation based semantic communication (RKD-SC) framework is proposed to enable efficient and \textcolor{black}{channel-noise-robust} LSM-powered SC. The framework addresses two key challenges: determining optimal compact model architectures and effectively transferring knowledge while maintaining robustness against channel noise. First, a knowledge distillation-based lightweight differentiable architecture search (KDL-DARTS) algorithm is proposed. This algorithm integrates knowledge distillation loss and a complexity penalty into the neural architecture search process to identify high-performance, lightweight semantic encoder architectures. Second, a novel two-stage robust knowledge distillation (RKD) algorithm is developed to transfer semantic capabilities from an LSM (teacher) to a compact encoder (student) and subsequently enhance system robustness. To further improve resilience to channel impairments, a channel-aware transformer (CAT) block is introduced as the channel codec, trained under diverse channel conditions with variable-length outputs. Extensive simulations on image classification tasks demonstrate that the RKD-SC framework significantly reduces model parameters while preserving a high degree of the teacher model's performance and exhibiting superior robustness compared to existing methods.
Abstract (translated)
大规模模型（LSMs）可以作为语义表示和理解的有效框架，从而为设计语义通信（SC）系统提供合适的工具。然而，它们的直接部署通常会因高计算复杂性和资源需求而受阻。本文提出了一种基于鲁棒知识蒸馏的语义通信（RKD-SC）框架，旨在实现高效且抗信道噪声能力强的大规模模型驱动的SC。该框架解决了两个关键挑战：确定最优紧凑型模型架构和在保持对抗信道噪声能力的同时有效转移知识。 首先，提出了一种基于知识蒸馏的轻量级可微架构搜索（KDL-DARTS）算法。此算法将知识蒸馏损失与复杂性惩罚结合到神经架构搜索过程中，以识别高性能且轻量级的语义编码器架构。 其次，开发了一种新颖的两阶段鲁棒知识蒸馏（RKD）算法，用于从大规模模型（教师模型）向紧凑型编码器（学生模型）转移语义能力，并随后增强系统的稳健性。为提高对信道损害的进一步适应性，引入了具有感知信道的变压器（CAT）模块作为信道编解码器，在各种不同的信道条件下进行训练，并产生可变长度的输出。 广泛的图像分类任务模拟显示，RKD-SC框架在保持教师模型高性能的同时显著减少了模型参数数量，并且与现有方法相比表现出更优的鲁棒性。
URL
https://arxiv.org/abs/2508.02148
PDF
-
ASNN: Learning to Suggest Neural Architectures from Performance Distributions
2025-07-27 07:39:33Jinwook HongAbstract
The architecture of a neural network (NN) plays a critical role in determining its performance. However, there is no general closed-form function that maps between network structure and accuracy, making the process of architecture design largely heuristic or search-based. In this study, we propose the Architecture Suggesting Neural Network (ASNN), a model designed to learn the relationship between NN architecture and its test accuracy, and to suggest improved architectures accordingly. To train ASNN, we constructed datasets using TensorFlow-based models with varying numbers of layers and nodes. Experimental results were collected for both 2-layer and 3-layer architectures across a grid of configurations, each evaluated with 10 repeated trials to account for stochasticity. Accuracy values were treated as inputs, and architectural parameters as outputs. The trained ASNN was then used iteratively to predict architectures that yield higher performance. In both 2-layer and 3-layer cases, ASNN successfully suggested architectures that outperformed the best results found in the original training data. Repeated prediction and retraining cycles led to the discovery of architectures with improved mean test accuracies, demonstrating the model's capacity to generalize the performance-structure relationship. These results suggest that ASNN provides an efficient alternative to random search for architecture optimization, and offers a promising approach toward automating neural network design. "Parts of the manuscript, including text editing and expression refinement, were supported by OpenAI's ChatGPT. All content was reviewed and verified by the authors."
Abstract (translated)
神经网络（NN）的架构在其性能中扮演着至关重要的角色。然而，没有通用的闭式函数可以映射网络结构与准确度之间的关系，这使得架构设计过程主要依赖于启发法或搜索方法。在这项研究中，我们提出了架构建议神经网络（ASNN），这是一种旨在学习NN架构与其测试精度之间关系，并据此推荐改进后的架构模型。 为了训练ASNN，我们使用基于TensorFlow的具有不同层数和节点数的模型构建了数据集。对于两层和三层架构，在一系列配置网格上收集实验结果，每个配置通过10次重复试验来考虑随机性的影响。将精度值作为输入处理，并将架构参数视为输出。训练后的ASNN随后被迭代地用于预测性能更高的架构。 在二层和三层结构的情况下，ASNN成功提出了优于原始训练数据中最佳结果的架构。反复进行预测和重新训练循环发现具有改进平均测试准确度的架构，展示了该模型概括性能-结构关系的能力。这些结果表明，ASNN为架构优化提供了一种比随机搜索更有效的替代方案，并且在自动化的神经网络设计方面展现出了有前景的方法。 “手稿的部分内容，包括文本编辑和表达润色，得到了OpenAI的ChatGPT的支持。所有内容均已由作者审查并确认。”
URL
https://arxiv.org/abs/2507.20164
PDF
-
Pareto-NRPA: A Novel Monte-Carlo Search Algorithm for Multi-Objective Optimization
2025-07-25 09:46:25No\'e Lallouet, Tristan Cazenave, Cyrille EnderliAbstract
We introduce Pareto-NRPA, a new Monte-Carlo algorithm designed for multi-objective optimization problems over discrete search spaces. Extending the Nested Rollout Policy Adaptation (NRPA) algorithm originally formulated for single-objective problems, Pareto-NRPA generalizes the nested search and policy update mechanism to multi-objective optimization. The algorithm uses a set of policies to concurrently explore different regions of the solution space and maintains non-dominated fronts at each level of search. Policy adaptation is performed with respect to the diversity and isolation of sequences within the Pareto front. We benchmark Pareto-NRPA on two classes of problems: a novel bi-objective variant of the Traveling Salesman Problem with Time Windows problem (MO-TSPTW), and a neural architecture search task on well-known benchmarks. Results demonstrate that Pareto-NRPA achieves competitive performance against state-of-the-art multi-objective algorithms, both in terms of convergence and diversity of solutions. Particularly, Pareto-NRPA strongly outperforms state-of-the-art evolutionary multi-objective algorithms on constrained search spaces. To our knowledge, this work constitutes the first adaptation of NRPA to the multi-objective setting.
Abstract (translated)
我们引入了Pareto-NRPA，这是一种为离散搜索空间中的多目标优化问题设计的新蒙特卡洛算法。该算法扩展了最初为单目标问题制定的嵌套滚动策略调整（NRPA）算法，并将嵌套搜索和策略更新机制推广到了多目标优化中。该算法使用一组策略同时探索解决方案空间的不同区域，并在每一级搜索过程中保持非支配前沿。通过序列间的多样性和隔离性，对策略进行适应性调整。我们在两类问题上对Pareto-NRPA进行了基准测试：一种是旅行推销员问题的时间窗口变体的新型双目标版本（MO-TSPTW），另一种是在知名基准上的神经架构搜索任务。结果表明，Pareto-NRPA在收敛性和解决方案多样性方面均达到了与现有最佳多目标算法相当甚至更好的性能。特别是在受限制的搜索空间中，Pareto-NRPA显著优于现有的进化型多目标优化算法。据我们所知，这项工作首次将NRPA应用于多目标场景。
URL
https://arxiv.org/abs/2507.19109
PDF
-
AlphaGo Moment for Model Architecture Discovery
2025-07-24 03:57:27Yixiu Liu, Yang Nan, Weixian Xu, Xiangkun Hu, Lyumanshan Ye, Zhen Qin, Pengfei LiuAbstract
While AI systems demonstrate exponentially improving capabilities, the pace of AI research itself remains linearly bounded by human cognitive capacity, creating an increasingly severe development bottleneck. We present ASI-Arch, the first demonstration of Artificial Superintelligence for AI research (ASI4AI) in the critical domain of neural architecture discovery--a fully autonomous system that shatters this fundamental constraint by enabling AI to conduct its own architectural innovation. Moving beyond traditional Neural Architecture Search (NAS), which is fundamentally limited to exploring human-defined spaces, we introduce a paradigm shift from automated optimization to automated innovation. ASI-Arch can conduct end-to-end scientific research in the domain of architecture discovery, autonomously hypothesizing novel architectural concepts, implementing them as executable code, training and empirically validating their performance through rigorous experimentation and past experience. ASI-Arch conducted 1,773 autonomous experiments over 20,000 GPU hours, culminating in the discovery of 106 innovative, state-of-the-art (SOTA) linear attention architectures. Like AlphaGo's Move 37 that revealed unexpected strategic insights invisible to human players, our AI-discovered architectures demonstrate emergent design principles that systematically surpass human-designed baselines and illuminate previously unknown pathways for architectural innovation. Crucially, we establish the first empirical scaling law for scientific discovery itself--demonstrating that architectural breakthroughs can be scaled computationally, transforming research progress from a human-limited to a computation-scalable process. We provide comprehensive analysis of the emergent design patterns and autonomous research capabilities that enabled these breakthroughs, establishing a blueprint for self-accelerating AI systems.
Abstract (translated)
尽管人工智能系统的功能呈指数级增长，但其研究本身仍然受到人类认知能力的线性限制，导致发展瓶颈日益严重。我们提出了ASI-Arch，这是第一个用于AI研究（ASI4AI）的人工超级智能演示，在关键领域——神经架构发现中的应用——这是一个完全自主系统，通过使AI能够进行自我创新来打破这一根本约束。 超越传统基于人类定义空间的神经架构搜索（NAS），我们的方法引入了一种从自动化优化向自动化创新转变的范式。ASI-Arch可以在架构发现领域进行全面的科学研究，它能自主提出新颖的概念构想，并将这些想法实现为可执行代码，通过严格的实验和以往的经验进行训练与实证验证其性能。 在17,73个自主实验中，经过20,000小时GPU计算时间的努力，ASI-Arch发现了106种创新的、最先进的线性注意力架构。就像AlphaGo的第37步揭示了人类棋手无法预见的战略洞察一样，我们通过AI发现的架构也展示了系统性的超出人类设计基准的设计原则，并揭示了以前未知的架构创新途径。 尤为重要的是，我们建立了首个关于科学发现本身的实证规模法则——证明了架构突破可以被计算化扩展，从而将研究进步从受制于人的限制转变为可计算扩增的过程。我们对这些突破背后的自出现模式和自主科研能力进行了全面分析，并确立了一个自我加速AI系统的蓝图。
URL
https://arxiv.org/abs/2507.18074
PDF
-
confopt: A Library for Implementation and Evaluation of Gradient-based One-Shot NAS Methods
2025-07-22 12:44:28Abhash Kumar Jha, Shakiba Moradian, Arjun Krishnakumar, Martin Rapp, Frank HutterAbstract
Gradient-based one-shot neural architecture search (NAS) has significantly reduced the cost of exploring architectural spaces with discrete design choices, such as selecting operations within a model. However, the field faces two major challenges. First, evaluations of gradient-based NAS methods heavily rely on the DARTS benchmark, despite the existence of other available benchmarks. This overreliance has led to saturation, with reported improvements often falling within the margin of noise. Second, implementations of gradient-based one-shot NAS methods are fragmented across disparate repositories, complicating fair and reproducible comparisons and further development. In this paper, we introduce Configurable Optimizer (confopt), an extensible library designed to streamline the development and evaluation of gradient-based one-shot NAS methods. Confopt provides a minimal API that makes it easy for users to integrate new search spaces, while also supporting the decomposition of NAS optimizers into their core components. We use this framework to create a suite of new DARTS-based benchmarks, and combine them with a novel evaluation protocol to reveal a critical flaw in how gradient-based one-shot NAS methods are currently assessed. The code can be found at this https URL.
Abstract (translated)
基于梯度的一次性神经架构搜索（NAS）显著降低了探索包含离散设计选择的架构空间的成本，例如在模型中选择操作。然而，该领域面临着两个主要挑战。首先，基于梯度的NAS方法的评估严重依赖于DARTS基准，尽管存在其他可用的基准。这种过度依赖导致了饱和状态，报告的改进往往处于噪声范围内。其次，基于梯度的一次性NAS方法的实现分散在不同的仓库中，这使得公平和可重复的比较及进一步开发变得复杂。 在这篇论文中，我们介绍了Configurable Optimizer (confopt)，这是一个可扩展的库，旨在简化基于梯度的一次性NAS方法的发展和评估。Confopt提供了一个最小化的API，使用户能够轻松集成新的搜索空间，并支持将NAS优化器分解为其核心组件。利用这一框架，我们创建了一套全新的DARTS基准测试集，并结合一种新颖的评价协议揭示了当前对基于梯度的一次性NAS方法评估的关键缺陷。 代码可在以下网址找到：[此URL]（请使用原文中的实际链接）。
URL
https://arxiv.org/abs/2507.16533
PDF
-
Neural Architecture Search with Mixed Bio-inspired Learning Rules
Abstract
Bio-inspired neural networks are attractive for their adversarial robustness, energy frugality, and closer alignment with cortical physiology, yet they often lag behind back-propagation (BP) based models in accuracy and ability to scale. We show that allowing the use of different bio-inspired learning rules in different layers, discovered automatically by a tailored neural-architecture-search (NAS) procedure, bridges this gap. Starting from standard NAS baselines, we enlarge the search space to include bio-inspired learning rules and use NAS to find the best architecture and learning rule to use in each layer. We show that neural networks that use different bio-inspired learning rules for different layers have better accuracy than those that use a single rule across all the layers. The resulting NN that uses a mix of bio-inspired learning rules sets new records for bio-inspired models: 95.16% on CIFAR-10, 76.48% on CIFAR-100, 43.42% on ImageNet16-120, and 60.51% top-1 on ImageNet. In some regimes, they even surpass comparable BP-based networks while retaining their robustness advantages. Our results suggest that layer-wise diversity in learning rules allows better scalability and accuracy, and motivates further research on mixing multiple bio-inspired learning rules in the same network.
Abstract (translated)
受生物启发的神经网络因其对抗鲁棒性、节能性和与皮质生理学更接近的一致性而受到青睐，但它们在准确性及扩展能力方面通常落后于基于反向传播（BP）的模型。我们证明，允许不同层使用由定制化的神经架构搜索（NAS）过程自动发现的不同生物启发式学习规则，在一定程度上弥合了这一差距。从标准NAS基线开始，我们将搜索空间扩大以包括生物启发式学习规则，并利用NAS来寻找在每一层中使用的最佳架构和学习规则。 我们展示了使用不同生物启发式学习规则的神经网络比在整个网络中使用单一规则具有更高的准确性。这些混合了多种生物启发式学习规则的结果神经网络为基于生物启发的学习模型设定了新的记录：CIFAR-10上的准确率为95.16%，CIFAR-100上的准确率为76.48%，ImageNet16-120上的准确率为43.42%，以及在ImageNet上达到60.51%的top-1准确率。在某些情况下，这些模型甚至超越了与其性能相当的BP基网络，同时保持了它们的稳健性优势。 我们的研究结果表明，在不同层使用多样化的学习规则有助于提高扩展性和准确性，并鼓励进一步探索在同一神经网络中混合多种生物启发式学习规则的研究方向。
URL
https://arxiv.org/abs/2507.13485
PDF
-
DASViT: Differentiable Architecture Search for Vision Transformer
2025-07-17 12:48:00Pengjin Wu, Ferrante Neri, Zhenhua FengAbstract
Designing effective neural networks is a cornerstone of deep learning, and Neural Architecture Search (NAS) has emerged as a powerful tool for automating this process. Among the existing NAS approaches, Differentiable Architecture Search (DARTS) has gained prominence for its efficiency and ease of use, inspiring numerous advancements. Since the rise of Vision Transformers (ViT), researchers have applied NAS to explore ViT architectures, often focusing on macro-level search spaces and relying on discrete methods like evolutionary algorithms. While these methods ensure reliability, they face challenges in discovering innovative architectural designs, demand extensive computational resources, and are time-intensive. To address these limitations, we introduce Differentiable Architecture Search for Vision Transformer (DASViT), which bridges the gap in differentiable search for ViTs and uncovers novel designs. Experiments show that DASViT delivers architectures that break traditional Transformer encoder designs, outperform ViT-B/16 on multiple datasets, and achieve superior efficiency with fewer parameters and FLOPs.
Abstract (translated)
设计有效的神经网络是深度学习的核心，而神经架构搜索（NAS）作为自动化这一过程的强有力工具已经崭露头角。在现有的NAS方法中，可微分架构搜索（DARTS）因其效率和易用性而备受推崇，并激发了众多的进步与发展。随着视觉变换器（ViT）的兴起，研究人员开始将NAS应用于探索ViT架构，通常集中在宏观级别的搜索空间上，并依赖于如进化算法之类的离散方法。尽管这些方法能够保证可靠性，但它们在发现创新性的架构设计方面存在挑战，同时需要大量的计算资源和时间。 为了解决这些问题，我们提出了用于视觉变换器（Vision Transformer）的可微分架构搜索（DASViT）。这一方法填补了ViTs中不同iable搜索的空白，并揭示出新颖的设计。实验表明，DASViT能够打破传统的Transformer编码器设计模式，在多个数据集上超越了ViT-B/16模型，并且在使用更少参数和计算量的情况下达到了更高的效率。 通过这种方式，DASViT不仅为ViTs架构搜索提供了一种新的视角，而且还展示了可微分搜索方法的巨大潜力。
URL
https://arxiv.org/abs/2507.13079
PDF
-
BenchRL-QAS: Benchmarking reinforcement learning algorithms for quantum architecture search
2025-07-16 12:43:25Azhar Ikhtiarudin, Aditi Das, Param Thakkar, Akash KunduAbstract
We introduce BenchRL-QAS, a unified benchmarking framework for systematically evaluating reinforcement learning (RL) algorithms in quantum architecture search (QAS) across diverse variational quantum algorithm tasks and system sizes ranging from 2- to 8-qubit. Our study benchmarks nine RL agents including both value-based and policy-gradient methods on representative quantum problems such as variational quantum eigensolver, variational quantum state diagonalization, quantum classification, and state preparation, spanning both noiseless and realistic noisy regimes. We propose a weighted ranking metric that balances accuracy, circuit depth, gate count, and computational efficiency, enabling fair and comprehensive comparison. Our results first reveal that RL-based quantum classifier outperforms baseline variational classifiers. Then we conclude that no single RL algorithm is universally optimal when considering a set of QAS tasks; algorithmic performance is highly context-dependent, varying with task structure, qubit count, and noise. This empirical finding provides strong evidence for the "no free lunch" principle in RL-based quantum circuit design and highlights the necessity of tailored algorithm selection and systematic benchmarking for advancing quantum circuit synthesis. This work represents the most comprehensive RL-QAS benchmarking effort to date, and BenchRL-QAS along with all experimental data are made publicly available to support reproducibility and future research this https URL.
Abstract (translated)
我们介绍了BenchRL-QAS，这是一个统一的基准框架，用于系统地评估量子架构搜索（QAS）中强化学习（RL）算法在各种变分量子算法任务和从2到8个量子位的不同规模系统的性能。我们的研究对包括基于价值的方法和策略梯度方法在内的九种RL代理进行了基准测试，这些代理在具有代表性的量子问题上进行评估，例如变分量子特征值求解器、变分量子态对角化、量子分类以及状态准备，在无噪声和现实中的有噪声环境中均进行了考虑。我们提出了一项加权排名指标，该指标平衡了准确性、电路深度、门数和计算效率，以实现公平且全面的比较。我们的研究首次揭示基于RL的量子分类器优于基线变分分类器的表现。然后我们得出结论：当考虑到一组QAS任务时，并没有一种通用的最佳RL算法；其性能高度依赖于具体环境，随着任务结构、量子位数量以及噪声程度的不同而变化。这一实证发现为基于强化学习的量子电路设计中的“不存在免费午餐”原则提供了强有力的支持，并强调了定制算法选择和系统基准测试对于推进量子电路合成的重要性。这项工作代表了迄今为止最全面的RL-QAS基准测试努力，BenchRL-QAS以及所有实验数据均向公众开放以支持可重复性和未来研究（请访问此链接获取更多信息：此 https URL）。
URL
https://arxiv.org/abs/2507.12189
PDF
-
An Memory-Efficient Framework for Deformable Transformer with Neural Architecture Search
2025-07-13 06:42:00Wendong Mao, Mingfan Zhao, Jianfeng Guan, Qiwei Dong, Zhongfeng WangAbstract
Deformable Attention Transformers (DAT) have shown remarkable performance in computer vision tasks by adaptively focusing on informative image regions. However, their data-dependent sampling mechanism introduces irregular memory access patterns, posing significant challenges for efficient hardware deployment. Existing acceleration methods either incur high hardware overhead or compromise model accuracy. To address these issues, this paper proposes a hardware-friendly optimization framework for DAT. First, a neural architecture search (NAS)-based method with a new slicing strategy is proposed to automatically divide the input feature into uniform patches during the inference process, avoiding memory conflicts without modifying model architecture. The method explores the optimal slice configuration by jointly optimizing hardware cost and inference accuracy. Secondly, an FPGA-based verification system is designed to test the performance of this framework on edge-side hardware. Algorithm experiments on the ImageNet-1K dataset demonstrate that our hardware-friendly framework can maintain have only 0.2% accuracy drop compared to the baseline DAT. Hardware experiments on Xilinx FPGA show the proposed method reduces DRAM access times to 18% compared with existing DAT acceleration methods.
Abstract (translated)
变形注意力变换器（DAT）在计算机视觉任务中表现出色，能够通过自适应聚焦于图像的显著区域来提升性能。然而，它们的数据依赖采样机制引入了不规则的记忆访问模式，这对高效的硬件部署提出了重大挑战。现有的加速方法要么导致较高的硬件开销，要么牺牲模型精度。为解决这些问题，本文提出了一种针对DAT友好的硬件优化框架。 首先，该研究提出了一种基于神经架构搜索（NAS）的新切片策略的方法，在推断过程中自动将输入特征划分为均匀的补丁，从而避免了内存冲突，且无需修改模型结构。此方法通过同时优化硬件成本和推理精度来探索最佳切片配置。其次，设计了一个基于FPGA的验证系统，用于测试该框架在边缘侧硬件上的性能。 在ImageNet-1K数据集上的算法实验表明，与基准DAT相比，我们的硬件友好型框架仅导致0.2%的准确率下降。而在Xilinx FPGA上的硬件实验显示，所提出的方法将DRAM访问时间减少了18%，相比于现有的DAT加速方法。
URL
https://arxiv.org/abs/2507.11549
PDF
-
Zero-Shot Neural Architecture Search with Weighted Response Correlation
2025-07-08 02:19:29Kun Jing, Luoyu Chen, Jungang Xu, Jianwei Tai, Yiyu Wang, Shuaimin LiAbstract
Neural architecture search (NAS) is a promising approach for automatically designing neural network architectures. However, the architecture estimation of NAS is computationally expensive and time-consuming because of training multiple architectures from scratch. Although existing zero-shot NAS methods use training-free proxies to accelerate the architecture estimation, their effectiveness, stability, and generality are still lacking. We present a novel training-free estimation proxy called weighted response correlation (WRCor). WRCor utilizes correlation coefficient matrices of responses across different input samples to calculate the proxy scores of estimated architectures, which can measure their expressivity and generalizability. Experimental results on proxy evaluation demonstrate that WRCor and its voting proxies are more efficient estimation strategies than existing proxies. We also apply them with different search strategies in architecture search. Experimental results on architecture search show that our zero-shot NAS algorithm outperforms most existing NAS algorithms in different search spaces. Our NAS algorithm can discover an architecture with a 22.1% test error on the ImageNet-1k dataset within 4 GPU hours. All codes are publicly available at this https URL.
Abstract (translated)
神经架构搜索（NAS）是一种有前景的自动设计神经网络架构的方法。然而，由于需要从头开始训练多个架构，NAS中的架构评估计算成本高且耗时。尽管现有的零样本NAS方法使用无训练代理来加速架构估计，但它们的有效性、稳定性和通用性仍然不足。我们提出了一种名为加权响应相关（WRCor）的新颖无训练估算代理。WRCor利用不同输入样本之间的响应的相关系数矩阵来计算所估架构的代理分数，以衡量其表达能力和泛化能力。在代理评估实验中表明，与现有代理相比，WRCor及其投票代理是更高效的估计策略。我们还将其与不同的搜索策略结合应用于架构搜索中。在架构搜索方面的实验证明，在各种搜索空间中，我们的零样本NAS算法优于大多数现有的NAS算法。我们的NAS算法能够在4个GPU小时内发现一个在ImageNet-1k数据集上具有22.1%测试错误率的架构。所有代码都公开发布在此网址：[此URL]（请使用原文中的链接）。
URL
https://arxiv.org/abs/2507.08841
PDF
