Abstract
Deep learning (DL) has achieved remarkable progress in the field of medical imaging. However, adapting DL models to medical tasks remains a significant challenge, primarily due to two key factors: (1) architecture selection, as different tasks necessitate specialized model designs, and (2) weight initialization, which directly impacts the convergence speed and final performance of the models. Although transfer learning from ImageNet is a widely adopted strategy, its effectiveness is constrained by the substantial differences between natural and medical images. To address these challenges, we introduce Medical Neural Network Search (MedNNS), the first Neural Network Search framework for medical imaging applications. MedNNS jointly optimizes architecture selection and weight initialization by constructing a meta-space that encodes datasets and models based on how well they perform together. We build this space using a Supernetwork-based approach, expanding the model zoo size by 51x times over previous state-of-the-art (SOTA) methods. Moreover, we introduce rank loss and Fréchet Inception Distance (FID) loss into the construction of the space to capture inter-model and inter-dataset relationships, thereby achieving more accurate alignment in the meta-space. Experimental results across multiple datasets demonstrate that MedNNS significantly outperforms both ImageNet pre-trained DL models and SOTA Neural Architecture Search (NAS) methods, achieving an average accuracy improvement of 1.7% across datasets while converging substantially faster. The code and the processed meta-space is available at this https URL.
Abstract (translated)
深度学习（DL）在医学成像领域取得了显著进展。然而，将深度学习模型应用于医疗任务仍面临重大挑战，主要是由于两个关键因素：一是架构选择，不同的任务需要专门设计的模型；二是权重初始化，这直接影响模型的收敛速度和最终性能。虽然从ImageNet进行迁移学习是一种广泛采用的战略，但其有效性受限于自然图像与医学图像之间的巨大差异。为解决这些挑战，我们引入了Medical Neural Network Search（MedNNS），这是首个针对医学成像应用的神经网络搜索框架。MedNNS通过构建一个元空间来联合优化架构选择和权重初始化，该元空间根据数据集和模型共同表现的程度编码它们。 我们采用基于超级网络的方法建立这一空间，并将模型动物园的规模扩大了51倍，超过了以往最先进的（SOTA）方法。此外，我们在空间构造中引入了等级损失和Fréchet inception距离（FID）损失，以捕捉跨模型及跨数据集的关系，从而在元空间内实现更精确的对齐。 多项实验结果显示，MedNNS显著优于ImageNet预训练的深度学习模型以及最先进的神经架构搜索（NAS）方法，在多个数据集中平均准确率提升了1.7%，同时收敛速度更快。该代码和处理后的元空间可在以下链接获取：[提供具体URL]。
URL
https://arxiv.org/abs/2504.15865
