Abstract
Several studies indicate that deep learning models can learn to detect breast cancer from mammograms (X-ray images of the breasts). However, challenges with overfitting and poor generalisability prevent their routine use in the clinic. Models trained on data from one patient population may not perform well on another due to differences in their data domains, emerging due to variations in scanning technology or patient characteristics. Data augmentation techniques can be used to improve generalisability by expanding the diversity of feature representations in the training data by altering existing examples. Image-to-image translation models are one approach capable of imposing the characteristic feature representations (i.e. style) of images from one dataset onto another. However, evaluating model performance is non-trivial, particularly in the absence of ground truths (a common reality in medical imaging). Here, we describe some key aspects that should be considered when evaluating style transfer algorithms, highlighting the advantages and disadvantages of popular metrics, and important factors to be mindful of when implementing them in practice. We consider two types of generative models: a cycle-consistent generative adversarial network (CycleGAN) and a diffusion-based SynDiff model. We learn unpaired image-to-image translation across three mammography datasets. We highlight that undesirable aspects of model performance may determine the suitability of some metrics, and also provide some analysis indicating the extent to which various metrics assess unique aspects of model performance. We emphasise the need to use several metrics for a comprehensive assessment of model performance.
Abstract (translated)
几项研究表明，深度学习模型可以从乳房X光片（乳腺的X射线图像）中学会检测乳腺癌。然而，由于过拟合和较差的泛化能力问题，这些模型还不能在临床上常规使用。在一个患者群体的数据上训练出的模型可能无法很好地应用于其他群体，这是因为不同数据集之间的扫描技术和患者特征差异导致的数据域不一致。通过数据增强技术可以改进模型的泛化能力，例如通过对现有样本进行修改来扩展训练数据中的特征表示多样性。图像到图像转换模型是一种能够将一个数据集中图像的独特特征表现（即风格）施加于另一个数据集的方法。然而，在缺乏地面真实值的情况下评估模型性能颇具挑战性，这是医学影像中常见的现实问题。在此，我们描述了在评估样式迁移算法时应考虑的一些关键方面，并强调了一些流行指标的优点和缺点以及在实践中实施这些指标的重要因素。我们将探讨两种生成模型：循环一致对抗网络（CycleGAN）和基于扩散的SynDiff模型，在三个乳腺X射线数据集中学习未配对图像到图像的转换。我们指出，模型性能中的某些不良方面可能会影响一些度量标准的适用性，并提供了一些分析以说明各种指标在评估模型独特性能方面的程度。我们强调需要使用多个指标来全面评估模型性能。
URL
https://arxiv.org/abs/2502.02475
