Abstract
AI Foundation models are gaining traction in various applications, including medical fields like radiology. However, medical foundation models are often tested on limited tasks, leaving their generalisability and biases unexplored. We present RayDINO, a large visual encoder trained by self-supervision on 873k chest X-rays. We compare RayDINO to previous state-of-the-art models across nine radiology tasks, from classification and dense segmentation to text generation, and provide an in depth analysis of population, age and sex biases of our model. Our findings suggest that self-supervision allows patient-centric AI proving useful in clinical workflows and interpreting X-rays holistically. With RayDINO and small task-specific adapters, we reach state-of-the-art results and improve generalization to unseen populations while mitigating bias, illustrating the true promise of foundation models: versatility and robustness.
Abstract (translated)
AI 基础模型在各种应用领域都取得了越来越广泛的应用，包括医学领域如 radiology。然而，这些模型通常仅在有限的任务上进行测试，导致其泛化能力和偏见未被探索。我们提出了 RayDINO，一种通过自监督学习在 873k 张胸部 X 光片上进行训练的大视觉编码器。我们比较了 RayDINO 与之前最先进的模型在九个放射学任务上的效果，从分类和密集分割到文本生成，并对我们模型的Population、Age和Gender偏见进行了深入分析。我们的研究结果表明，自监督训练使患者为中心的 AI 在临床工作流程中具有实际价值，并能够全面地解释 X 光片。与 RayDINO 和针对具体任务的适配器相结合，我们达到了最先进的结果，并在未见过的受众上进行了泛化能力的提高，同时减轻了偏见，证明了基础模型的真正价值：多样性和稳健性。
URL
https://arxiv.org/abs/2405.01469
