Abstract
Medical ultrasonography is an essential imaging technique for examining superficial organs and tissues, including lymph nodes, breast, and thyroid. It employs high-frequency ultrasound waves to generate detailed images of the internal structures of the human body. However, manually contouring regions of interest in these images is a labor-intensive task that demands expertise and often results in inconsistent interpretations among individuals. Vision-language foundation models, which have excelled in various computer vision applications, present new opportunities for enhancing ultrasound image analysis. Yet, their performance is hindered by the significant differences between natural and medical imaging domains. This research seeks to overcome these challenges by developing domain adaptation methods for vision-language foundation models. In this study, we explore the fine-tuning pipeline for vision-language foundation models by utilizing large language model as text refiner with special-designed adaptation strategies and task-driven heads. Our approach has been extensively evaluated on six ultrasound datasets and two tasks: segmentation and classification. The experimental results show that our method can effectively improve the performance of vision-language foundation models for ultrasound image analysis, and outperform the existing state-of-the-art vision-language and pure foundation models. The source code of this study is available at \href{this https URL}{GitHub}.
Abstract (translated)
医学超声成像是检查浅表器官和组织（包括淋巴结、乳腺和甲状腺）的重要影像技术。它利用高频超声波生成人体内部结构的详细图像。然而，手动勾画这些图像中感兴趣区域轮廓是一项劳动密集型任务，需要专业知识，并且通常会导致个体间解释的一致性问题。视觉-语言基础模型在各种计算机视觉应用中的表现优异，为提升超声影像分析提供了新的机会。然而，由于自然和医学成像领域之间的显著差异，其性能受到了限制。 本研究旨在通过开发面向视觉-语言基础模型的领域适应方法来克服这些挑战。在这项研究中，我们探索了使用大规模语言模型作为文本精炼器并结合特殊设计的适应策略和任务驱动头部的微调流程。我们的方法已经在六种超声数据集以及分割和分类两项任务上进行了广泛的评估。实验结果表明，本方法能够有效提升视觉-语言基础模型在超声图像分析中的性能，并优于现有的最先进的视觉-语言及纯基础模型。 这项研究的源代码可在[GitHub](https://this%20URL)获取。
URL
https://arxiv.org/abs/2506.08849
