Abstract
Robotic ultrasound (US) systems have shown great potential to make US examinations easier and more accurate. Recently, various machine learning techniques have been proposed to realize automatic US image interpretation for robotic US acquisition tasks. However, obtaining large amounts of real US imaging data for training is usually expensive or even unfeasible in some clinical applications. An alternative is to build a simulator to generate synthetic US data for training, but the differences between simulated and real US images may result in poor model performance. This work presents a Sim2Real framework to efficiently learn robotic US image analysis tasks based only on simulated data for real-world deployment. A style transfer module is proposed based on unsupervised contrastive learning and used as a preprocessing step to convert the real US images into the simulation style. Thereafter, a task-relevant model is designed to combine CNNs with vision transformers to generate the task-dependent prediction with improved generalization ability. We demonstrate the effectiveness of our method in an image regression task to predict the probe position based on US images in robotic transesophageal echocardiography (TEE). Our results show that using only simulated US data and a small amount of unlabelled real data for training, our method can achieve comparable performance to semi-supervised and fully supervised learning methods. Moreover, the effectiveness of our previously proposed CT-based US image simulation method is also indirectly confirmed.
Abstract (translated)
机器人超声波(US)系统已经展现出了让超声波检查更加容易和更准确的潜力。近年来,已经提出了多种机器学习技术来实现机器人超声波摄取任务中的自动超声波图像解释。然而,获取大量的真实超声波图像用于训练通常非常昂贵或在某些临床应用程序中甚至不可行。一种替代方法是建立一个模拟器来生成合成的超声波数据用于训练,但模拟和真实超声波图像之间的差异可能会导致模型性能不佳。本文提出了一个Sim2Real框架,以高效地学习机器人超声波图像分析任务,仅基于模拟数据进行实际部署。基于无监督比较学习的样式转移模块被提出,并用作预处理步骤,将真实的超声波图像转换为模拟风格。此后,一个与任务相关的模型被设计,结合卷积神经网络和视觉转换器,生成任务相关的增强泛化能力的预测。我们证明了我们的方法在图像回归任务中,以预测机器人超声波心动图(TEE)中探针位置的超声波图像,的方法的有效性。我们的结果表明,仅使用模拟的超声波数据和少量的未标记的真实数据用于训练,我们的方法可以与半监督和完全监督学习方法相媲美。此外,我们之前提出的基于CT的超声波图像模拟方法的效果也间接得到了确认。
URL
https://arxiv.org/abs/2305.09169