Abstract
The amount of medical images stored in hospitals is increasing faster than ever; however, utilizing the accumulated medical images has been limited. This is because existing content-based medical image retrieval (CBMIR) systems usually require example images to construct query vectors; nevertheless, example images cannot always be prepared. Besides, there can be images with rare characteristics that make it difficult to find similar example images, which we call isolated samples. Here, we introduce a novel sketch-based medical image retrieval (SBMIR) system that enables users to find images of interest without example images. The key idea lies in feature decomposition of medical images, whereby the entire feature of a medical image can be decomposed into and reconstructed from normal and abnormal features. By extending this idea, our SBMIR system provides an easy-to-use two-step graphical user interface: users first select a template image to specify a normal feature and then draw a semantic sketch of the disease on the template image to represent an abnormal feature. Subsequently, it integrates the two kinds of input to construct a query vector and retrieves reference images with the closest reference vectors. Using two datasets, ten healthcare professionals with various clinical backgrounds participated in the user test for evaluation. As a result, our SBMIR system enabled users to overcome previous challenges, including image retrieval based on fine-grained image characteristics, image retrieval without example images, and image retrieval for isolated samples. Our SBMIR system achieves flexible medical image retrieval on demand, thereby expanding the utility of medical image databases.
Abstract (translated)
医院中存储的医疗图像数量正在以前所未有的速度增加,然而,利用累积的医疗图像仍然受到限制。这是因为现有的基于内容的医学图像检索(CBMIR)系统通常需要示例图像来构建查询向量,但示例图像通常无法 always 准备。此外,可能存在具有罕见特征的图像,使得难以找到类似示例图像,我们称之为孤立样本。在这里,我们介绍了一种新的基于 Sketch 的医疗图像检索系统(SBMIR),它使用户可以在没有示例图像的情况下找到感兴趣的图像。关键思想在于医疗图像的特征分解,从而使医疗图像的所有特征可以从正常和异常特征中分解和重构。通过扩展这个思想,我们的 SBMIR 系统提供了易于使用的两个步骤图形用户界面:用户首先选择一个模板图像以指定正常特征,然后在该模板图像上绘制疾病语义 Sketch 以表示异常特征。随后,它整合两种输入来构建查询向量,并检索最接近参考向量的恢复图像。使用两个数据集,十名来自不同临床背景的医疗保健专业人员参加了用户测试,评估其性能。因此,我们的 SBMIR 系统使用户可以克服以前的挑战,包括基于精细图像特征的图像检索、在没有示例图像的情况下进行图像检索以及孤立样本的图像检索。我们的 SBMIR 系统实现了灵活的医学图像检索,从而扩大了医学图像数据库的实用性。
URL
https://arxiv.org/abs/2303.03633
