Abstract
Annotating lots of 3D medical images for training segmentation models is time-consuming. The goal of weakly supervised semantic segmentation is to train segmentation models without using any ground truth segmentation masks. Our work addresses the case where only image-level categorical labels, indicating the presence or absence of a particular region of interest (such as tumours or lesions), are available. Most existing methods rely on class activation mapping (CAM). We propose a novel approach, ToNNO, which is based on the Tomographic reconstruction of a Neural Network's Output. Our technique extracts stacks of slices with different angles from the input 3D volume, feeds these slices to a 2D encoder, and applies the inverse Radon transform in order to reconstruct a 3D heatmap of the encoder's predictions. This generic method allows to perform dense prediction tasks on 3D volumes using any 2D image encoder. We apply it to weakly supervised medical image segmentation by training the 2D encoder to output high values for slices containing the regions of interest. We test it on four large scale medical image datasets and outperform 2D CAM methods. We then extend ToNNO by combining tomographic reconstruction with CAM methods, proposing Averaged CAM and Tomographic CAM, which obtain even better results.
Abstract (translated)
给大量的3D医疗图像进行注释是一项耗时的工作。弱监督语义分割的目标是训练无需使用任何真实分割掩膜的分割模型。我们的工作解决了一个只有图像级别分类标签（表示兴趣区域的存在或缺失，如肿瘤或病变）的情况。大多数现有方法依赖于类激活映射（CAM）。我们提出了一种新方法ToNNO，它是基于神经网络输出Tomographic重构的。我们的技术从输入3D体积中提取不同角度的切片，将这些切片输入2D编码器，并应用逆Radon变换来重构编码器的预测的3D热图。这种通用方法允许使用任何2D图像编码器对3D体积进行密集预测。我们将它应用于弱监督医疗图像分割，通过训练2D编码器为包含感兴趣区域的切片提供高值。我们在四个大型医疗图像数据集上测试它，并优于2D CAM方法。然后，我们将ToNNO扩展，通过结合断层扫描和CAM方法，提出平均CAM和断层扫描CAM，获得更好的结果。
URL
https://arxiv.org/abs/2404.13103
