Abstract
Mesh reconstruction of the cardiac anatomy from medical images is useful for shape and motion measurements and biophysics simulations to facilitate the assessment of cardiac function and health. However, 3D medical images are often acquired as 2D slices that are sparsely sampled and noisy, and mesh reconstruction on such data is a challenging task. Traditional voxel-based approaches rely on pre- and post-processing that compromises image fidelity, while mesh-level deep learning approaches require mesh annotations that are difficult to get. Therefore, direct cross-domain supervision from 2D images to meshes is a key technique for advancing 3D learning in medical imaging, but it has not been well-developed. While there have been attempts to approximate the optimized meshes' slicing, few existing methods directly use 2D slices to supervise mesh reconstruction in a differentiable manner. Here, we propose a novel explicit differentiable voxelization and slicing (DVS) algorithm that allows gradient backpropagation to a mesh from its slices, facilitating refined mesh optimization directly supervised by the losses defined on 2D images. Further, we propose an innovative framework for extracting patient-specific left ventricle (LV) meshes from medical images by coupling DVS with a graph harmonic deformation (GHD) mesh morphing descriptor of cardiac shape that naturally preserves mesh quality and smoothness during optimization. Experimental results demonstrate that our method achieves state-of-the-art performance in cardiac mesh reconstruction tasks from CT and MRI, with an overall Dice score of 90% on multi-datasets, outperforming existing approaches. The proposed method can further quantify clinically useful parameters such as ejection fraction and global myocardial strains, closely matching the ground truth and surpassing the traditional voxel-based approach in sparse images.
Abstract (translated)
从医学图像中进行网格重建心脏解剖结构有助于形状和运动测量以及生物物理学模拟,以促进评估心脏功能和健康。然而,3D医疗图像通常以稀疏采样和噪音噪声的方式获取,因此在这样的数据上进行网格重建是一个具有挑战性的任务。传统的基于体素的方法依赖于前处理和后处理,这会降低图像的保真度。而网格级别深度学习方法需要网格注释,这些注释很难获得。因此,直接从2D图像到网格的跨域监督是推动医学影像中3D学习的重要技术,但尚未得到很好的发展。虽然已经尝试过用近似优化网格的切片来估计最优网格,但很少有现有方法直接使用2D切片进行不同维度的网格重建。在这里,我们提出了一种新颖的显式可导体积和切片(DVS)算法,可以从其切片进行网格的网格级优化,并通过在DVS中结合心脏形状的神经网络网格变形描述符来自然保留网格质量和平滑度。进一步,我们提出了一个创新的方法,通过将DVS与心脏形状的神经网络网格变形描述符耦合,从医学图像中提取患者特定的左心室(LV)网格。该方法将DVS与GHD网格变形描述符相结合,可以自然保留网格质量和平滑度,并在优化过程中实现更好的网格质量。实验结果表明,我们的方法在CT和MRI中的心脏网格重建任务中达到了最先进的性能,多数据集的Dice分数为90%,超越了现有方法。所提出的方法还可以进一步量化临床有用的参数,如射血分数和全局心肌应变,与真实值非常接近,并在稀疏图像中超越了传统基于体素的方法。
URL
https://arxiv.org/abs/2409.02070
