Abstract
A 3D understanding of anatomy is central to diagnosis and treatment planning, yet volumetric imaging remains costly with long wait times. Image-to-3D foundations models can solve this issue by reconstructing 3D data from 2D modalites. Current foundation models are trained on natural image distributions to reconstruct naturalistic objects from a single image by leveraging geometric priors across pixels. However, it is unclear whether these learned geometric priors transfer to medical data. In this study, we present a controlled zero-shot benchmark of single slice medical image-to-3D reconstruction across five state-of-the-art image-to-3D models: SAM3D, Hunyuan3D-2.1, Direct3D, Hi3DGen, and TripoSG. These are evaluated across six medical datasets spanning anatomical and pathological structures and two natrual datasets, using voxel based metrics and point cloud distance metrics. Across medical datasets, voxel based overlap remains moderate for all models, consistent with a depth reconstruction failure mode when inferring volume from a single slice. In contrast, global distance metrics show more separation between methods: SAM3D achieves the strongest overall topological similarity to ground truth medical 3D data, while alternative models are more prone to over-simplication of reconstruction. Our results quantify the limits of single-slice medical reconstruction and highlight depth ambiguity caused by the planar nature of 2D medical data, motivating multi-view image-to-3D reconstruction to enable reliable medical 3D inference.
Abstract (translated)
对解剖学的三维理解是诊断和治疗计划的核心，然而体积成像仍然成本高昂且等待时间长。图像到三维基础模型可以通过从二维模式重建三维数据来解决这一问题。当前的基础模型是在自然图像分布上训练的，能够利用像素间的几何先验从单张图像中重构出现实主义对象。然而，这些学习到的几何先验是否能转移到医学数据上尚不清楚。在本研究中，我们提出了一项针对五个最先进的图像到三维重建模型（SAM3D、Hunyuan3D-2.1、Direct3D、Hi3DGen和TripoSG）的单切片医疗图像到三维重建控制零样本基准测试。这些模型是在六个医学数据集上进行评估，涵盖了解剖学和病理结构以及两个自然数据集，并使用基于体素的度量标准和点云距离度量进行了评测。 在医学数据集中，所有模型的基于体素的重叠保持中等水平，在从单片图像推断体积时显示出深度重建失败模式的一致性。相比之下，全局距离指标显示出了方法之间的更大差异：SAM3D实现了与地面真实三维医学数据最接近的整体拓扑相似度，而其他模型更容易出现过度简化重构的问题。 我们的结果量化了单一切片医疗重建的局限，并突显了由二维医疗数据平面性质引起的深度歧义问题，从而推动多视角图像到三维重建以实现可靠的医学三维推理。
URL
https://arxiv.org/abs/2602.09407
