Abstract
Soft tissue simulation in virtual environments is becoming increasingly important for medical applications. However, the high deformability of soft tissue poses significant challenges. Existing methods rely on segmentation, meshing and estimation of stiffness properties of tissues. In addition, the integration of haptic feedback requires precise force estimation to enable a more immersive experience. We introduce a novel data-driven model, a conditional graph neural network (cGNN) to tackle this complexity. Our model takes surface points and the location of applied forces, and is specifically designed to predict the deformation of the points and the forces exerted on them. We trained our model on experimentally collected surface tracking data of a soft tissue phantom and used transfer learning to overcome the data scarcity by initially training it with mass-spring simulations and fine-tuning it with the experimental data. This approach improves the generalisation capability of the model and enables accurate predictions of tissue deformations and corresponding interaction forces. The results demonstrate that the model can predict deformations with a distance error of 0.35$\pm$0.03 mm for deformations up to 30 mm and the force with an absolute error of 0.37$\pm$0.05 N for forces up to 7.5 N. Our data-driven approach presents a promising solution to the intricate challenge of simulating soft tissues within virtual environments. Beyond its applicability in medical simulations, this approach holds the potential to benefit various fields where realistic soft tissue simulations are required.
Abstract (translated)
在虚拟环境中进行软组织模拟对于医学应用变得越来越重要。然而，由于软组织的高度可变形性，这对现有技术提出了重大挑战。目前的方法依赖于分割、网格划分和组织硬度属性的估算。此外，为了实现更加沉浸式的体验，触觉反馈的集成需要精确的力量估计。我们引入了一种基于数据驱动的新模型——条件图神经网络（cGNN），以解决这一复杂性问题。我们的模型接受表面点及其受力位置作为输入，并专门设计用于预测这些点的变形及作用在它们上的力量。 我们在软组织仿体实验收集到的表面跟踪数据上训练了该模型，并通过转移学习克服数据稀缺的问题，即先使用质量-弹簧仿真进行初步训练，然后用实验数据进行微调。这种方法提升了模型的泛化能力，使其能够准确预测组织变形及其相互作用力。实验证明，在30毫米范围内的变形距离误差为0.35±0.03毫米，在7.5牛顿范围内的力量绝对误差为0.37±0.05牛顿。 我们基于数据驱动的方法为在虚拟环境中模拟软组织提供了一个有前景的解决方案。除了医学仿真领域的应用，这种方法还具有潜在的应用价值，适用于任何需要现实软组织仿真的领域。
URL
https://arxiv.org/abs/2507.05315
