Abstract
Tissue deformation recovery based on stereo endoscopic images is crucial for tool-tissue interaction analysis and benefits surgical navigation and autonomous soft tissue manipulation. Previous research suffers from the problems raised from camera motion, occlusion, large tissue deformation, lack of tissue-specific biomechanical priors, and reliance on offline processing. Unlike previous studies where the tissue geometry and deformation are represented by 3D points and displacements, the proposed method models tissue geometry as the 3D point and derivative map and tissue deformation as the 3D displacement and local deformation map. For a single surface point, 6 parameters are used to describe its rigid motion and 3 parameters for its local deformation. The method is formulated under the camera-centric setting, where all motions are regarded as the scene motion with respect to the camera. Inter-frame alignment is realized by optimizing the inter-frame deformation, making it unnecessary to estimate camera pose. The concept of the canonical map is introduced to optimize tissue geometry and deformation in an online approach. Quantitative and qualitative experiments were conducted using in vivo and ex vivo laparoscopic datasets. With the inputs of depth and optical flow, the method stably models tissue geometry and deformation even when the tissue is partially occluded or moving outside the field of view. Results show that the 3D reconstruction accuracy in the non-occluded and occluded areas reaches 0.37$\pm$0.27 mm and 0.39$\pm$0.21 mm in terms of surface distance, respectively. The method can also estimate surface strain distribution during various manipulations as an extra modality for mechanical-based analysis.
Abstract (translated)
基于立体内窥镜图像的组织变形恢复对于工具-组织交互分析至关重要，能够改善手术导航和自主软组织操作。然而，以往的研究面临摄像机运动、遮挡、组织大范围变形、缺乏特定生物力学先验知识以及依赖于离线处理等问题。不同于之前研究中用3D点和位移来表示组织几何形状和变形的方式，所提出的方法将组织的几何结构建模为3D点及其导数图，并且将组织的变形建模为3D位移及局部变形图。对于单个表面点，使用6个参数描述其刚性运动，并用3个参数描述其局部变形。该方法以摄像机为中心设置进行构建，在此框架下所有运动都被视为相对于摄像机的场景运动。通过优化帧间变形实现帧间对齐，从而无需估计摄像机姿态。引入“标准映射”的概念来在线优化组织几何形状和变形。 使用体内和体外腹腔镜数据集进行了定量和定性实验，在输入深度信息和光流的情况下，即使在部分遮挡或运动超出视野范围时，该方法也能稳定地建模组织的几何形状和变形。结果显示，在未被遮挡和被遮挡区域中的3D重建准确度分别为0.37±0.27毫米和0.39±0.21毫米（以表面距离为衡量标准）。此外，该方法还可以估算在各种操作期间的表面应变分布，作为基于力学分析的一个额外模态。
URL
https://arxiv.org/abs/2506.19388
