Abstract
Continuum robots are advancing bronchoscopy procedures by accessing complex lung airways and enabling targeted interventions. However, their development is limited by the lack of realistic training and test environments: Real data is difficult to collect due to ethical constraints and patient safety concerns, and developing autonomy algorithms requires realistic imaging and physical feedback. We present ROOM (Realistic Optical Observation in Medicine), a comprehensive simulation framework designed for generating photorealistic bronchoscopy training data. By leveraging patient CT scans, our pipeline renders multi-modal sensor data including RGB images with realistic noise and light specularities, metric depth maps, surface normals, optical flow and point clouds at medically relevant scales. We validate the data generated by ROOM in two canonical tasks for medical robotics -- multi-view pose estimation and monocular depth estimation, demonstrating diverse challenges that state-of-the-art methods must overcome to transfer to these medical settings. Furthermore, we show that the data produced by ROOM can be used to fine-tune existing depth estimation models to overcome these challenges, also enabling other downstream applications such as navigation. We expect that ROOM will enable large-scale data generation across diverse patient anatomies and procedural scenarios that are challenging to capture in clinical settings. Code and data: this https URL.
Abstract (translated)
连续机器人通过访问复杂的肺部气道并实现靶向干预，正在推进支气管镜检查程序。然而，其发展受限于缺乏现实的训练和测试环境：由于伦理限制和患者安全问题，收集真实数据较为困难；开发自主算法需要真实的成像及物理反馈。我们提出了ROOM（医学中的逼真光学观察），这是一个旨在生成逼真的支气管镜培训数据的全面模拟框架。通过利用患者的CT扫描，我们的流水线能够渲染包括带现实噪声和光斑点的RGB图像、度量深度图、表面法向量、光流及点云在内的多模态传感器数据，在医学相关的尺度下进行呈现。 我们用ROOM生成的数据在两个典型的医疗机器人任务——多视角姿态估计和单目深度估计中进行了验证，展示了最先进的方法必须克服的各种挑战才能转移到这些医疗环境中。此外，我们还表明，由ROOM产生的数据可以用于微调现有的深度估算模型以应对这些挑战，并促进其他下游应用如导航的实现。 我们预计，ROOM将能够生成大规模的数据，覆盖各种患者解剖结构和程序场景，在临床环境中难以捕捉。代码与数据：[此处提供具体链接]（请根据实际情况插入具体链接）。
URL
https://arxiv.org/abs/2509.13177
