Abstract
We address the challenging problem of dense dynamic scene reconstruction and camera pose estimation from multiple freely moving cameras -- a setting that arises naturally when multiple observers capture a shared event. Prior approaches either handle only single-camera input or require rigidly mounted, pre-calibrated camera rigs, limiting their practical applicability. We propose a two-stage optimization framework that decouples the task into robust camera tracking and dense depth refinement. In the first stage, we extend single-camera visual SLAM to the multi-camera setting by constructing a spatiotemporal connection graph that exploits both intra-camera temporal continuity and inter-camera spatial overlap, enabling consistent scale and robust tracking. To ensure robustness under limited overlap, we introduce a wide-baseline initialization strategy using feed-forward reconstruction models. In the second stage, we refine depth and camera poses by optimizing dense inter- and intra-camera consistency using wide-baseline optical flow. Additionally, we introduce MultiCamRobolab, a new real-world dataset with ground-truth poses from a motion capture system. Finally, we demonstrate that our method significantly outperforms state-of-the-art feed-forward models on both synthetic and real-world benchmarks, while requiring less memory.
Abstract (translated)
我们解决了从多个自由移动的摄像机重建密集动态场景和估计相机姿态这一难题——这种情况自然地出现在多名观察者捕捉共享事件时。之前的解决方案要么只能处理单一摄像头输入，要么需要固定安装且预先校准好的多摄系统，这限制了它们的实际应用性。为此，我们提出了一种两阶段优化框架，将任务分解为稳健的摄像机跟踪和密集深度细化两个部分。 在第一阶段，我们将单摄视觉SLAM（同步定位与地图构建）扩展到多摄设置中，通过构建一个时空连接图来利用每个摄像头内的时间连续性和跨摄像头的空间重叠，从而使尺度一致且跟踪更加稳定。为了确保在有限重叠的情况下也能保持稳健性，我们引入了一种使用前馈重建模型的宽基线初始化策略。 第二阶段，我们通过对多摄和单摄内部的一致性进行优化来细化深度信息和摄像机姿态，具体而言就是通过宽基线光学流来实现密集的跨摄一致性优化。此外，为了验证我们的方法有效性，我们还引入了MultiCamRobolab，这是一个新的真实世界数据集，包含来自动作捕捉系统的地面真值位置。 最后，我们在合成和实际世界的基准测试中展示了我们的方法显著优于最新的前馈模型，并且在内存需求上也更为节省。
URL
https://arxiv.org/abs/2603.12064
