Abstract
We present Mono-STAR, the first real-time 3D reconstruction system that simultaneously supports semantic fusion, fast motion tracking, non-rigid object deformation, and topological change under a unified framework. The proposed system solves a new optimization problem incorporating optical-flow-based 2D constraints to deal with fast motion and a novel semantic-aware deformation graph (SAD-graph) for handling topology change. We test the proposed system under various challenging scenes and demonstrate that it significantly outperforms existing state-of-the-art methods.
Abstract (translated)
我们介绍了 Mono-STAR,它是第一个在统一框架下同时支持语义融合、快速运动追踪、非定长的对象的变形和拓扑变化实时三维重建系统。该系统通过引入基于光学流的2D约束来解决快速运动问题,并使用一种新的语义相关的变形图(SAD-图)来处理拓扑变化问题。在各种不同的挑战场景中,我们测试了该系统,并证明了它比现有的最先进的方法具有显著的优越性。
URL
https://arxiv.org/abs/2301.13244
