Abstract
Stereo vision between images faces a range of challenges, including occlusions, motion, and camera distortions, across applications in autonomous driving, robotics, and face analysis. Due to parameter sensitivity, further complications arise for stereo matching with sparse features, such as facial landmarks. To overcome this ill-posedness and enable unsupervised sparse matching, we consider line constraints of the camera geometry from an optimal transport (OT) viewpoint. Formulating camera-projected points as (half)lines, we propose the use of the classical epipolar distance as well as a 3D ray distance to quantify matching quality. Employing these distances as a cost function of a (partial) OT problem, we arrive at efficiently solvable assignment problems. Moreover, we extend our approach to unsupervised object matching by formulating it as a hierarchical OT problem. The resulting algorithms allow for efficient feature and object matching, as demonstrated in our numerical experiments. Here, we focus on applications in facial analysis, where we aim to match distinct landmarking conventions.
Abstract (translated)
图像间的立体视觉面临多种挑战，包括遮挡、运动和相机畸变等问题，在自主驾驶、机器人技术和面部分析等应用中尤为突出。特别是在稀疏特征（如面部标志）的立体匹配任务中，由于参数敏感性而引发更多复杂情况。为了解决这一不适定问题，并实现无监督下的稀疏匹配，我们从最优传输(OT)的角度出发考虑相机几何学中的线约束条件。我们将摄像机投影点表示为（半）线，并提出使用经典的极线距离以及三维射线距离来量化匹配质量。将这些距离作为（部分）OT问题的成本函数，从而转化成可以高效解决的分配问题。此外，我们通过将其表述为分层最优传输(OT)问题的形式，扩展了我们的方法至无监督的对象匹配任务中。在数值实验中展示的结果表明，该算法允许有效地进行特征和对象的匹配工作。特别地，在面部分析的应用场景下，我们的目标是将不同的地标标记规范进行匹配。
URL
https://arxiv.org/abs/2601.12423
