Abstract
Local feature matching aims at establishing sparse correspondences between a pair of images. Recently, detectorfree methods present generally better performance but are not satisfactory in image pairs with large scale differences. In this paper, we propose Patch Area Transportation with Subdivision (PATS) to tackle this issue. Instead of building an expensive image pyramid, we start by splitting the original image pair into equal-sized patches and gradually resizing and subdividing them into smaller patches with the same scale. However, estimating scale differences between these patches is non-trivial since the scale differences are determined by both relative camera poses and scene structures, and thus spatially varying over image pairs. Moreover, it is hard to obtain the ground truth for real scenes. To this end, we propose patch area transportation, which enables learning scale differences in a self-supervised manner. In contrast to bipartite graph matching, which only handles one-to-one matching, our patch area transportation can deal with many-to-many relationships. PATS improves both matching accuracy and coverage, and shows superior performance in downstream tasks, such as relative pose estimation, visual localization, and optical flow estimation. The source code will be released to benefit the community.
Abstract (translated)
局部特征匹配旨在建立两图像之间的稀疏对应关系。近年来,无检测算法方法通常表现更好,但在图像 pairs 中存在大规模差异时并不令人满意。在本文中,我们提出了 patch 区域传输(PATS)方法来解决这个问题。我们不再需要建造昂贵的图像金字塔,而是从原始图像对中切分成相等大小的区块,并逐渐缩小并分割成相同的小区块,每个区块的大小相同。但是,对这些区块之间尺度差异的估计是一项艰巨的任务,因为尺度差异是由相对相机姿态和场景结构决定的,因此它们在图像对中的空间变化。此外,对于真实的场景,很难获得初始 truth。因此,我们提出了 patch 区域传输方法,它可以通过自我监督的方式学习尺度差异。与只会处理一对一匹配的二分图匹配不同,我们的 patch 区域传输可以处理多对多的关系。PATS 改善了匹配精度和覆盖范围,并在后续任务,如相对姿态估计、视觉定位和光学流估计中表现出更好的性能。源代码将用于促进社区。
URL
https://arxiv.org/abs/2303.07700
