Abstract
This work presents PanMatch, a versatile foundation model for robust correspondence matching. Unlike previous methods that rely on task-specific architectures and domain-specific fine-tuning to support tasks like stereo matching, optical flow or feature matching, our key insight is that any two-frame correspondence matching task can be addressed within a 2D displacement estimation framework using the same model weights. Such a formulation eliminates the need for designing specialized unified architectures or task-specific ensemble models. Instead, it achieves multi-task integration by endowing displacement estimation algorithms with unprecedented generalization capabilities. To this end, we highlight the importance of a robust feature extractor applicable across multiple domains and tasks, and propose the feature transformation pipeline that leverage all-purpose features from Large Vision Models to endow matching baselines with zero-shot cross-view matching capabilities. Furthermore, we assemble a cross-domain dataset with near 1.8 million samples from stereo matching, optical flow, and feature matching domains to pretrain PanMatch. We demonstrate the versatility of PanMatch across a wide range of domains and downstream tasks using the same model weights. Our model outperforms UniMatch and Flow-Anything on cross-task evaluations, and achieves comparable performance to most state-of-the-art task-specific algorithms on task-oriented benchmarks. Additionally, PanMatch presents unprecedented zero-shot performance in abnormal scenarios, such as rainy day and satellite imagery, where most existing robust algorithms fail to yield meaningful results.
Abstract (translated)
这项工作提出了PanMatch，这是一种灵活的基础模型，用于稳健的对应匹配。与之前依赖于特定任务架构和领域特定微调的方法（例如立体匹配、光流或特征匹配）不同，我们的关键见解是任何两帧对应的匹配任务都可以在2D位移估计框架内通过相同的模型权重解决。这种形式化消除了设计专门统一架构或特定任务集成模型的需求，并通过赋予位移估计算法前所未有的泛化能力来实现多任务整合。 为此，我们强调了跨多个领域和任务的稳健特征提取器的重要性，并提出了一个特征转换管道，该管道利用大型视觉模型中的通用功能，使匹配基线具备零样本跨视图匹配的能力。此外，我们还组装了一个包含近180万样本的跨域数据集，这些样本来自立体匹配、光流和特征匹配领域，用于预训练PanMatch。 我们在广泛的领域和下游任务中展示了使用相同模型权重时PanMatch的多功能性。我们的模型在跨任务评估中超越了UniMatch和Flow-Anything，并在面向特定任务的基准测试中达到了与大多数最先进的特定任务算法相当的表现水平。此外，在异常场景（如雨天或卫星图像）下，PanMatch展示出了前所未有的零样本性能，而大多数现有的稳健算法在这种情况下通常无法提供有意义的结果。
URL
https://arxiv.org/abs/2507.08400
