Abstract
Image alignment is a fundamental task in computer vision with broad applications. Existing methods predominantly employ optical flow-based image warping. However, this technique is susceptible to common challenges such as occlusions and illumination variations, leading to degraded alignment visual quality and compromised accuracy in downstream tasks. In this paper, we present DMAligner, a diffusion-based framework for image alignment through alignment-oriented view synthesis. DMAligner is crafted to tackle the challenges in image alignment from a new perspective, employing a generation-based solution that showcases strong capabilities and avoids the problems associated with flow-based image warping. Specifically, we propose a Dynamics-aware Diffusion Training approach for learning conditional image generation, synthesizing a novel view for image alignment. This incorporates a Dynamics-aware Mask Producing (DMP) module to adaptively distinguish dynamic foreground regions from static backgrounds, enabling the diffusion model to more effectively handle challenges that classical methods struggle to solve. Furthermore, we develop the Dynamic Scene Image Alignment (DSIA) dataset using Blender, which includes 1,033 indoor and outdoor scenes with over 30K image pairs tailored for image alignment. Extensive experimental results demonstrate the superiority of the proposed approach on DSIA benchmarks, as well as on a series of widely-used video datasets for qualitative comparisons. Our code is available at this https URL.
Abstract (translated)
图像对齐是计算机视觉中的基本任务，具有广泛的应用。现有方法主要采用基于光学流的图像扭曲技术。然而，这种方法容易受到遮挡和光照变化等常见问题的影响，导致对齐视觉质量下降，并且在下游任务中准确性降低。本文介绍了DMAligner，这是一种通过面向对齐视角合成的扩散框架进行图像对齐的方法。DMAligner 旨在从一个新的角度解决图像对齐中的挑战，采用基于生成的方法，展示出强大的能力并避免了与流基图像扭曲相关的问题。 具体而言，我们提出了一个感知动态性的扩散训练方法，用于学习条件下的图像生成，为图像对齐合成新视角。这包括了一个感知动态性掩码产生的（DMP）模块，能够自适应地区分动态前景区域和静态背景，使扩散模型更有效地处理传统方法难以解决的挑战。 此外，我们使用Blender开发了Dynamic Scene Image Alignment (DSIA) 数据集，其中包含1,033个室内和室外场景以及超过30K张图像对，专为图像对齐设计。广泛的实验结果展示了该方法在DSIA基准测试中的优越性，并且在一系列广泛使用的视频数据集中进行定性比较时也表现良好。 我们的代码可在[此链接](https://这个URL需要替换为你提供的具体URL)获取。
URL
https://arxiv.org/abs/2602.23022
