Abstract
Image stitching from different captures often results in non-rectangular boundaries, which is often considered unappealing. To solve non-rectangular boundaries, current solutions involve cropping, which discards image content, inpainting, which can introduce unrelated content, or warping, which can distort non-linear features and introduce artifacts. To overcome these issues, we introduce a novel diffusion-based learning framework, \textbf{RecDiffusion}, for image stitching rectangling. This framework combines Motion Diffusion Models (MDM) to generate motion fields, effectively transitioning from the stitched image's irregular borders to a geometrically corrected intermediary. Followed by Content Diffusion Models (CDM) for image detail refinement. Notably, our sampling process utilizes a weighted map to identify regions needing correction during each iteration of CDM. Our RecDiffusion ensures geometric accuracy and overall visual appeal, surpassing all previous methods in both quantitative and qualitative measures when evaluated on public benchmarks. Code is released at this https URL.
Abstract (translated)
图像拼接从不同捕获通常会导致非矩形边界，这通常被认为是不吸引人的。为解决非矩形边界，目前的解决方案包括裁剪、修复和扭曲，这些方法都会放弃图像内容或引入无关内容，或扭曲，从而扭曲非线性特征并引入伪影。为了克服这些问题，我们引入了一个新的扩散为基础的学习框架， RecDiffusion，用于图像拼接和矩形化。该框架结合了运动扩散模型 (MDM) 来生成运动场，有效地将拼接图像的不规则边界转换为几何校正的中间结果。然后是内容扩散模型 (CDM) 来修复图像细节。值得注意的是，我们的采样过程利用加权图在每次迭代 CDM 时确定需要修复的区域。我们的 RecDiffusion 确保几何准确性和整体视觉吸引力，在评估公共基准时超越了所有以前方法。代码发布在 这个链接上：
URL
https://arxiv.org/abs/2403.19164
