Abstract
We introduce the task of generative panoramic image stitching, which aims to synthesize seamless panoramas that are faithful to the content of multiple reference images containing parallax effects and strong variations in lighting, camera capture settings, or style. In this challenging setting, traditional image stitching pipelines fail, producing outputs with ghosting and other artifacts. While recent generative models are capable of outpainting content consistent with multiple reference images, they fail when tasked with synthesizing large, coherent regions of a panorama. To address these limitations, we propose a method that fine-tunes a diffusion-based inpainting model to preserve a scene's content and layout based on multiple reference images. Once fine-tuned, the model outpaints a full panorama from a single reference image, producing a seamless and visually coherent result that faithfully integrates content from all reference images. Our approach significantly outperforms baselines for this task in terms of image quality and the consistency of image structure and scene layout when evaluated on captured datasets.
Abstract (translated)
我们介绍了一项名为生成全景图像拼接的任务，其目标是合成无缝的全景图，这些全景图忠实于包含视差效果和强烈光照变化、相机拍摄设置或风格差异的多张参考图像的内容。在这一挑战性的场景中，传统的图像拼接流水线无法正常工作，会产生鬼影等其他伪影。虽然最近的一些生成模型能够根据多张参考图像扩展内容一致的新区域，但在合成大型且连贯的全景图区域时却会失败。为了克服这些限制，我们提出了一种方法，即微调基于扩散的修复模型，以依据多张参考图像来保留场景的内容和布局。一旦经过微调，该模型可以从单个参考图像中扩展出完整的全景图，并生成无缝且视觉连贯的结果，忠实整合所有参考图像的内容。我们的方法在捕获数据集上评估时，在图像质量以及图像结构和场景布局的一致性方面显著优于基准方法。
URL
https://arxiv.org/abs/2507.07133
