Abstract
Fisheye image rectification has a long-term unresolved issue with synthetic-to-real generalization. In most previous works, the model trained on the synthetic images obtains unsatisfactory performance on the real-world fisheye image. To this end, we propose a Dual Diffusion Architecture (DDA) for the fisheye rectification with a better generalization ability. The proposed DDA is simultaneously trained with paired synthetic fisheye images and unlabeled real fisheye images. By gradually introducing noises, the synthetic and real fisheye images can eventually develop into a consistent noise distribution, improving the generalization and achieving unlabeled real fisheye correction. The original image serves as the prior guidance in existing DDPMs (Denoising Diffusion Probabilistic Models). However, the non-negligible indeterminate relationship between the prior condition and the target affects the generation performance. Especially in the rectification task, the radial distortion can cause significant artifacts. Therefore, we provide an unsupervised one-pass network that produces a plausible new condition to strengthen guidance. This network can be regarded as an alternate scheme for fast producing reliable results without iterative inference. Compared with the state-of-the-art methods, our approach can reach superior performance in both synthetic and real fisheye image corrections.
Abstract (translated)
眼镜型图像纠正存在着与合成到真实的泛化问题。在大多数先前研究中,训练在合成图像上的模型在真实世界眼镜型图像上的表现并不理想。为此,我们提出了一种具有更好泛化能力的双扩散结构(DDA),以进行眼镜型图像纠正。 proposed DDA 同时训练着配对的的合成眼镜型和未标记的真实眼镜型图像。通过逐渐引入噪声,合成和真实眼镜型图像最终发展出了一致的噪声分布,提高了泛化能力,并实现了未标记的真实眼镜型修正。原始图像在现有的DDPMs(去噪扩散概率模型)中用作先前指导。然而,先前条件与目标之间存在的非确定关系会影响生成性能。特别是在图像纠正任务中,Radial 失真可能会产生显著的噪声痕迹。因此,我们提供了一种 unsupervised 的一阶网络,以产生一个合理的新条件来加强指导。这个网络可以被视为一种无需迭代推理的快速可靠的结果生成备选方案。与当前的方法相比,我们的 approach 在合成和真实眼镜型图像纠正中能够取得更好的性能。
URL
https://arxiv.org/abs/2301.11785
