Abstract
In this paper, we develop a modified differential Structure from Motion (SfM) algorithm that can estimate relative pose from two consecutive frames despite of Rolling Shutter (RS) artifacts. In particular, we show that under constant velocity assumption, the errors induced by the rolling shutter effect can be easily rectified by a linear scaling operation on each optical flow. We further propose a 9-point algorithm to recover the relative pose of a rolling shutter camera that undergoes constant acceleration motion. We demonstrate that the dense depth maps recovered from the relative pose of the RS camera can be used in a RS-aware warping for image rectification to recover high-quality Global Shutter (GS) images. Experiments on both synthetic and real RS images show that our RS-aware differential SfM algorithm produces more accurate results on relative pose estimation and 3D reconstruction from images distorted by RS effect compared to standard SfM algorithms that assume a GS camera model. We also demonstrate that our RS-aware warping for image rectification method outperforms state-of-the-art commercial software products, i.e. Adobe After Effects and Apple Imovie, at removing RS artifacts.
Abstract (translated)
在本文中，我们开发了一种改进的运动微分结构（SFM）算法，该算法可以从两个连续帧中估计相对姿态，尽管存在卷帘门（RS）伪影。特别地，我们证明了在等速假设下，通过对每一光流进行线性缩放操作，可以很容易地纠正卷帘效应引起的误差。我们进一步提出了一个9点算法来恢复卷帘相机在恒加速度运动下的相对姿态。我们证明，从遥感相机的相对姿态恢复的密集深度图可以用于遥感感知的图像矫正，以恢复高质量的全局快门（GS）图像。对合成图像和真实图像的实验表明，与采用GS相机模型的标准SFM算法相比，我们的RS感知差分SFM算法在相对姿态估计和基于RS效应失真图像的三维重建方面产生了更精确的结果。我们还证明，我们的RS感知图像矫正扭曲方法在去除RS伪影方面优于最先进的商业软件产品，即Adobe After Effects和Apple iMovie。
URL
https://arxiv.org/abs/1903.03943
