Abstract
Photometric constraint is indispensable for self-supervised monocular depth estimation. It involves warping a source image onto a target view using estimated depth&pose, and then minimizing the difference between the warped and target images. However, the endoscopic built-in light causes significant brightness fluctuations, and thus makes the photometric constraint unreliable. Previous efforts only mitigate this relying on extra models to calibrate image brightness. In this paper, we propose MonoPCC to address the brightness inconsistency radically by reshaping the photometric constraint into a cycle form. Instead of only warping the source image, MonoPCC constructs a closed loop consisting of two opposite forward-backward warping paths: from target to source and then back to target. Thus, the target image finally receives an image cycle-warped from itself, which naturally makes the constraint invariant to brightness changes. Moreover, MonoPCC transplants the source image's phase-frequency into the intermediate warped image to avoid structure lost, and also stabilizes the training via an exponential moving average (EMA) strategy to avoid frequent changes in the forward warping. The comprehensive and extensive experimental results on three datasets demonstrate that our proposed MonoPCC shows a great robustness to the brightness inconsistency, and exceeds other state-of-the-arts by reducing the absolute relative error by at least 7.27%.
Abstract (translated)
光度约束对于自监督单目深度估计是不可或缺的。它涉及将估计深度&姿态的源图像扭曲到目标视图，然后最小化扭曲后的源图像和目标图像之间的差异。然而，内窥镜内置的光导致显著的亮度波动，因此使得光度约束不可靠。之前的努力仅通过额外模型校准图像亮度来缓解这种依赖。在本文中，我们提出MonoPCC来通过将光度约束变换为环形形式来解决亮度不一致问题。 instead of only warping the source image, MonoPCC构建了一个由两个相反的前向-反向扭曲路径组成的闭环：从目标到源，然后回到目标。因此，目标图像最终从自身获得了一个周期扭曲，这自然使得约束对亮度变化不变。此外，MonoPCC通过指数移动平均（EMA）策略将源图像的相频移植到中间扭曲图像中，以避免结构丢失，并通过EMA策略稳定训练，以避免经常改变前向扭曲。三个数据集上的全面和广泛的实验结果表明，我们提出的MonoPCC在亮度不一致性方面具有很大的稳健性，超过了其他现有技术的水平，至少减少了7.27%的绝对相对误差。
URL
https://arxiv.org/abs/2404.16571
