Abstract
Conventional training for optical flow and stereo depth models typically employs a uniform loss function across all pixels. However, this one-size-fits-all approach often overlooks the significant variations in learning difficulty among individual pixels and contextual regions. This paper investigates the uncertainty-based confidence maps which capture these spatially varying learning difficulties and introduces tailored solutions to address them. We first present the Difficulty Balancing (DB) loss, which utilizes an error-based confidence measure to encourage the network to focus more on challenging pixels and regions. Moreover, we identify that some difficult pixels and regions are affected by occlusions, resulting from the inherently ill-posed matching problem in the absence of real correspondences. To address this, we propose the Occlusion Avoiding (OA) loss, designed to guide the network into cycle consistency-based confident regions, where feature matching is more reliable. By combining the DB and OA losses, we effectively manage various types of challenging pixels and regions during training. Experiments on both optical flow and stereo depth tasks consistently demonstrate significant performance improvements when applying our proposed combination of the DB and OA losses.
Abstract (translated)
传统的光流和立体深度模型训练通常采用在整个图像的所有像素上统一应用的损失函数。然而，这种一刀切的方法往往忽视了各个像素及其上下文区域在学习难度上的显著差异。本文研究了一种基于不确定性的置信度图，该图捕捉到了这些空间变化的学习困难，并提出了定制化解决方案来应对这些问题。 首先，我们介绍了难度平衡（DB）损失函数，它使用一种基于误差的置信度测量方法，鼓励网络更加关注那些具有挑战性的像素和区域。此外，我们发现一些难以处理的像素和区域受到遮挡的影响，这是由于在缺乏真实对应关系的情况下匹配问题本质上是病态的。为了解决这个问题，我们提出了避免遮挡（OA）损失函数，它旨在引导网络进入基于循环一致性的可靠特征匹配区域。 通过结合DB和OA损失函数，在训练过程中有效地管理各种类型的具有挑战性的像素和区域。实验结果表明，在光流和立体深度任务上应用我们提出的DB和OA损失组合方法时，性能有显著提升。
URL
https://arxiv.org/abs/2506.00324
