Abstract
Multiview stereo aims to reconstruct scene depth from images acquired by a camera under arbitrary motion. Recent methods address this problem through deep learning, which can utilize semantic cues to deal with challenges such as textureless and reflective regions. In this paper, we present a convolutional neural network called DPSNet (Deep Plane Sweep Network) whose design is inspired by best practices of traditional geometry-based approaches for dense depth reconstruction. Rather than directly estimating depth and/or optical flow correspondence from image pairs as done in many previous deep learning methods, DPSNet takes a plane sweep approach that involves building a cost volume from deep features using the plane sweep algorithm, regularizing the cost volume via a context-aware cost aggregation, and regressing the dense depth map from the cost volume. The cost volume is constructed using a differentiable warping process that allows for end-to-end training of the network. Through the effective incorporation of conventional multiview stereo concepts within a deep learning framework, DPSNet achieves state-of-the-art reconstruction results on a variety of challenging datasets.
Abstract (translated)
多视点立体成像的目的是从相机在任意运动下获取的图像中重建场景的深度。最近的方法通过深度学习来解决这个问题，它可以利用语义提示来处理诸如无纹理和反射区域等挑战。本文提出了一种卷积神经网络，称为深平面扫描网络（DPSNET），其设计灵感来自于传统的基于几何的密集深度重建方法的最佳实践。DPSNET不是像许多以前的深度学习方法那样直接估计图像对的深度和/或光流对应关系，而是采用平面扫描方法，该方法涉及使用平面扫描算法从深度特征构建成本量，通过上下文感知的成本聚集调整成本量，并对根据成本量绘制的密集深度图。成本量是通过一个允许网络端到端培训的可微分翘曲过程来构建的。通过在深度学习框架内有效地融合传统的多视图立体概念，DPSNET在各种具有挑战性的数据集上实现了最先进的重建结果。
URL
https://arxiv.org/abs/1905.00538
