Abstract
Depth estimation is crucial for intelligent systems, enabling applications from autonomous navigation to augmented reality. While traditional stereo and active depth sensors have limitations in cost, power, and robustness, dual-pixel (DP) technology, ubiquitous in modern cameras, offers a compelling alternative. This paper introduces DiFuse-Net, a novel modality decoupled network design for disentangled RGB and DP based depth estimation. DiFuse-Net features a window bi-directional parallax attention mechanism (WBiPAM) specifically designed to capture the subtle DP disparity cues unique to smartphone cameras with small aperture. A separate encoder extracts contextual information from the RGB image, and these features are fused to enhance depth prediction. We also propose a Cross-modal Transfer Learning (CmTL) mechanism to utilize large-scale RGB-D datasets in the literature to cope with the limitations of obtaining large-scale RGB-DP-D dataset. Our evaluation and comparison of the proposed method demonstrates its superiority over the DP and stereo-based baseline methods. Additionally, we contribute a new, high-quality, real-world RGB-DP-D training dataset, named Dual-Camera Dual-Pixel (DCDP) dataset, created using our novel symmetric stereo camera hardware setup, stereo calibration and rectification protocol, and AI stereo disparity estimation method.
Abstract (translated)
深度估计对于智能系统至关重要，可支持从自主导航到增强现实等多种应用。虽然传统的立体视觉和主动测距传感器在成本、功耗和鲁棒性方面存在局限性，但如今广泛应用于现代相机的双像素（DP）技术提供了一种颇具吸引力的替代方案。本文介绍了DiFuse-Net，这是一种新型的模态解耦网络设计，用于基于RGB和DP的深度估计。DiFuse-Net具备一种专门针对手机摄像头的小孔径特性而设计的窗口双向视差注意力机制（WBiPAM），能够捕捉双像素特有的细微视差线索。该网络还包括一个单独的编码器从RGB图像中提取上下文信息，并将这些特征融合以增强深度预测。我们还提出了一种跨模态迁移学习（CmTL）机制，利用文献中的大规模RGB-D数据集来克服获取大规模RGB-DP-D数据集的局限性。我们的评估和对比试验表明，所提方法优于基于DP和立体视觉的基线方法。此外，我们贡献了一个新的高质量的真实世界RGB-DP-D训练数据集——双摄像头双像素（DCDP）数据集，该数据集使用我们创新性的对称立体相机硬件设置、立体校准和直方图矫正协议以及AI立体视差估计方法创建而成。
URL
https://arxiv.org/abs/2506.14709
