Abstract
Many real-world computer vision tasks, such as depth completion, must handle inputs with arbitrarily shaped regions of missing or invalid data. For Convolutional Neural Networks (CNNs), Partial Convolutions solved this by a mask-aware re-normalization conditioned only on valid pixels. Recently, State Space Models (SSMs) like Mamba have emerged, offering high performance with linear complexity. However, these architectures lack an inherent mechanism for handling such arbitrarily shaped invalid data at inference time. To bridge this gap, we introduce Partial Vision Mamba (PVM), a novel architectural component that ports the principles of partial operations to the Mamba backbone. We also define a series of rules to design architectures using PVM. We show the efficacy and generalizability of our approach in the tasks of depth completion, image inpainting, and classification with invalid data.
Abstract (translated)
许多现实世界的计算机视觉任务，如深度完成（depth completion），需要处理具有任意形状的缺失或无效数据区域的输入。对于卷积神经网络（CNN）而言，部分卷积通过使用仅基于有效像素进行掩码感知重归一化解决了这一问题。最近，像Mamba这样的状态空间模型(SSMs)已经出现，提供了高效率和线性复杂度的性能表现。然而，这些架构在推理时缺乏处理此类任意形状无效数据的内在机制。 为弥补这一不足，我们引入了部分视觉曼巴（Partial Vision Mamba, PVM），这是一个将部分操作原则移植到Mamba骨干网络上的新型架构组件。同时，我们也定义了一系列规则来使用PVM设计架构。我们在深度完成、图像修复以及带有无效数据的分类任务中展示了我们的方法的有效性和泛化能力。
URL
https://arxiv.org/abs/2603.04568
