Abstract
In this paper, we revisit multimodal few-shot 3D point cloud semantic segmentation (FS-PCS), identifying a conflict in "Fuse-then-Refine" paradigms: the "Plasticity-Stability Dilemma." In addition, CLIP's inter-class confusion can result in semantic blindness. To address these issues, we present the Decoupled-experts Arbitration Few-Shot SegNet (DA-FSS), a model that effectively distinguishes between semantic and geometric paths and mutually regularizes their gradients to achieve better generalization. DA-FSS employs the same backbone and pre-trained text encoder as MM-FSS to generate text embeddings, which can increase free modalities' utilization rate and better leverage each modality's information space. To achieve this, we propose a Parallel Expert Refinement module to generate each modal correlation. We also propose a Stacked Arbitration Module (SAM) to perform convolutional fusion and arbitrate correlations for each modality pathway. The Parallel Experts decouple two paths: a Geometric Expert maintains plasticity, and a Semantic Expert ensures stability. They are coordinated via a Decoupled Alignment Module (DAM) that transfers knowledge without propagating confusion. Experiments on popular datasets (S3DIS, ScanNet) demonstrate the superiority of DA-FSS over MM-FSS. Meanwhile, geometric boundaries, completeness, and texture differentiation are all superior to the baseline. The code is available at: this https URL.
Abstract (translated)
在这篇论文中，我们重新审视了多模态少量样本的3D点云语义分割（FS-PCS），发现“融合再细化”范式存在一种矛盾：“可塑性与稳定性困境”。此外，CLIP 的类别间混淆可能导致语义盲区。为了解决这些问题，我们提出了解耦专家仲裁少量样本分割网络（DA-FSS），该模型能够有效区分语义和几何路径，并相互正则化其梯度以实现更好的泛化能力。DA-FSS 使用与MM-FSS相同的骨干网和预训练的文本编码器来生成文本嵌入，这可以增加自由模态的利用率并更好地利用每个模态的信息空间。 为此，我们提出了一个平行专家细化模块来生成每种模式的相关性，并提出堆叠仲裁模块（SAM）执行卷积融合并对每条路径的相关性进行仲裁。平行专家将两条路径解耦：几何专家保持可塑性，而语义专家确保稳定性。它们通过去耦对齐模块（DAM）协调工作，该模块在不传播混淆的情况下转移知识。 在流行数据集（S3DIS、ScanNet）上的实验表明，DA-FSS 在多模态少量样本分割方面优于MM-FSS。同时，在几何边界、完整性和纹理区分度上都比基线表现更优。代码可以在以下链接获得：this https URL.
URL
https://arxiv.org/abs/2601.01456
