Abstract
Open-vocabulary change detection (OVCD) seeks to recognize arbitrary changes of interest by enabling generalization beyond a fixed set of predefined classes. We reformulate OVCD as a two-stage pipeline: first generate class-agnostic change proposals using visual foundation models (VFMs) such as SAM and DINOv2, and then perform category identification with vision-language models (VLMs) such as CLIP. We reveal that category identification errors are the primary bottleneck of OVCD, mainly due to the limited ability of VLMs based on image-text matching to represent fine-grained land-cover categories. To address this, we propose OpenDPR, a training-free vision-centric diffusion-guided prototype retrieval framework. OpenDPR leverages diffusion models to construct diverse prototypes for target categories offline, and to perform similarity retrieval with change proposals in the visual space during inference. The secondary bottleneck lies in change localization, due to the inherent lack of change priors in VFMs. To bridge this gap, we design a spatial-to-change weakly supervised change detection module named S2C to adapt their strong spatial modeling capabilities for change localization. Integrating the pretrained S2C into OpenDPR leads to an optional weakly supervised variant named OpenDPR-W, which further improves OVCD with minimal supervision. Experimental results on four benchmark datasets demonstrate that the proposed methods achieve state-of-the-art performance under both supervision modes. Code is available at this https URL.
Abstract (translated)
开放词汇变化检测（OVCD）旨在通过超越预定义固定类别的泛化能力，识别任意感兴趣的变化。我们将OVCD重构为两阶段流程：首先利用SAM和DINOv2等视觉基础模型（VFMs）生成与类别无关的变化提案，随后通过CLIP等视觉-语言模型（VLMs）进行类别识别。研究发现，类别识别错误是OVCD的主要瓶颈，这主要源于基于图像-文本匹配的VLM在表征细粒度土地覆盖类别方面的能力有限。为此，我们提出OpenDPR——一个无需训练的、以视觉为中心的扩散引导原型检索框架。OpenDPR利用扩散模型离线构建目标类别的多样化原型，并在推理阶段在视觉空间与变化提案进行相似性检索。次要瓶颈在于变化定位，这源于VFMs本身缺乏变化先验知识。为弥合这一差距，我们设计了名为S2C的空间到变化弱监督变化检测模块，使其强大的空间建模能力适配变化定位任务。将预训练的S2C集成到OpenDPR中，形成了可选的弱监督变体OpenDPR-W，该变体通过最小监督进一步提升了OVCD性能。在四个基准数据集上的实验结果表明，所提方法在两种监督模式下均达到最先进性能。代码已公开于此链接。
URL
https://arxiv.org/abs/2603.27645
