Abstract
In this work, we focus on the task of weakly supervised affordance grounding, where a model is trained to identify affordance regions on objects using human-object interaction images and egocentric object images without dense labels. Previous works are mostly built upon class activation maps, which are effective for semantic segmentation but may not be suitable for locating actions and functions. Leveraging recent advanced foundation models, we develop a supervised training pipeline based on pseudo labels. The pseudo labels are generated from an off-the-shelf part segmentation model, guided by a mapping from affordance to part names. Furthermore, we introduce three key enhancements to the baseline model: a label refining stage, a fine-grained feature alignment process, and a lightweight reasoning module. These techniques harness the semantic knowledge of static objects embedded in off-the-shelf foundation models to improve affordance learning, effectively bridging the gap between objects and actions. Extensive experiments demonstrate that the performance of the proposed model has achieved a breakthrough improvement over existing methods. Our codes are available at this https URL .
Abstract (translated)
在这项工作中，我们专注于弱监督下的功能区域定位任务，即在没有密集标签的情况下，通过使用人类与物体交互的图像和第一人称视角的对象图像来训练模型识别对象的功能区域。以往的工作大多基于类激活图（CAMs），这类方法对语义分割非常有效，但对于定位动作和功能可能不太适用。我们利用最近出现的基础模型，开发了一个基于伪标签的监督训练流程。这些伪标签是由现成的部分分割模型生成，并通过功能到部件名称的映射进行引导。 此外，我们还为基准模型引入了三个关键改进：标签精炼阶段、细粒度特征对齐过程以及轻量级推理模块。这些技术利用嵌入在现成基础模型中的静态物体语义知识来增强功能学习能力，并有效地弥合了物体与动作之间的差距。广泛的实验表明，所提出的模型的性能在现有方法的基础上取得了重大突破性改进。 我们的代码可在 [此链接](https://this https URL) 获取。
URL
https://arxiv.org/abs/2505.24103
