Abstract
Referring Expression Segmentation (RES) is a widely explored multi-modal task, which endeavors to segment the pre-existing object within a single image with a given linguistic expression. However, in broader real-world scenarios, it is not always possible to determine if the described object exists in a specific image. Typically, we have a collection of images, some of which may contain the described objects. The current RES setting curbs its practicality in such situations. To overcome this limitation, we propose a more realistic and general setting, named Group-wise Referring Expression Segmentation (GRES), which expands RES to a collection of related images, allowing the described objects to be present in a subset of input images. To support this new setting, we introduce an elaborately compiled dataset named Grouped Referring Dataset (GRD), containing complete group-wise annotations of target objects described by given expressions. We also present a baseline method named Grouped Referring Segmenter (GRSer), which explicitly captures the language-vision and intra-group vision-vision interactions to achieve state-of-the-art results on the proposed GRES and related tasks, such as Co-Salient Object Detection and RES. Our dataset and codes will be publicly released in this https URL.
Abstract (translated)
refering expression segmentation (RES) 是一种被广泛探索的多模态任务，旨在在一个图像中以给定的语言学表达式分割预先存在的物体。然而，在更广泛的现实世界场景中，并不一定能够确定所描述的物体是否存在于特定的图像中。通常我们有一个集合的图像，其中一些可能包含所描述的对象。当前 RES 设定在这种情况中限制了其实用性。为了克服这一限制，我们提出了一种更现实和一般性的设定，称为群体 refering expression segmentation (GRES)，将 RES扩展到一组相关的图像，使所描述的对象能够在输入图像的子集中找到。为了支持这个新设定，我们介绍了一个精心构造的dataset，名为群体 refering Dataset (GRD)，其中包含由给定表达式描述的目标对象的完整的群体注释。我们还介绍了一个基线方法，称为群体 refering Segmenter (GRSer)，它 explicitly 捕捉到语言-视觉和群体内部视觉-视觉交互，以实现所提出的 GRES 和相关的任务，如共同关键对象检测和 RES 的结果。我们的数据和代码将在这个 https URL 上公开发布。
URL
https://arxiv.org/abs/2305.12452
