Abstract
2D image understanding is a complex problem within Computer Vision, but it holds the key to providing human level scene comprehension. It goes further than identifying the objects in an image, and instead it attempts to understand the scene. Solutions to this problem form the underpinning of a range of tasks, including image captioning, Visual Question Answering (VQA), and image retrieval. Graphs provide a natural way to represent the relational arrangement between objects in an image, and thus in recent years Graph Neural Networks (GNNs) have become a standard component of many 2D image understanding pipelines, becoming a core architectural component especially in the VQA group of tasks. In this survey, we review this rapidly evolving field and we provide a taxonomy of graph types used in 2D image understanding approaches, a comprehensive list of the GNN models used in this domain, and a roadmap of future potential developments. To the best of our knowledge, this is the first comprehensive survey that covers image captioning, visual question answering, and image retrieval techniques that focus on using GNNs as the main part of their architecture.
Abstract (translated)
2D图像理解是计算机视觉中的复杂问题,但这是提供人类水平场景理解的关键。它不仅仅是识别图像中的物体,而是试图理解场景。解决这个问题的解决方案是一系列任务的基础,包括图像标题制作、视觉问答(VQA)和图像检索。Graphs提供了一种自然的方式来代表图像中物体之间的关系安排,因此Graph Neural Networks(GNNs)在近年来已成为许多2D图像理解管道的标准组件,特别是在VQA任务组中成为了核心建筑组件。在这个调查中,我们回顾了这一快速发展的领域,并提供了2D图像理解方法中使用的Graph类型的分类,了一份这个领域内GNN模型的全面列表,以及未来潜在发展的路线图。据我们所知,这是第一个涵盖了图像标题制作、视觉问答和图像检索技巧的全面调查,重点使用GNNs将其架构的主要部分作为主要部分。
URL
https://arxiv.org/abs/2303.03761
