Abstract
Scene Graph Generation (SGG) is a task that encodes visual relationships between objects in images as graph structures. SGG shows significant promise as a foundational component for downstream tasks, such as reasoning for embodied agents. To enable real-time applications, SGG must address the trade-off between performance and inference speed. However, current methods tend to focus on one of the following: (1) improving relation prediction accuracy, (2) enhancing object detection accuracy, or (3) reducing latency, without aiming to balance all three objectives simultaneously. To address this limitation, we build on the powerful Real-time Efficiency and Accuracy Compromise for Tradeoffs in Scene Graph Generation (REACT) architecture and propose REACT++, a new state-of-the-art model for real-time SGG. By leveraging efficient feature extraction and subject-to-object cross-attention within the prototype space, REACT++ balances latency and representational power. REACT++ achieves the highest inference speed among existing SGG models, improving relation prediction accuracy without sacrificing object detection performance. Compared to the previous REACT version, REACT++ is 20% faster with a gain of 10% in relation prediction accuracy on average. The code is available at this https URL.
Abstract (translated)
场景图生成（SGG）是一项任务，旨在将图像中对象之间的视觉关系编码为图结构。SGG作为下游任务的基础组件表现出巨大潜力，例如用于具身代理的推理。为了实现实时应用，SGG必须解决性能和推断速度之间的权衡问题。然而，目前的方法往往集中于以下三个目标之一：(1) 提高关系预测准确性；(2) 增强对象检测准确性；或 (3) 减少延迟，并没有同时兼顾这三个目标。 为了解决这一局限性，我们基于强大的“实时效率与精度权衡的场景图生成架构”（REACT）构建了 REACT++，这是一种新的、面向实时SGG的状态-of-the-art模型。通过利用高效的特征提取和原型空间内的主客体交叉注意机制，REACT++ 平衡了延迟和表示能力之间的关系。REACT++ 达到了现有 SGG 模型中最快的推断速度，在不牺牲对象检测性能的情况下提高了关系预测准确性。 与之前的 REACT 版本相比，REACT++ 在关系预测准确性的平均值上提升了10%，并且速度快20%。代码可在该网址获得：[此链接指向的是原文中的URL位置，请根据实际情况提供或省略]。
URL
https://arxiv.org/abs/2603.06386
