Abstract
Recent advances in multi-modal large language models (MLLMs) have significantly improved object-level grounding and region captioning, but remain limited in visual relation understanding (\eg, scene graph generation), particularly in modeling \textit{N}-ary relationships that identify multiple semantic roles among an action event. Such a lack of \textit{semantic dependencies} modeling among multi-entities leads to unreliable outputs, intensifying MLLMs' hallucinations and over-reliance on language priors. To this end, we propose Relation-R1, the first unified relational comprehension framework that explicitly integrates cognitive chain-of-thought (CoT)-guided Supervised Fine-Tuning (SFT) and Group Relative Policy Optimization (GRPO) within a reinforcement learning (RL) paradigm. Specifically, we first establish foundational reasoning capabilities via SFT, enforcing structured outputs with thinking processes. Then, GRPO is utilized to refine these outputs via multi-reward optimization, prioritizing visual-semantic grounding over language-induced biases, thereby improving generalization capability. Extensive experiments on widely-used PSG and SWiG datasets demonstrate that Relation-R1 achieves state-of-the-art performance in both binary and \textit{N}-ary relation understanding.
Abstract (translated)
最近在多模态大型语言模型（MLLMs）方面的进展显著提高了对象级定位和区域描述的能力，但在视觉关系理解方面仍存在局限性（例如，场景图生成），特别是在建模涉及事件中多个实体的多重语义角色的\textit{N}-元关系时。这种对多实体之间缺乏\textit{语义依赖}模型导致了不可靠的结果，加剧了MLLMs的幻想倾向和过度依赖语言先验知识。 为此，我们提出了Relation-R1，这是第一个统一的关系理解框架，它明确地在强化学习（RL）范式内整合了认知链式思考（CoT）引导的监督微调（SFT）和组相对策略优化（GRPO）。具体来说，我们首先通过SFT建立了基础推理能力，并强制执行结构化的输出和思维过程。然后利用GRPO通过多奖励优化来精炼这些输出，优先考虑视觉语义定位而非语言诱导偏差，从而提高泛化能力。 在广泛使用的PSG和SWiG数据集上的大量实验表明，Relation-R1在二元关系理解和\textit{N}-元关系理解方面都达到了最先进的性能。
URL
https://arxiv.org/abs/2504.14642
