Abstract
We introduce SceneLinker, a novel framework that generates compositional 3D scenes via semantic scene graph from RGB sequences. To adaptively experience Mixed Reality (MR) content based on each user's space, it is essential to generate a 3D scene that reflects the real-world layout by compactly capturing the semantic cues of the surroundings. Prior works struggled to fully capture the contextual relationship between objects or mainly focused on synthesizing diverse shapes, making it challenging to generate 3D scenes aligned with object arrangements. We address these challenges by designing a graph network with cross-check feature attention for scene graph prediction and constructing a graph-variational autoencoder (graph-VAE), which consists of a joint shape and layout block for 3D scene generation. Experiments on the 3RScan/3DSSG and SG-FRONT datasets demonstrate that our approach outperforms state-of-the-art methods in both quantitative and qualitative evaluations, even in complex indoor environments and under challenging scene graph constraints. Our work enables users to generate consistent 3D spaces from their physical environments via scene graphs, allowing them to create spatial MR content. Project page is this https URL.
Abstract (translated)
我们介绍了一种名为SceneLinker的新框架，该框架通过从RGB序列生成语义场景图来创建组合式3D场景。为了根据每个用户的实际空间自适应地体验混合现实（MR）内容，有必要生成一个能够捕捉周围环境语义线索并反映真实世界布局的3D场景。先前的研究难以全面捕捉对象之间的上下文关系，或者主要关注于合成各种形状，这使得在与物体排列一致的情况下生成3D场景变得具有挑战性。 我们通过设计一种带有交叉检查特征注意机制的图网络来解决这些问题，并构建了一个图变分自动编码器（graph-VAE），其中包括一个用于3D场景生成的联合形状和布局块。在3RScan/3DSSG和SG-FRONT数据集上的实验表明，我们的方法在定量和定性评估中均优于现有技术，在复杂的室内环境以及具有挑战性的场景图约束下也表现良好。 我们的工作使得用户能够通过场景图从物理环境中生成一致的3D空间，并允许他们创建基于空间的MR内容。项目页面可以在这里访问：[https URL]（请将方括号中的文本替换为实际链接）。
URL
https://arxiv.org/abs/2602.02974
