Abstract
Recent advances in text-to-image (T2I) generation have shown remarkable success in producing high-quality images from text. However, existing T2I models show decayed performance in compositional image generation involving multiple objects and intricate relationships. We attribute this problem to limitations in existing datasets of image-text pairs, which lack precise inter-object relationship annotations with prompts only. To address this problem, we construct LAION-SG, a large-scale dataset with high-quality structural annotations of scene graphs (SG), which precisely describe attributes and relationships of multiple objects, effectively representing the semantic structure in complex scenes. Based on LAION-SG, we train a new foundation model SDXL-SG to incorporate structural annotation information into the generation process. Extensive experiments show advanced models trained on our LAION-SG boast significant performance improvements in complex scene generation over models on existing datasets. We also introduce CompSG-Bench, a benchmark that evaluates models on compositional image generation, establishing a new standard for this domain.
Abstract (translated)
近期的文本到图像（T2I）生成技术在从文本生产高质量图像方面取得了显著的成功。然而，现有的T2I模型在涉及多个对象和复杂关系的组合图像生成中表现不佳。我们把这个问题归因于现有图像-文本配对数据集的局限性，这些数据集中缺乏精准的对象间关系标注，仅有提示词。为了解决这个问题，我们构建了LAION-SG，这是一个大规模的数据集，包含高质量的场景图（SG）结构化标注，能够精确描述多个对象的属性和它们之间的关系，有效地表达复杂场景中的语义结构。基于LAION-SG，我们训练了一个新的基础模型SDXL-SG，将结构化注释信息融入生成过程中。广泛的实验显示，在我们的LAION-SG数据集上训练的先进模型在复杂场景生成方面显著优于现有数据集上的模型。此外，我们还引入了CompSG-Bench，这是一个评估组合图像生成能力的标准基准测试，为该领域设立了新的标准。
URL
https://arxiv.org/abs/2412.08580
