Abstract
Despite the promising progress in subject-driven image generation, current models often deviate from the reference identities and struggle in complex scenes with multiple subjects. To address this challenge, we introduce OpenSubject, a video-derived large-scale corpus with 2.5M samples and 4.35M images for subject-driven generation and manipulation. The dataset is built with a four-stage pipeline that exploits cross-frame identity priors. (i) Video Curation. We apply resolution and aesthetic filtering to obtain high-quality clips. (ii) Cross-Frame Subject Mining and Pairing. We utilize vision-language model (VLM)-based category consensus, local grounding, and diversity-aware pairing to select image pairs. (iii) Identity-Preserving Reference Image Synthesis. We introduce segmentation map-guided outpainting to synthesize the input images for subject-driven generation and box-guided inpainting to generate input images for subject-driven manipulation, together with geometry-aware augmentations and irregular boundary erosion. (iv) Verification and Captioning. We utilize a VLM to validate synthesized samples, re-synthesize failed samples based on stage (iii), and then construct short and long captions. In addition, we introduce a benchmark covering subject-driven generation and manipulation, and then evaluate identity fidelity, prompt adherence, manipulation consistency, and background consistency with a VLM judge. Extensive experiments show that training with OpenSubject improves generation and manipulation performance, particularly in complex scenes.
Abstract (translated)
尽管主题驱动的图像生成在取得显著进展的同时，当前模型仍然常常偏离参考身份，并且在处理包含多个主体的复杂场景时存在困难。为了解决这一挑战，我们引入了OpenSubject，这是一个基于视频的大规模语料库，包含250万样本和435万张图片，旨在用于主题驱动生成与操作。该数据集通过一个四阶段管道构建，并利用跨帧身份先验进行优化。 (i) 视频筛选：应用分辨率和美学过滤以获取高质量的片段。 (ii) 跨帧主体挖掘与配对：采用基于视觉-语言模型（VLM）的类别共识、局部定位以及多样性感知匹配来选择图像对。 (iii) 保持身份一致性参考图像合成：引入分割图引导的扩展技术来合成用于主题驱动生成的输入图像，并使用边界框指导的填充技术为操作任务生成输入图片。此外，还结合了几何增强和不规则边界的侵蚀处理。 (iv) 验证与描述：利用VLM验证合成样本，针对阶段(iii)中失败的样本进行重新合成，并构建短长双语描述。 此外，我们还提出了一项涵盖主题驱动生成与操作的新基准测试，并使用基于视觉-语言模型（VLM）的评估标准来评估身份保真度、指令遵循性、操作一致性以及背景一致性。广泛的实验表明，使用OpenSubject进行训练可以提高在复杂场景中的图像生成和编辑性能。
URL
https://arxiv.org/abs/2512.08294
