Abstract
In light of recent breakthroughs in text-to-image (T2I) generation, particularly with diffusion transformers (DiT), subject-driven technologies are increasingly being employed for high-fidelity customized production that preserves subject identity from reference inputs, enabling thrilling design workflows and engaging entertainment. Existing alternatives typically require either per-subject optimization via trainable text embeddings or training specialized encoders for subject feature extraction on large-scale datasets. Such dependencies on training procedures fundamentally constrain their practical applications. More importantly, current methodologies fail to fully leverage the inherent zero-shot potential of modern diffusion transformers (e.g., the Flux series) for authentic subject-driven synthesis. To bridge this gap, we propose FreeCus, a genuinely training-free framework that activates DiT's capabilities through three key innovations: 1) We introduce a pivotal attention sharing mechanism that captures the subject's layout integrity while preserving crucial editing flexibility. 2) Through a straightforward analysis of DiT's dynamic shifting, we propose an upgraded variant that significantly improves fine-grained feature extraction. 3) We further integrate advanced Multimodal Large Language Models (MLLMs) to enrich cross-modal semantic representations. Extensive experiments reflect that our method successfully unlocks DiT's zero-shot ability for consistent subject synthesis across diverse contexts, achieving state-of-the-art or comparable results compared to approaches that require additional training. Notably, our framework demonstrates seamless compatibility with existing inpainting pipelines and control modules, facilitating more compelling experiences. Our code is available at: this https URL.
Abstract (translated)
鉴于近期在文本到图像（T2I）生成方面，特别是扩散变压器（DiT）方面的突破，基于主题的驱动技术越来越多地被用于高保真定制化生产，该过程从参考输入中保持主体身份的同时支持令人兴奋的设计工作流程和娱乐活动。现有的替代方法通常需要通过可训练的文本嵌入进行每主体优化或在大规模数据集上对特定于主体特征提取的编码器进行专门培训。这种依赖于训练程序的因素从根本上限制了它们的实际应用范围。更重要的是，当前的方法未能充分利用现代扩散变压器（例如Flux系列）固有的零样本潜力来进行真实的主题驱动合成。 为了解决这一差距，我们提出了FreeCus，这是一个真正的无需训练的框架，通过三个关键创新来激活DiT的能力：1）我们引入了一个至关重要的注意力共享机制，该机制在捕捉主体布局完整性的同时保持了必要的编辑灵活性。2）通过对DiT动态偏移的简单分析，我们提出了一种改进的变体，显著提升了细粒度特征提取能力。3）我们进一步整合了先进的多模态大型语言模型（MLLM），以丰富跨模式语义表示。 广泛的实验表明，我们的方法成功地解锁了DiT在各种上下文中的零样本能力，进行一致的主题合成，并且与需要额外训练的方法相比取得了最先进的或可比较的结果。值得注意的是，我们的框架展示了与其现有的修补管线和控制模块无缝兼容的特性，从而促进了更加引人入胜的体验。我们的代码可在以下网址获取：this https URL。
URL
https://arxiv.org/abs/2507.15249
