Abstract
Recent advances in large-scale text-to-image diffusion models (e.g., FLUX.1) have greatly improved visual fidelity in consistent character generation and editing. However, existing methods rarely unify these tasks within a single framework. Generation-based approaches struggle with fine-grained identity consistency across instances, while editing-based methods often lose spatial controllability and instruction alignment. To bridge this gap, we propose ReMix, a unified framework for character-consistent generation and editing. It constitutes two core components: the ReMix Module and IP-ControlNet. The ReMix Module leverages the multimodal reasoning ability of MLLMs to edit semantic features of input images and adapt instruction embeddings to the native DiT backbone without fine-tuning. While this ensures coherent semantic layouts, pixel-level consistency and pose controllability remain challenging. To address this, IP-ControlNet extends ControlNet to decouple semantic and layout cues from reference images and introduces an {\epsilon}-equivariant latent space that jointly denoises the reference and target images within a shared noise space. Inspired by convergent evolution and quantum decoherence,i.e., where environmental noise drives state convergence, this design promotes feature alignment in the hidden space, enabling consistent object generation while preserving identity. ReMix supports a wide range of tasks, including personalized generation, image editing, style transfer, and multi-condition synthesis. Extensive experiments validate its effectiveness and efficiency as a unified framework for character-consistent image generation and editing.
Abstract (translated)
最近在大规模文本到图像扩散模型（例如FLUX.1）方面的进展，极大地提升了连贯人物生成和编辑的视觉保真度。然而，现有的方法很少将这些任务整合在一个单一框架内完成。基于生成的方法难以实现实例间的细微身份一致性，而基于编辑的方法则往往失去空间控制性和指令对齐性。为解决这一问题，我们提出了ReMix框架，旨在统一处理人物连贯性的生成和编辑任务。该框架包含两个核心组件：ReMix模块和IP-ControlNet。 - **ReMix模块**利用多模态推理能力来编辑输入图像的语义特征，并将指令嵌入到DiT骨干网络中，而无需进行微调。尽管这确保了连贯的语义布局，但像素级一致性以及姿态控制仍然具有挑战性。 - **IP-ControlNet**则在ControlNet的基础上进行了扩展，以从参考图像中解耦出语义和布局线索，并引入了一个{\epsilon}-等变隐空间，该空间能在共享噪声空间内同时对齐参考图和目标图的去噪过程。这种设计灵感来源于趋同进化与量子退相干现象，即环境中的噪声驱动状态收敛。这一设计在隐藏空间中促进了特征的对齐，在保持身份一致性的前提下实现了物体生成的一致性。 ReMix框架支持多种任务，包括个性化生成、图像编辑、风格迁移和多条件合成等，并通过广泛的实验验证了其作为统一框架的有效性和效率，在连贯人物图像生成和编辑领域表现卓越。
URL
https://arxiv.org/abs/2510.10156
