Abstract
Maintaining stylistic consistency is crucial for the cohesion and aesthetic appeal of images, a fundamental requirement in effective image editing and inpainting. However, existing methods primarily focus on the semantic control of generated content, often neglecting the critical task of preserving this consistency. In this work, we introduce the Neural Scene Designer (NSD), a novel framework that enables photo-realistic manipulation of user-specified scene regions while ensuring both semantic alignment with user intent and stylistic consistency with the surrounding environment. NSD leverages an advanced diffusion model, incorporating two parallel cross-attention mechanisms that separately process text and style information to achieve the dual objectives of semantic control and style consistency. To capture fine-grained style representations, we propose the Progressive Self-style Representational Learning (PSRL) module. This module is predicated on the intuitive premise that different regions within a single image share a consistent style, whereas regions from different images exhibit distinct styles. The PSRL module employs a style contrastive loss that encourages high similarity between representations from the same image while enforcing dissimilarity between those from different images. Furthermore, to address the lack of standardized evaluation protocols for this task, we establish a comprehensive benchmark. This benchmark includes competing algorithms, dedicated style-related metrics, and diverse datasets and settings to facilitate fair comparisons. Extensive experiments conducted on our benchmark demonstrate the effectiveness of the proposed framework.
Abstract (translated)
保持风格一致性对于图像的连贯性和审美吸引力至关重要，这是有效图像编辑和修复的基本要求。然而，现有方法主要集中在生成内容的语义控制上，常常忽略了维持这种一致性的关键任务。在这项工作中，我们引入了神经场景设计器（NSD），这是一种新颖的框架，它能够实现用户指定场景区域的真实感操作，同时确保与用户意图的语义对齐和周围环境的一致风格。NSD利用了一个先进的扩散模型，并采用了两个并行的交叉注意力机制来分别处理文本信息和样式信息，从而实现了语义控制和风格一致性这两大目标。 为了捕捉精细的风格表示，我们提出了渐进式自风格表征学习（PSRL）模块。该模块基于这样一个直观的前提：单个图像内的不同区域具有相同的风格，而来自不同图像的不同区域则具有不同的风格。PSRL模块采用了样式对比损失来鼓励同一图像中的表示相似性，并确保不同图像之间的表示差异。 此外，为了应对这一任务缺乏标准化评估协议的问题，我们建立了一个全面的基准测试。该基准测试包括竞争算法、专门针对样式的相关度量指标以及多样化的数据集和设置，以促进公平比较。我们在我们的基准上进行了广泛的实验，证明了所提出的框架的有效性。
URL
https://arxiv.org/abs/2509.01405
