Abstract
Previous studies on visual customization primarily rely on the objective alignment between various control signals (e.g., language, layout and canny) and the edited images, which largely ignore the subjective emotional contents, and more importantly lack general-purpose foundation models for affective visual customization. With this in mind, this paper proposes an LLM-centric Affective Visual Customization (L-AVC) task, which focuses on generating images within modifying their subjective emotions via Multimodal LLM. Further, this paper contends that how to make the model efficiently align emotion conversion in semantics (named inter-emotion semantic conversion) and how to precisely retain emotion-agnostic contents (named exter-emotion semantic retaining) are rather important and challenging in this L-AVC task. To this end, this paper proposes an Efficient and Precise Emotion Manipulating approach for editing subjective emotions in images. Specifically, an Efficient Inter-emotion Converting (EIC) module is tailored to make the LLM efficiently align emotion conversion in semantics before and after editing, followed by a Precise Exter-emotion Retaining (PER) module to precisely retain the emotion-agnostic contents. Comprehensive experimental evaluations on our constructed L-AVC dataset demonstrate the great advantage of the proposed EPEM approach to the L-AVC task over several state-of-the-art baselines. This justifies the importance of emotion information for L-AVC and the effectiveness of EPEM in efficiently and precisely manipulating such information.
Abstract (translated)
之前的研究在视觉定制领域主要依赖于不同控制信号（如语言、布局和边缘检测）与编辑后图像之间的客观对齐，而忽视了主观情感内容，并且缺乏用于情感视觉定制的通用基础模型。鉴于此，本文提出了一个以大型多模态语言模型为中心的情感视觉定制（L-AVC）任务，该任务专注于通过多模态LLM生成和修改图像中的主观情绪。此外，本文认为在L-AVC任务中，如何使模型有效地对齐编辑前后的情绪语义转换（称为跨情感语义转换），以及如何精确地保留与情感无关的内容（称为外情感语义保持）是非常重要且具有挑战性的。 为此，本文提出了一种有效而精准的情感操控方法（EPEM），用于在图像中编辑主观情绪。具体来说，设计了一个有效的跨情感转换（EIC）模块，使LLM能够高效地对齐编辑前后的语义情绪转换，并随后使用一个精确的外情感保留（PER）模块来保持与情感无关的内容。 本文构建了L-AVC数据集并进行了全面的实验评估，结果表明所提出的EPEM方法在L-AVC任务中相对于几个最先进的基线模型具有明显优势。这证明了情感信息对于L-AVC的重要性以及EPEM在高效和精准操控此类信息方面的有效性。
URL
https://arxiv.org/abs/2602.18016
