Style_Transfer
-
PaintFlow: A Unified Framework for Interactive Oil Paintings Editing and Generation
2025-12-09 12:31:00Zhangli Hu, Ye Chen, Jiajun Yao, Bingbing NiAbstract
Oil painting, as a high-level medium that blends human abstract thinking with artistic expression, poses substantial challenges for digital generation and editing due to its intricate brushstroke dynamics and stylized characteristics. Existing generation and editing techniques are often constrained by the distribution of training data and primarily focus on modifying real photographs. In this work, we introduce a unified multimodal framework for oil painting generation and editing. The proposed system allows users to incorporate reference images for precise semantic control, hand-drawn sketches for spatial structure alignment, and natural language prompts for high-level semantic guidance, while consistently maintaining a unified painting style across all outputs. Our method achieves interactive oil painting creation through three crucial technical advancements. First, we enhance the training stage with spatial alignment and semantic enhancement conditioning strategy, which map masks and sketches into spatial constraints, and encode contextual embedding from reference images and text into feature constraints, enabling object-level semantic alignment. Second, to overcome data scarcity, we propose a self-supervised style transfer pipeline based on Stroke-Based Rendering (SBR), which simulates the inpainting dynamics of oil painting restoration, converting real images into stylized oil paintings with preserved brushstroke textures to construct a large-scale paired training dataset. Finally, during inference, we integrate features using the AdaIN operator to ensure stylistic consistency. Extensive experiments demonstrate that our interactive system enables fine-grained editing while preserving the artistic qualities of oil paintings, achieving an unprecedented level of imagination realization in stylized oil paintings generation and editing.
Abstract (translated)
油画作为一种将人类抽象思维与艺术表达相结合的高级媒介，由于其复杂的笔触动态和风格化特征，在数字生成和编辑方面面临着重大挑战。现有的生成和编辑技术通常受到训练数据分布的限制，并且主要侧重于修改真实照片。在这项工作中，我们引入了一个统一的多模态框架，用于油画的生成和编辑。该系统允许用户通过参考图像进行精确语义控制、手绘草图进行空间结构对齐以及自然语言提示进行高层次语义指导的同时，在所有输出中保持一致的绘画风格。我们的方法通过三项关键技术进步实现了互动式油画创作。 首先，我们加强了训练阶段的空间对齐和语义增强条件策略，将蒙版和草图映射为空间约束，并从参考图像和文本编码上下文嵌入以形成特征约束，从而实现对象级别的语义对齐。其次，为了克服数据稀缺性的问题，我们提出了一种基于笔触基渲染（SBR）的自我监督风格迁移管道，该方法模拟了油画修复中的填色动态，将真实图像转换为具有保存下来的笔触纹理的风格化油画，以构建大规模成对训练数据集。最后，在推理阶段，我们使用AdaIN操作符整合特征，确保风格一致性。 广泛的实验表明，我们的互动系统能够在保留油画艺术品质的同时进行精细编辑，实现了在风格化油画生成和编辑方面前所未有的想象力实现水平。
URL
https://arxiv.org/abs/2512.08534
PDF
-
DIST-CLIP: Arbitrary Metadata and Image Guided MRI Harmonization via Disentangled Anatomy-Contrast Representations
2025-12-08 16:09:10Mehmet Yigit Avci, Pedro Borges, Virginia Fernandez, Paul Wright, Mehmet Yigitsoy, Sebastien Ourselin, Jorge CardosoAbstract
Deep learning holds immense promise for transforming medical image analysis, yet its clinical generalization remains profoundly limited. A major barrier is data heterogeneity. This is particularly true in Magnetic Resonance Imaging, where scanner hardware differences, diverse acquisition protocols, and varying sequence parameters introduce substantial domain shifts that obscure underlying biological signals. Data harmonization methods aim to reduce these instrumental and acquisition variability, but existing approaches remain insufficient. When applied to imaging data, image-based harmonization approaches are often restricted by the need for target images, while existing text-guided methods rely on simplistic labels that fail to capture complex acquisition details or are typically restricted to datasets with limited variability, failing to capture the heterogeneity of real-world clinical environments. To address these limitations, we propose DIST-CLIP (Disentangled Style Transfer with CLIP Guidance), a unified framework for MRI harmonization that flexibly uses either target images or DICOM metadata for guidance. Our framework explicitly disentangles anatomical content from image contrast, with the contrast representations being extracted using pre-trained CLIP encoders. These contrast embeddings are then integrated into the anatomical content via a novel Adaptive Style Transfer module. We trained and evaluated DIST-CLIP on diverse real-world clinical datasets, and showed significant improvements in performance when compared against state-of-the-art methods in both style translation fidelity and anatomical preservation, offering a flexible solution for style transfer and standardizing MRI data. Our code and weights will be made publicly available upon publication.
Abstract (translated)
深度学习在医学图像分析方面展现出巨大的潜力，然而其临床适用性的推广仍然受到严重限制。一个主要障碍是数据异质性问题，在磁共振成像（MRI）中尤其明显。MRI扫描仪硬件差异、多样的采集协议和不同的序列参数引入了显著的领域偏移，从而掩盖了潜在的生物学信号。为了减少这些仪器和采集过程中的变异性，人们开发了一系列数据校准方法，但现有的方法仍然不够完善。在应用于图像数据时，基于图像的方法通常受到需要目标图像限制，而现有文本引导的方法则依赖于简单的标签，无法捕捉复杂的采集细节或仅限于具有有限变异性的数据集，从而不能适应现实世界临床环境中的异质性。 为了解决这些问题，我们提出了一种名为DIST-CLIP（使用CLIP指导的解耦风格迁移）的统一框架，用于MRI校准。该框架灵活地利用目标图像或DICOM元数据进行引导，并明确分离出解剖内容和图像对比度。使用预先训练好的CLIP编码器从输入图像中提取对比度表示，这些对比度嵌入随后通过新颖的自适应风格迁移模块融入到解剖结构当中。 我们在多种真实世界的临床数据集上对DIST-CLIP进行了训练和评估，并显示了其在风格翻译准确性和解剖细节保留方面的性能相比现有最先进的方法有了显著提高。这为MRI图像的数据标准化提供了一个灵活且高效的解决方案。我们的代码和权重将在发表后公开发布。
URL
https://arxiv.org/abs/2512.07674
PDF
-
Controllable Long-term Motion Generation with Extended Joint Targets
2025-12-04 05:44:15Eunjong Lee, Eunhee Kim, Sanghoon Hong, Eunho Jung, Jihoon KimAbstract
Generating stable and controllable character motion in real-time is a key challenge in computer animation. Existing methods often fail to provide fine-grained control or suffer from motion degradation over long sequences, limiting their use in interactive applications. We propose COMET, an autoregressive framework that runs in real time, enabling versatile character control and robust long-horizon synthesis. Our efficient Transformer-based conditional VAE allows for precise, interactive control over arbitrary user-specified joints for tasks like goal-reaching and in-betweening from a single model. To ensure long-term temporal stability, we introduce a novel reference-guided feedback mechanism that prevents error accumulation. This mechanism also serves as a plug-and-play stylization module, enabling real-time style transfer. Extensive evaluations demonstrate that COMET robustly generates high-quality motion at real-time speeds, significantly outperforming state-of-the-art approaches in complex motion control tasks and confirming its readiness for demanding interactive applications.
Abstract (translated)
实时生成稳定且可控的角色动画是计算机动画中的一个关键挑战。现有方法往往难以提供精细的控制，或者在长时间序列中会出现动作退化的问题，从而限制了它们在交互式应用程序中的应用。我们提出了COMET（条件变分自编码器框架），这是一个自回归框架，在实时运行时能够实现多样化的角色控制和长期稳定的合成。 我们的高效Transformer基条件VAE模型允许用户对任意指定的关节进行精确、互动式的控制，适用于目标到达和过渡动作等任务，并且仅需一个模型即可完成。为了确保长时间序列中的时间稳定性，我们引入了一种新颖的参考引导反馈机制，以防止错误积累。这一机制还作为插件式的风格化模块存在，能够实现实时风格转换。 广泛的评估表明，COMET能够在实时速度下生成高质量的动作，并在复杂动作控制任务中显著优于现有最先进的方法，确认了它对于苛刻交互式应用的适用性。
URL
https://arxiv.org/abs/2512.04487
PDF
-
ClusterStyle: Modeling Intra-Style Diversity with Prototypical Clustering for Stylized Motion Generation
2025-12-02 06:24:14Kerui Chen, Jianrong Zhang, Ming Li, Zhonglong Zheng, Hehe FanAbstract
Existing stylized motion generation models have shown their remarkable ability to understand specific style information from the style motion, and insert it into the content motion. However, capturing intra-style diversity, where a single style should correspond to diverse motion variations, remains a significant challenge. In this paper, we propose a clustering-based framework, ClusterStyle, to address this limitation. Instead of learning an unstructured embedding from each style motion, we leverage a set of prototypes to effectively model diverse style patterns across motions belonging to the same style category. We consider two types of style diversity: global-level diversity among style motions of the same category, and local-level diversity within the temporal dynamics of motion sequences. These components jointly shape two structured style embedding spaces, i.e., global and local, optimized via alignment with non-learnable prototype anchors. Furthermore, we augment the pretrained text-to-motion generation model with the Stylistic Modulation Adapter (SMA) to integrate the style features. Extensive experiments demonstrate that our approach outperforms existing state-of-the-art models in stylized motion generation and motion style transfer.
Abstract (translated)
现有的风格化运动生成模型已经展示了其从风格动作中理解特定风格信息，并将其插入内容运动的卓越能力。然而，捕捉跨风格多样性（即单一风格应当对应多种多样的运动变化）仍然是一个重大挑战。在本文中，我们提出了一种基于聚类的框架ClusterStyle来解决这一限制。该方法不再从每个风格化的动作学习无结构嵌入，而是利用一组原型有效建模同一类别内不同运动中的多样风格模式。我们考虑两种类型的风格多样性：同类风格动作之间的全局层次多样性以及运动序列时间动态内的局部层次多样性。这些组成部分共同塑造了两个有组织的风格嵌入空间，即全局和局部，并通过与非可学习的原型锚点对齐进行优化。 此外，我们将预训练的文本到运动生成模型用带有风格化调制适配器（SMA）增强，以整合风格特征。广泛的实验表明，我们的方法在风格化动作生成和动作风格转换方面超过了现有的最先进的模型性能。
URL
https://arxiv.org/abs/2512.02453
PDF
-
SAGE: Style-Adaptive Generalization for Privacy-Constrained Semantic Segmentation Across Domains
2025-12-02 03:20:22Qingmei Li, Yang Zhang, Peifeng Zhang, Haohuan Fu, Juepeng ZhengAbstract
Domain generalization for semantic segmentation aims to mitigate the degradation in model performance caused by domain shifts. However, in many real-world scenarios, we are unable to access the model parameters and architectural details due to privacy concerns and security constraints. Traditional fine-tuning or adaptation is hindered, leading to the demand for input-level strategies that can enhance generalization without modifying model weights. To this end, we propose a \textbf{S}tyle-\textbf{A}daptive \textbf{GE}neralization framework (\textbf{SAGE}), which improves the generalization of frozen models under privacy constraints. SAGE learns to synthesize visual prompts that implicitly align feature distributions across styles instead of directly fine-tuning the backbone. Specifically, we first utilize style transfer to construct a diverse style representation of the source domain, thereby learning a set of style characteristics that can cover a wide range of visual features. Then, the model adaptively fuses these style cues according to the visual context of each input, forming a dynamic prompt that harmonizes the image appearance without touching the interior of the model. Through this closed-loop design, SAGE effectively bridges the gap between frozen model invariance and the diversity of unseen domains. Extensive experiments on five benchmark datasets demonstrate that SAGE achieves competitive or superior performance compared to state-of-the-art methods under privacy constraints and outperforms full fine-tuning baselines in all settings.
Abstract (translated)
领域泛化在语义分割中的目标是缓解由于领域偏移导致的模型性能下降。然而，在许多现实场景中，由于隐私和安全限制的原因，我们无法访问模型参数和架构细节。这使得传统的微调或适应方法受到阻碍，从而产生了需求：通过不修改模型权重的方式来提高泛化能力的方法。为此，我们提出了一个名为**SAGE（Style-Adaptive Generalization框架）**的方案，该框架旨在改进在隐私限制下的冻结模型的泛化性能。 SAGE的学习方式是合成视觉提示，以隐式地跨风格对齐特征分布，而不是直接微调骨干网络。具体来说，我们首先利用风格迁移构建源领域的多样化的风格表示，从而学习一套覆盖广泛视觉特性的风格特征。然后，该模型根据每个输入的视觉上下文自适应融合这些风格线索，形成一个动态提示，这有助于协调图像外观而不触及模型内部结构。通过这种闭环设计，SAGE有效弥合了冻结模型不变性和未见领域多样性的差距。 在五个基准数据集上的广泛实验表明，在隐私限制下，SAGE与现有最佳方法相比取得了竞争性或优越的性能，并且在所有设置中均优于全量微调基线。
URL
https://arxiv.org/abs/2512.02369
PDF
-
LLM2Fx-Tools: Tool Calling For Music Post-Production
2025-12-01 11:30:21Seungheon Doh, Junghyun Koo, Marco A. Mart\'inez-Ram\'irez, Woosung Choi, Wei-Hsiang Liao, Qiyu Wu, Juhan Nam, Yuki MitsufujiAbstract
This paper introduces LLM2Fx-Tools, a multimodal tool-calling framework that generates executable sequences of audio effects (Fx-chain) for music post-production. LLM2Fx-Tools uses a large language model (LLM) to understand audio inputs, select audio effects types, determine their order, and estimate parameters, guided by chain-of-thought (CoT) planning. We also present LP-Fx, a new instruction-following dataset with structured CoT annotations and tool calls for audio effects modules. Experiments show that LLM2Fx-Tools can infer an Fx-chain and its parameters from pairs of unprocessed and processed audio, enabled by autoregressive sequence modeling, tool calling, and CoT reasoning. We further validate the system in a style transfer setting, where audio effects information is transferred from a reference source and applied to new content. Finally, LLM-as-a-judge evaluation demonstrates that our approach generates appropriate CoT reasoning and responses for music production queries. To our knowledge, this is the first work to apply LLM-based tool calling to audio effects modules, enabling interpretable and controllable music production.
Abstract (translated)
本文介绍了LLM2Fx-Tools，这是一个多模态工具调用框架，用于生成音乐后期制作中的可执行音频效果序列（Fx链）。LLM2Fx-Tools 使用大型语言模型 (LLM) 来理解音频输入、选择音频效果类型、确定其顺序并估算参数，通过链条思维 (CoT) 规划来进行引导。我们还介绍了一个新的指令跟随数据集LP-Fx，该数据集带有结构化的CoT注释和音频效果模块的工具调用。实验表明，LLM2Fx-Tools 可以从未经处理和已处理的音频对中推断出 Fx链及其参数，这得益于自回归序列建模、工具调用和 CoT 推理。在风格转移设置下，我们进一步验证了该系统能够将参考源中的音频效果信息转移到新内容上。最后，作为裁判的LLM评估表明，我们的方法可以为音乐制作查询生成合适的CoT推理和响应。据我们所知，这是首次将基于LLM的工具调用应用于音频效果模块的工作，从而使音乐制作变得可解释且可控。
URL
https://arxiv.org/abs/2512.01559
PDF
-
Charts Are Not Images: On the Challenges of Scientific Chart Editing
2025-11-30 06:13:48Shawn Li, Ryan Rossi, Sungchul Kim, Sunav Choudhary, Franck Dernoncourt, Puneet Mathur, Zhengzhong Tu, Yue ZhaoAbstract
Generative models, such as diffusion and autoregressive approaches, have demonstrated impressive capabilities in editing natural images. However, applying these tools to scientific charts rests on a flawed assumption: a chart is not merely an arrangement of pixels but a visual representation of structured data governed by a graphical grammar. Consequently, chart editing is not a pixel-manipulation task but a structured transformation problem. To address this fundamental mismatch, we introduce \textit{FigEdit}, a large-scale benchmark for scientific figure editing comprising over 30,000 samples. Grounded in real-world data, our benchmark is distinguished by its diversity, covering 10 distinct chart types and a rich vocabulary of complex editing instructions. The benchmark is organized into five distinct and progressively challenging tasks: single edits, multi edits, conversational edits, visual-guidance-based edits, and style transfer. Our evaluation of a range of state-of-the-art models on this benchmark reveals their poor performance on scientific figures, as they consistently fail to handle the underlying structured transformations required for valid edits. Furthermore, our analysis indicates that traditional evaluation metrics (e.g., SSIM, PSNR) have limitations in capturing the semantic correctness of chart edits. Our benchmark demonstrates the profound limitations of pixel-level manipulation and provides a robust foundation for developing and evaluating future structure-aware models. By releasing \textit{FigEdit} (this https URL), we aim to enable systematic progress in structure-aware figure editing, provide a common ground for fair comparison, and encourage future research on models that understand both the visual and semantic layers of scientific charts.
Abstract (translated)
生成模型，如扩散和自回归方法，在编辑自然图像方面展示了令人印象深刻的性能。然而，将这些工具应用于科学图表时存在一个基本假设上的缺陷：图表不仅仅是像素的排列，而是遵循图形语法的结构化数据的视觉表示形式。因此，图表编辑不是一个像素操作任务，而是一个需要进行结构化转换的问题。为了解决这一根本性的不匹配问题，我们引入了\textit{FigEdit}，这是一个用于科学图表编辑的大规模基准测试集，包含超过30,000个样本。基于真实世界的数据，我们的基准测试以其多样性和涵盖10种不同类型的图表以及丰富复杂的编辑指令词汇表而著称。该基准测试由五项独特且难度递增的任务组成：单一编辑、多任务编辑、对话式编辑、基于视觉指导的编辑和风格转换。 我们对一系列最先进的模型在这一基准上的评估显示，它们在科学图表上表现不佳，因为这些模型始终无法处理有效编辑所需的底层结构化转换。此外，我们的分析表明，传统的评估指标（如SSIM和PSNR）在捕捉图表编辑中的语义正确性方面存在局限性。 \textit{FigEdit}基准测试揭示了基于像素级别的操作所存在的深刻限制，并为开发和评估未来具备结构感知能力的模型提供了坚实的基础。通过发布\textit{FigEdit}(此链接)，我们的目标是促进结构感知图表编辑方面的系统进步，提供一个公平比较的共同基础，并鼓励对理解科学图表视觉和语义层面模型的研究。
URL
https://arxiv.org/abs/2512.00752
PDF
-
Inversion-Free Style Transfer with Dual Rectified Flows
2025-11-26 02:28:51Yingying Deng, Xiangyu He, Fan Tang, Weiming Dong, Xucheng YinAbstract
Style transfer, a pivotal task in image processing, synthesizes visually compelling images by seamlessly blending realistic content with artistic styles, enabling applications in photo editing and creative design. While mainstream training-free diffusion-based methods have greatly advanced style transfer in recent years, their reliance on computationally inversion processes compromises efficiency and introduces visual distortions when inversion is inaccurate. To address these limitations, we propose a novel \textit{inversion-free} style transfer framework based on dual rectified flows, which tackles the challenge of finding an unknown stylized distribution from two distinct inputs (content and style images), \textit{only with forward pass}. Our approach predicts content and style trajectories in parallel, then fuses them through a dynamic midpoint interpolation that integrates velocities from both paths while adapting to the evolving stylized image. By jointly modeling the content, style, and stylized distributions, our velocity field design achieves robust fusion and avoids the shortcomings of naive overlays. Attention injection further guides style integration, enhancing visual fidelity, content preservation, and computational efficiency. Extensive experiments demonstrate generalization across diverse styles and content, providing an effective and efficient pipeline for style transfer.
Abstract (translated)
URL
https://arxiv.org/abs/2511.20986
PDF
-
ScriptViT: Vision Transformer-Based Personalized Handwriting Generation
2025-11-23 06:38:23Sajjan Acharya, Rajendra BaskotaAbstract
Styled handwriting generation aims to synthesize handwritten text that looks both realistic and aligned with a specific writer's style. While recent approaches involving GAN, transformer and diffusion-based models have made progress, they often struggle to capture the full spectrum of writer-specific attributes, particularly global stylistic patterns that span long-range spatial dependencies. As a result, capturing subtle writer-specific traits such as consistent slant, curvature or stroke pressure, while keeping the generated text accurate is still an open problem. In this work, we present a unified framework designed to address these limitations. We introduce a Vision Transformer-based style encoder that learns global stylistic patterns from multiple reference images, allowing the model to better represent long-range structural characteristics of handwriting. We then integrate these style cues with the target text using a cross-attention mechanism, enabling the system to produce handwritten images that more faithfully reflect the intended style. To make the process more interpretable, we utilize Salient Stroke Attention Analysis (SSAA), which reveals the stroke-level features the model focuses on during style transfer. Together, these components lead to handwriting synthesis that is not only more stylistically coherent, but also easier to understand and analyze.
Abstract (translated)
URL
https://arxiv.org/abs/2511.18307
PDF
-
UI-Styler: Ultrasound Image Style Transfer with Class-Aware Prompts for Cross-Device Diagnosis Using a Frozen Black-Box Inference Network
2025-11-21 11:19:57Nhat-Tuong Do-Tran, Ngoc-Hoang-Lam Le, Ching-Chun HuangAbstract
The appearance of ultrasound images varies across acquisition devices, causing domain shifts that degrade the performance of fixed black-box downstream inference models when reused. To mitigate this issue, it is practical to develop unpaired image translation (UIT) methods that effectively align the statistical distributions between source and target domains, particularly under the constraint of a reused inference-blackbox setting. However, existing UIT approaches often overlook class-specific semantic alignment during domain adaptation, resulting in misaligned content-class mappings that can impair diagnostic accuracy. To address this limitation, we propose UI-Styler, a novel ultrasound-specific, class-aware image style transfer framework. UI-Styler leverages a pattern-matching mechanism to transfer texture patterns embedded in the target images onto source images while preserving the source structural content. In addition, we introduce a class-aware prompting strategy guided by pseudo labels of the target domain, which enforces accurate semantic alignment with diagnostic categories. Extensive experiments on ultrasound cross-device tasks demonstrate that UI-Styler consistently outperforms existing UIT methods, achieving state-of-the-art performance in distribution distance and downstream tasks, such as classification and segmentation.
Abstract (translated)
URL
https://arxiv.org/abs/2511.17155
PDF
-
Device-Guided Music Transfer
2025-11-21 10:57:11Manh Pham Hung, Changshuo Hu, Ting Dang, Dong MaAbstract
Device-guided music transfer adapts playback across unseen devices for users who lack them. Existing methods mainly focus on modifying the timbre, rhythm, harmony, or instrumentation to mimic genres or artists, overlooking the diverse hardware properties of the playback device (i.e., speaker). Therefore, we propose DeMT, which processes a speaker's frequency response curve as a line graph using a vision-language model to extract device embeddings. These embeddings then condition a hybrid transformer via feature-wise linear modulation. Fine-tuned on a self-collected dataset, DeMT enables effective speaker-style transfer and robust few-shot adaptation for unseen devices, supporting applications like device-style augmentation and quality enhancement.
Abstract (translated)
URL
https://arxiv.org/abs/2511.17136
PDF
-
Voiced-Aware Style Extraction and Style Direction Adjustment for Expressive Text-to-Speech
Abstract
Recent advances in expressive text-to-speech (TTS) have introduced diverse methods based on style embedding extracted from reference speech. However, synthesizing high-quality expressive speech remains challenging. We propose SpotlightTTS, which exclusively emphasizes style via voiced-aware style extraction and style direction adjustment. Voiced-aware style extraction focuses on voiced regions highly related to style while maintaining continuity across different speech regions to improve expressiveness. We adjust the direction of the extracted style for optimal integration into the TTS model, which improves speech quality. Experimental results demonstrate that Spotlight-TTS achieves superior performance compared to baseline models in terms of expressiveness, overall speech quality, and style transfer capability.
Abstract (translated)
URL
https://arxiv.org/abs/2511.14824
PDF
-
Consistency Change Detection Framework for Unsupervised Remote Sensing Change Detection
2025-11-12 02:28:02Yating Liu, Yan LuAbstract
Unsupervised remote sensing change detection aims to monitor and analyze changes from multi-temporal remote sensing images in the same geometric region at different times, without the need for labeled training data. Previous unsupervised methods attempt to achieve style transfer across multi-temporal remote sensing images through reconstruction by a generator network, and then capture the unreconstructable areas as the changed regions. However, it often leads to poor performance due to generator overfitting. In this paper, we propose a novel Consistency Change Detection Framework (CCDF) to address this challenge. Specifically, we introduce a Cycle Consistency (CC) module to reduce the overfitting issues in the generator-based reconstruction. Additionally, we propose a Semantic Consistency (SC) module to enable detail reconstruction. Extensive experiments demonstrate that our method outperforms other state-of-the-art approaches.
Abstract (translated)
URL
https://arxiv.org/abs/2511.08904
PDF
-
AnoStyler: Text-Driven Localized Anomaly Generation via Lightweight Style Transfer
Abstract
Anomaly generation has been widely explored to address the scarcity of anomaly images in real-world data. However, existing methods typically suffer from at least one of the following limitations, hindering their practical deployment: (1) lack of visual realism in generated anomalies; (2) dependence on large amounts of real images; and (3) use of memory-intensive, heavyweight model architectures. To overcome these limitations, we propose AnoStyler, a lightweight yet effective method that frames zero-shot anomaly generation as text-guided style transfer. Given a single normal image along with its category label and expected defect type, an anomaly mask indicating the localized anomaly regions and two-class text prompts representing the normal and anomaly states are generated using generalizable category-agnostic procedures. A lightweight U-Net model trained with CLIP-based loss functions is used to stylize the normal image into a visually realistic anomaly image, where anomalies are localized by the anomaly mask and semantically aligned with the text prompts. Extensive experiments on the MVTec-AD and VisA datasets show that AnoStyler outperforms existing anomaly generation methods in generating high-quality and diverse anomaly images. Furthermore, using these generated anomalies helps enhance anomaly detection performance.
Abstract (translated)
URL
https://arxiv.org/abs/2511.06687
PDF
-
V-Shuffle: Zero-Shot Style Transfer via Value Shuffle
2025-11-09 13:07:23Haojun Tang, Qiwei Lin, Tongda Xu, Lida Huang, Yan WangAbstract
Attention injection-based style transfer has achieved remarkable progress in recent years. However, existing methods often suffer from content leakage, where the undesired semantic content of the style image mistakenly appears in the stylized output. In this paper, we propose V-Shuffle, a zero-shot style transfer method that leverages multiple style images from the same style domain to effectively navigate the trade-off between content preservation and style fidelity. V-Shuffle implicitly disrupts the semantic content of the style images by shuffling the value features within the self-attention layers of the diffusion model, thereby preserving low-level style representations. We further introduce a Hybrid Style Regularization that complements these low-level representations with high-level style textures to enhance style fidelity. Empirical results demonstrate that V-Shuffle achieves excellent performance when utilizing multiple style images. Moreover, when applied to a single style image, V-Shuffle outperforms previous state-of-the-art methods.
Abstract (translated)
URL
https://arxiv.org/abs/2511.06365
PDF
-
AStF: Motion Style Transfer via Adaptive Statistics Fusor
2025-11-06 08:51:24Hanmo Chen, Chenghao Xu, Jiexi Yan, Cheng DengAbstract
Human motion style transfer allows characters to appear less rigidity and more realism with specific style. Traditional arbitrary image style transfer typically process mean and variance which is proved effective. Meanwhile, similar methods have been adapted for motion style transfer. However, due to the fundamental differences between images and motion, relying on mean and variance is insufficient to fully capture the complex dynamic patterns and spatiotemporal coherence properties of motion data. Building upon this, our key insight is to bring two more coefficient, skewness and kurtosis, into the analysis of motion style. Specifically, we propose a novel Adaptive Statistics Fusor (AStF) which consists of Style Disentanglement Module (SDM) and High-Order Multi-Statistics Attention (HOS-Attn). We trained our AStF in conjunction with a Motion Consistency Regularization (MCR) discriminator. Experimental results show that, by providing a more comprehensive model of the spatiotemporal statistical patterns inherent in dynamic styles, our proposed AStF shows proficiency superiority in motion style transfers over state-of-the-arts. Our code and model are available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2511.04192
PDF
-
Text to Sketch Generation with Multi-Styles
2025-11-06 07:13:56Tengjie Li, Shikui Tu, Lei XuAbstract
Recent advances in vision-language models have facilitated progress in sketch generation. However, existing specialized methods primarily focus on generic synthesis and lack mechanisms for precise control over sketch styles. In this work, we propose a training-free framework based on diffusion models that enables explicit style guidance via textual prompts and referenced style sketches. Unlike previous style transfer methods that overwrite key and value matrices in self-attention, we incorporate the reference features as auxiliary information with linear smoothing and leverage a style-content guidance mechanism. This design effectively reduces content leakage from reference sketches and enhances synthesis quality, especially in cases with low structural similarity between reference and target sketches. Furthermore, we extend our framework to support controllable multi-style generation by integrating features from multiple reference sketches, coordinated via a joint AdaIN module. Extensive experiments demonstrate that our approach achieves high-quality sketch generation with accurate style alignment and improved flexibility in style control. The official implementation of M3S is available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2511.04123
PDF
-
Cross-fluctuation phase transitions reveal sampling dynamics in diffusion models
2025-10-31 09:40:59Sai Niranjan Ramachandran, Manish Krishan Lal, Suvrit SraAbstract
We analyse how the sampling dynamics of distributions evolve in score-based diffusion models using cross-fluctuations, a centered-moment statistic from statistical physics. Specifically, we show that starting from an unbiased isotropic normal distribution, samples undergo sharp, discrete transitions, eventually forming distinct events of a desired distribution while progressively revealing finer structure. As this process is reversible, these transitions also occur in reverse, where intermediate states progressively merge, tracing a path back to the initial distribution. We demonstrate that these transitions can be detected as discontinuities in $n^{\text{th}}$-order cross-fluctuations. For variance-preserving SDEs, we derive a closed-form for these cross-fluctuations that is efficiently computable for the reverse trajectory. We find that detecting these transitions directly boosts sampling efficiency, accelerates class-conditional and rare-class generation, and improves two zero-shot tasks--image classification and style transfer--without expensive grid search or retraining. We also show that this viewpoint unifies classical coupling and mixing from finite Markov chains with continuous dynamics while extending to stochastic SDEs and non Markovian samplers. Our framework therefore bridges discrete Markov chain theory, phase analysis, and modern generative modeling.
Abstract (translated)
URL
https://arxiv.org/abs/2511.00124
PDF
-
Security Risk of Misalignment between Text and Image in Multi-modal Model
2025-10-30 03:31:20Xiaosen Wang, Zhijin Ge, Shaokang WangAbstract
Despite the notable advancements and versatility of multi-modal diffusion models, such as text-to-image models, their susceptibility to adversarial inputs remains underexplored. Contrary to expectations, our investigations reveal that the alignment between textual and Image modalities in existing diffusion models is inadequate. This misalignment presents significant risks, especially in the generation of inappropriate or Not-Safe-For-Work (NSFW) content. To this end, we propose a novel attack called Prompt-Restricted Multi-modal Attack (PReMA) to manipulate the generated content by modifying the input image in conjunction with any specified prompt, without altering the prompt itself. PReMA is the first attack that manipulates model outputs by solely creating adversarial images, distinguishing itself from prior methods that primarily generate adversarial prompts to produce NSFW content. Consequently, PReMA poses a novel threat to the integrity of multi-modal diffusion models, particularly in image-editing applications that operate with fixed prompts. Comprehensive evaluations conducted on image inpainting and style transfer tasks across various models confirm the potent efficacy of PReMA.
Abstract (translated)
URL
https://arxiv.org/abs/2510.26105
PDF
-
GeoDiffusion: A Training-Free Framework for Accurate 3D Geometric Conditioning in Image Generation
2025-10-25 15:40:34Phillip Mueller, Talip Uenlue, Sebastian Schmidt, Marcel Kollovieh, Jiajie Fan, Stephan Guennemann, Lars MikelsonsAbstract
Precise geometric control in image generation is essential for engineering \& product design and creative industries to control 3D object features accurately in image space. Traditional 3D editing approaches are time-consuming and demand specialized skills, while current image-based generative methods lack accuracy in geometric conditioning. To address these challenges, we propose GeoDiffusion, a training-free framework for accurate and efficient geometric conditioning of 3D features in image generation. GeoDiffusion employs a class-specific 3D object as a geometric prior to define keypoints and parametric correlations in 3D space. We ensure viewpoint consistency through a rendered image of a reference 3D object, followed by style transfer to meet user-defined appearance specifications. At the core of our framework is GeoDrag, improving accuracy and speed of drag-based image editing on geometry guidance tasks and general instructions on DragBench. Our results demonstrate that GeoDiffusion enables precise geometric modifications across various iterative design workflows.
Abstract (translated)
URL
https://arxiv.org/abs/2510.22337
PDF
