Style_Transfer
-
Style4D-Bench: A Benchmark Suite for 4D Stylization
2025-08-26 17:59:17Beiqi Chen, Shuai Shao, Haitang Feng, Jianhuang Lai, Jianlou Si, Guangcong WangAbstract
We introduce Style4D-Bench, the first benchmark suite specifically designed for 4D stylization, with the goal of standardizing evaluation and facilitating progress in this emerging area. Style4D-Bench comprises: 1) a comprehensive evaluation protocol measuring spatial fidelity, temporal coherence, and multi-view consistency through both perceptual and quantitative metrics, 2) a strong baseline that make an initial attempt for 4D stylization, and 3) a curated collection of high-resolution dynamic 4D scenes with diverse motions and complex backgrounds. To establish a strong baseline, we present Style4D, a novel framework built upon 4D Gaussian Splatting. It consists of three key components: a basic 4DGS scene representation to capture reliable geometry, a Style Gaussian Representation that leverages lightweight per-Gaussian MLPs for temporally and spatially aware appearance control, and a Holistic Geometry-Preserved Style Transfer module designed to enhance spatio-temporal consistency via contrastive coherence learning and structural content preservation. Extensive experiments on Style4D-Bench demonstrate that Style4D achieves state-of-the-art performance in 4D stylization, producing fine-grained stylistic details with stable temporal dynamics and consistent multi-view rendering. We expect Style4D-Bench to become a valuable resource for benchmarking and advancing research in stylized rendering of dynamic 3D scenes. Project page: this https URL . Code: this https URL .
Abstract (translated)
我们介绍了Style4D-Bench，这是首个专门为4D风格化设计的基准测试套件，旨在标准化评估并促进这一新兴领域的进步。Style4D-Bench包括： 1）一个全面的评估协议，通过感知和定量指标衡量空间保真度、时间连贯性和多视角一致性。 2）一个强大的基线模型，首次尝试进行4D风格化。 3）一组经过精心策划的高分辨率动态4D场景集合，包含多样化的运动和复杂背景。 为了建立一个强大的基线，我们提出了Style4D，这是一个基于4D高斯点阵的新框架。它包括三个关键组成部分：基本的4DGS场景表示，用于捕获可靠的几何信息；风格高斯表示，利用轻量级的每高斯MLP实现时间感知和空间感知的外观控制；以及一个整体保持几何结构的风格迁移模块，通过对比连贯性学习和结构性内容保留来增强时空一致性。 在Style4D-Bench上的广泛实验表明，Style4D在4D风格化中实现了最先进的性能，能够产生具有稳定时间动态和一致多视角渲染的细腻风格细节。我们期望Style4D-Bench将成为评估和推进动态3D场景风格化研究的重要资源。 项目页面：[此处插入URL] 代码：[此处插入URL]
URL
https://arxiv.org/abs/2508.19243
PDF
-
ClearMask: Noise-Free and Naturalness-Preserving Protection Against Voice Deepfake Attacks
2025-08-25 04:46:35Yuanda Wang, Bocheng Chen, Hanqing Guo, Guangjing Wang, Weikang Ding, Qiben YanAbstract
Voice deepfake attacks, which artificially impersonate human speech for malicious purposes, have emerged as a severe threat. Existing defenses typically inject noise into human speech to compromise voice encoders in speech synthesis models. However, these methods degrade audio quality and require prior knowledge of the attack approaches, limiting their effectiveness in diverse scenarios. Moreover, real-time audios, such as speech in virtual meetings and voice messages, are still exposed to voice deepfake threats. To overcome these limitations, we propose ClearMask, a noise-free defense mechanism against voice deepfake attacks. Unlike traditional approaches, ClearMask modifies the audio mel-spectrogram by selectively filtering certain frequencies, inducing a transferable voice feature loss without injecting noise. We then apply audio style transfer to further deceive voice decoders while preserving perceived sound quality. Finally, optimized reverberation is introduced to disrupt the output of voice generation models without affecting the naturalness of the speech. Additionally, we develop LiveMask to protect streaming speech in real-time through a universal frequency filter and reverberation generator. Our experimental results show that ClearMask and LiveMask effectively prevent voice deepfake attacks from deceiving speaker verification models and human listeners, even for unseen voice synthesis models and black-box API services. Furthermore, ClearMask demonstrates resilience against adaptive attackers who attempt to recover the original audio signal from the protected speech samples.
Abstract (translated)
声纹深度伪造攻击，这种利用人工智能技术模仿人类语音以达到恶意目的的行为，已经成为了严重的威胁。现有的防御措施通常会在人的声音中加入噪声来破坏语音合成模型中的语音编码器。然而，这些方法会降低音频质量，并且需要事先了解攻击的手段，这在多变的情景下限制了它们的有效性。此外，在线实时音频（如虚拟会议和语音消息）仍然容易受到声纹深度伪造攻击。 为克服以上局限性，我们提出了ClearMask这一种无噪音防御机制来对抗声纹深度伪造攻击。与传统的做法不同，ClearMask通过选择性地过滤某些频率来修改音频的梅尔频谱图，从而在不注入噪声的情况下诱导出可转移的语音特征损失。之后，我们应用了音频风格迁移技术进一步欺骗语音解码器同时保持声音的质量不受影响。最后，引入优化后的混响处理以干扰声纹生成模型输出的结果而不影响自然说话的声音特性。 此外，为了保护实时流媒体中的讲话内容，我们开发了一种名为LiveMask的方法，通过通用频率滤波和回音发生器来实现这一目标。 实验结果表明，ClearMask和LiveMask在面对未知的语音合成模型以及黑盒API服务时，都能有效阻止声纹深度伪造攻击欺骗说话人验证模型及人类听者。此外，ClearMask还展现出了对那些试图从保护后的音频样本中恢复出原始信号的自适应攻击者的抗性。
URL
https://arxiv.org/abs/2508.17660
PDF
-
MahaParaphrase: A Marathi Paraphrase Detection Corpus and BERT-based Models
2025-08-24 16:48:58Suramya Jadhav, Abhay Shanbhag, Amogh Thakurdesai, Ridhima Sinare, Ananya Joshi, Raviraj JoshiAbstract
Paraphrases are a vital tool to assist language understanding tasks such as question answering, style transfer, semantic parsing, and data augmentation tasks. Indic languages are complex in natural language processing (NLP) due to their rich morphological and syntactic variations, diverse scripts, and limited availability of annotated data. In this work, we present the L3Cube-MahaParaphrase Dataset, a high-quality paraphrase corpus for Marathi, a low resource Indic language, consisting of 8,000 sentence pairs, each annotated by human experts as either Paraphrase (P) or Non-paraphrase (NP). We also present the results of standard transformer-based BERT models on these datasets. The dataset and model are publicly shared at this https URL
Abstract (translated)
paraphrases 是一种重要的工具，用于支持如问答、文体转换、语义解析以及数据增强等语言理解任务。由于其丰富的形态和句法变化、多样化的书写系统以及注释数据的有限性，印地语系语言在自然语言处理（NLP）中尤为复杂。在这项工作中，我们介绍了 L3Cube-MahaParaphrase 数据集，这是一个针对马拉地语（一种资源稀缺的印地语系语言）的高质量 paraphrase 语料库，包含8,000对句子，并由人工专家标注为“paraphrase”或“非 paraphrase”。此外，我们还展示了基于标准 transformer 的 BERT 模型在这些数据集上的表现。该数据集和模型已公开分享在此链接中（原链接用此URL代替）。
URL
https://arxiv.org/abs/2508.17444
PDF
-
RephraseTTS: Dynamic Length Text based Speech Insertion with Speaker Style Transfer
2025-08-23 14:12:49Neeraj Matiyali, Siddharth Srivastava, Gaurav SharmaAbstract
We propose a method for the task of text-conditioned speech insertion, i.e. inserting a speech sample in an input speech sample, conditioned on the corresponding complete text transcript. An example use case of the task would be to update the speech audio when corrections are done on the corresponding text transcript. The proposed method follows a transformer-based non-autoregressive approach that allows speech insertions of variable lengths, which are dynamically determined during inference, based on the text transcript and tempo of the available partial input. It is capable of maintaining the speaker's voice characteristics, prosody and other spectral properties of the available speech input. Results from our experiments and user study on LibriTTS show that our method outperforms baselines based on an existing adaptive text to speech method. We also provide numerous qualitative results to appreciate the quality of the output from the proposed method.
Abstract (translated)
我们提出了一种基于文本条件的语音插入方法，即在输入语音样本中插入一段新的语音样本，该操作依据对应的完整文本转录本进行。一个实际的应用场景是当对相应的文本转录本进行了修改时，更新音频中的相应部分。 所提出的这种方法采用了一个基于Transformer的非自回归（non-autoregressive）框架，允许根据文本转录本和现有不完整输入片段的速度，在推理过程中动态确定不同长度的语音插入。该方法能够保持说话人的音色、语调以及其他频谱属性，确保输出语音与原始语音样本的高度一致性。 我们通过在LibriTTS数据集上的实验及用户研究证明了所提出的方法优于基于现有自适应文本到语音转换技术的基线模型。此外，我们也提供了大量定性结果以展示该方法生成输出的质量。
URL
https://arxiv.org/abs/2508.17031
PDF
-
Retrieval-Augmented Review Generation for Poisoning Recommender Systems
2025-08-21 05:25:22Shiyi Yang, Xinshu Li, Guanglin Zhou, Chen Wang, Xiwei Xu, Liming Zhu, Lina YaoAbstract
Recent studies have shown that recommender systems (RSs) are highly vulnerable to data poisoning attacks, where malicious actors inject fake user profiles, including a group of well-designed fake ratings, to manipulate recommendations. Due to security and privacy constraints in practice, attackers typically possess limited knowledge of the victim system and thus need to craft profiles that have transferability across black-box RSs. To maximize the attack impact, the profiles often remains imperceptible. However, generating such high-quality profiles with the restricted resources is challenging. Some works suggest incorporating fake textual reviews to strengthen the profiles; yet, the poor quality of the reviews largely undermines the attack effectiveness and imperceptibility under the practical setting. To tackle the above challenges, in this paper, we propose to enhance the quality of the review text by harnessing in-context learning (ICL) capabilities of multimodal foundation models. To this end, we introduce a demonstration retrieval algorithm and a text style transfer strategy to augment the navie ICL. Specifically, we propose a novel practical attack framework named RAGAN to generate high-quality fake user profiles, which can gain insights into the robustness of RSs. The profiles are generated by a jailbreaker and collaboratively optimized on an instructional agent and a guardian to improve the attack transferability and imperceptibility. Comprehensive experiments on various real-world datasets demonstrate that RAGAN achieves the state-of-the-art poisoning attack performance.
Abstract (translated)
最近的研究表明，推荐系统（RS）极易受到数据投毒攻击的影响。在这种攻击中，恶意行为者会注入虚假用户资料，包括一组精心设计的假评分，以操纵推荐结果。由于实际操作中的安全性和隐私限制，攻击者通常对受害系统的了解有限，因此需要创建可以在黑盒推荐系统中通用的虚假用户档案。为了最大化攻击影响，这些虚假用户档案往往必须保持隐蔽性。然而，在资源受限的情况下生成高质量的虚假用户资料极具挑战性。一些研究建议通过加入虚假文本评论来增强这些资料的质量；但是，在实际设置下，这种低质量的评论会严重削弱攻击的有效性和隐蔽性。 为了解决上述挑战，本文提出利用多模态基础模型的上下文学习（ICL）能力来提高评论文本的质量。为此，我们引入了一个演示检索算法和一个文本风格转换策略来增强简单的ICL方法。具体来说，我们提出了一个新的实用攻击框架——RAGAN，用于生成高质量的虚假用户资料，以评估推荐系统的鲁棒性。这些假用户档案通过破狱者（jailbreaker）生成，并由指导代理（instructional agent）和守护者（guardian）共同优化，以此来提高攻击的通用性和隐蔽性。 在各种真实数据集上的全面实验表明，RAGAN实现了最先进的投毒攻击性能。
URL
https://arxiv.org/abs/2508.15252
PDF
-
MeSS: City Mesh-Guided Outdoor Scene Generation with Cross-View Consistent Diffusion
2025-08-21 02:16:15Xuyang Chen, Zhijun Zhai, Kaixuan Zhou, Zengmao Wang, Jianan He, Dong Wang, Yanfeng Zhang, mingwei Sun, R\"udiger Westermann, Konrad Schindler, Liqiu MengAbstract
Mesh models have become increasingly accessible for numerous cities; however, the lack of realistic textures restricts their application in virtual urban navigation and autonomous driving. To address this, this paper proposes MeSS (Meshbased Scene Synthesis) for generating high-quality, styleconsistent outdoor scenes with city mesh models serving as the geometric prior. While image and video diffusion models can leverage spatial layouts (such as depth maps or HD maps) as control conditions to generate street-level perspective views, they are not directly applicable to 3D scene generation. Video diffusion models excel at synthesizing consistent view sequences that depict scenes but often struggle to adhere to predefined camera paths or align accurately with rendered control videos. In contrast, image diffusion models, though unable to guarantee cross-view visual consistency, can produce more geometry-aligned results when combined with ControlNet. Building on this insight, our approach enhances image diffusion models by improving cross-view consistency. The pipeline comprises three key stages: first, we generate geometrically consistent sparse views using Cascaded Outpainting ControlNets; second, we propagate denser intermediate views via a component dubbed AGInpaint; and third, we globally eliminate visual inconsistencies (e.g., varying exposure) using the GCAlign module. Concurrently with generation, a 3D Gaussian Splatting (3DGS) scene is reconstructed by initializing Gaussian balls on the mesh surface. Our method outperforms existing approaches in both geometric alignment and generation quality. Once synthesized, the scene can be rendered in diverse styles through relighting and style transfer techniques.
Abstract (translated)
网格模型（Mesh models）对于许多城市来说已经变得越来越容易获取；然而，缺乏逼真的纹理限制了它们在虚拟城市导航和自动驾驶中的应用。为了应对这一问题，本文提出了 MeSS (基于网格的场景合成)，旨在利用城市的网格模型作为几何先验条件来生成高质量、风格一致的户外场景。虽然图像和视频扩散模型可以借助空间布局（如深度图或高清地图）作为控制条件来生成街道视角视图，但它们并不直接适用于3D场景生成。视频扩散模型擅长于合成描绘场景的一致性视图序列，但在遵循预定义的相机路径或与渲染的控制视频对齐方面常常遇到困难。相比之下，尽管图像扩散模型无法保证跨视角视觉一致性，但如果结合使用ControlNet，则可以产生更具几何一致性的结果。基于这一洞察，我们的方法通过改进图像扩散模型之间的视角一致性来增强它们的功能。 该流程包含三个关键阶段：首先，我们利用级联Outpainting ControlNets生成几何上一致的稀疏视图；其次，我们将更密集的中间视图通过名为AGInpaint的组件进行传播；第三，我们使用GCAlign模块全局消除视觉不一致性（例如不同的曝光度）。与此同时，在生成过程中，通过在网格表面上初始化高斯球来重建3D高斯点阵（3DGS）场景。我们的方法在几何对齐和生成质量方面都优于现有的方法。一旦合成完成，可以通过重新照明和风格转移技术以多种样式渲染场景。 这种综合方法不仅提升了基于图像扩散模型的三维场景生成能力，还为未来在虚拟城市导航和自动驾驶领域中的应用铺平了道路，使得利用现有网格数据创建逼真的虚拟环境成为可能。
URL
https://arxiv.org/abs/2508.15169
PDF
-
Sketch3DVE: Sketch-based 3D-Aware Scene Video Editing
2025-08-19 12:57:31Feng-Lin Liu, Shi-Yang Li, Yan-Pei Cao, Hongbo Fu, Lin GaoAbstract
Recent video editing methods achieve attractive results in style transfer or appearance modification. However, editing the structural content of 3D scenes in videos remains challenging, particularly when dealing with significant viewpoint changes, such as large camera rotations or zooms. Key challenges include generating novel view content that remains consistent with the original video, preserving unedited regions, and translating sparse 2D inputs into realistic 3D video outputs. To address these issues, we propose Sketch3DVE, a sketch-based 3D-aware video editing method to enable detailed local manipulation of videos with significant viewpoint changes. To solve the challenge posed by sparse inputs, we employ image editing methods to generate edited results for the first frame, which are then propagated to the remaining frames of the video. We utilize sketching as an interaction tool for precise geometry control, while other mask-based image editing methods are also supported. To handle viewpoint changes, we perform a detailed analysis and manipulation of the 3D information in the video. Specifically, we utilize a dense stereo method to estimate a point cloud and the camera parameters of the input video. We then propose a point cloud editing approach that uses depth maps to represent the 3D geometry of newly edited components, aligning them effectively with the original 3D scene. To seamlessly merge the newly edited content with the original video while preserving the features of unedited regions, we introduce a 3D-aware mask propagation strategy and employ a video diffusion model to produce realistic edited videos. Extensive experiments demonstrate the superiority of Sketch3DVE in video editing. Homepage and code: http://http://geometrylearning.com/Sketch3DVE/
Abstract (translated)
最近的视频编辑方法在风格转换或外观修改方面取得了引人注目的成果。然而，对具有显著视角变化（如大角度旋转或变焦）的三维场景结构内容进行编辑仍然存在挑战性。关键问题包括生成与原始视频一致的新视图内容、保持未编辑区域不变以及将稀疏的2D输入转换为真实的3D视频输出。为了应对这些问题，我们提出了Sketch3DVE，这是一种基于草图的3D感知视频编辑方法，旨在实现具有显著视角变化的视频的详细局部操作。 针对由稀疏输入带来的挑战，我们利用图像编辑技术生成第一个帧的编辑结果，并将其传播到视频中的其余帧中。我们使用绘图作为精确几何控制的交互工具，同时支持其他基于蒙版的图像编辑方法。为了处理视角的变化，我们在视频中对3D信息进行了详细的分析和操作。具体而言，我们采用了一种密集立体法来估计输入视频的点云及其摄像机参数。接着，我们提出了一种点云编辑的方法，利用深度图表示新编辑组件的3D几何结构，并将其有效地与原始场景对齐。 为了无缝融合新的编辑内容并保持未修改区域的特征，我们引入了基于3D感知的蒙版传播策略，并使用视频扩散模型生成逼真的编辑视频。大量实验表明Sketch3DVE在视频编辑方面的优越性。官方网站和代码：http://geometrylearning.com/Sketch3DVE/
URL
https://arxiv.org/abs/2508.13797
PDF
-
Leveraging Diffusion Models for Stylization using Multiple Style Images
2025-08-18 10:00:41Dan Ruta, Abdelaziz Djelouah, Raphael Ortiz, Christopher SchroersAbstract
Recent advances in latent diffusion models have enabled exciting progress in image style transfer. However, several key issues remain. For example, existing methods still struggle to accurately match styles. They are often limited in the number of style images that can be used. Furthermore, they tend to entangle content and style in undesired ways. To address this, we propose leveraging multiple style images which helps better represent style features and prevent content leaking from the style images. We design a method that leverages both image prompt adapters and statistical alignment of the features during the denoising process. With this, our approach is designed such that it can intervene both at the cross-attention and the self-attention layers of the denoising UNet. For the statistical alignment, we employ clustering to distill a small representative set of attention features from the large number of attention values extracted from the style samples. As demonstrated in our experimental section, the resulting method achieves state-of-the-art results for stylization.
Abstract (translated)
最近在潜在扩散模型方面的进展已经在图像风格转换领域取得了令人兴奋的成果。然而，仍存在一些关键问题亟待解决。例如，现有的方法仍然难以准确匹配样式特征，并且通常局限于使用有限数量的样例图片。此外，它们倾向于以非预期的方式纠缠内容和样式信息。为了解决这些问题，我们提出了一种利用多张风格图像的方法，这有助于更好地表示风格特性并防止从风格图像中泄露内容。为此，我们设计了一种方法，该方法结合了图像提示适配器以及在去噪过程中对特征的统计一致性处理。 我们的方法被特别设计为可以在扩散UNet中的交叉注意力层和自我注意力层进行干预。对于统计一致性处理部分，我们采用了聚类技术来从大量的样例图片中提取的关注值中提炼出一小批具有代表性的关注特性。实验结果表明，所提出的方法在风格化任务上取得了当前最先进的性能表现。
URL
https://arxiv.org/abs/2508.12784
PDF
-
StyleMM: Stylized 3D Morphable Face Model via Text-Driven Aligned Image Translation
2025-08-15 04:29:46Seungmi Lee, Kwan Yun, Junyong NohAbstract
We introduce StyleMM, a novel framework that can construct a stylized 3D Morphable Model (3DMM) based on user-defined text descriptions specifying a target style. Building upon a pre-trained mesh deformation network and a texture generator for original 3DMM-based realistic human faces, our approach fine-tunes these models using stylized facial images generated via text-guided image-to-image (i2i) translation with a diffusion model, which serve as stylization targets for the rendered mesh. To prevent undesired changes in identity, facial alignment, or expressions during i2i translation, we introduce a stylization method that explicitly preserves the facial attributes of the source image. By maintaining these critical attributes during image stylization, the proposed approach ensures consistent 3D style transfer across the 3DMM parameter space through image-based training. Once trained, StyleMM enables feed-forward generation of stylized face meshes with explicit control over shape, expression, and texture parameters, producing meshes with consistent vertex connectivity and animatability. Quantitative and qualitative evaluations demonstrate that our approach outperforms state-of-the-art methods in terms of identity-level facial diversity and stylization capability. The code and videos are available at [this http URL](this http URL).
Abstract (translated)
我们介绍了一种新颖的框架StyleMM，它可以基于用户定义的文字描述构建具有特定目标风格的个性化3D可变模型（3DMM）。此方法在预训练的网格变形网络和用于原始3DMM真实感人脸的纹理生成器基础上进行微调，并使用文本引导的图像到图像（i2i）转换的扩散模型来生成风格化的面部图像。这些风格化后的图像作为渲染网格的风格化目标，以构建基于用户定义文字描述的个性化3D可变模型。 为了防止在i2i翻译过程中身份、面部对齐或表情的变化不受控制，我们引入了一种可以显式保留源图关键属性的风格化方法。通过在图像风格化过程中保持这些重要属性，所提出的方法确保了基于图像训练的一致性3D风格转换跨越整个3DMM参数空间。 一旦完成训练，StyleMM能够进行前馈生成具有明确形状、表情和纹理参数控制的风格化面部网格，并且这些网格具备一致的顶点连接性和可动画性。定量和定性的评估表明，在身份层面的面部多样性及风格化能力方面，我们的方法超过了当前最先进的技术。 相关代码和视频可在[此链接](this http URL)获取。
URL
https://arxiv.org/abs/2508.11203
PDF
-
Improving Text Style Transfer using Masked Diffusion Language Models with Inference-time Scaling
2025-08-14 18:01:22Tejomay Kishor Padole, Suyash P Awate, Pushpak BhattacharyyaAbstract
Masked diffusion language models (MDMs) have recently gained traction as a viable generative framework for natural language. This can be attributed to its scalability and ease of training compared to other diffusion model paradigms for discrete data, establishing itself as the state-of-the-art non-autoregressive generator for discrete data. Diffusion models, in general, have shown excellent ability to improve the generation quality by leveraging inference-time scaling either by increasing the number of denoising steps or by using external verifiers on top of the outputs of each step to guide the generation. In this work, we propose a verifier-based inference-time scaling method that aids in finding a better candidate generation during the denoising process of the MDM. Our experiments demonstrate the application of MDMs for standard text-style transfer tasks and establish MDMs as a better alternative to autoregressive language models. Additionally, we show that a simple soft-value-based verifier setup for MDMs using off-the-shelf pre-trained embedding models leads to significant gains in generation quality even when used on top of typical classifier-free guidance setups in the existing literature.
Abstract (translated)
最近，遮蔽扩散语言模型（MDMs）因其在自然语言生成框架中的可行性和可扩展性而受到关注。这可以归因于它们相较于其他离散数据的扩散模型范式具有更高的可训练性，并确立了其作为最先进的非自回归离散数据生成器的地位。总体而言，扩散模型通过增加去噪步骤的数量或利用每个步骤输出上的外部验证者来引导生成过程，在推断时间扩展方面表现出卓越的能力，从而提高了生成质量。 在这项工作中，我们提出了一种基于验证者的推理时扩展方法，该方法有助于在MDM的去噪过程中找到更好的候选生成。我们的实验展示了MDMs在标准文本风格转换任务中的应用，并证明了它们是自回归语言模型的一个更好替代方案。此外，我们还表明，通过使用现成的预训练嵌入模型为MDMs设置一个简单的基于软值的验证者架构，在现有的无分类器指导设定之上也能显著提高生成质量。 翻译总结如下： 最近，遮蔽扩散语言模型（MDM）因其在自然语言生成中的高效率和可扩展性而受到关注。MDM通过改进训练过程和其他离散数据扩散模型相比具有优势，并且被确立为最先进的非自回归离散数据生成器。一般而言，扩散模型通过增加去噪步骤的数量或使用外部验证者来指导每个步骤的输出，从而提高了生成质量。 本文提出了一种基于验证者的推理时扩展方法，在MDM的去噪过程中找到更好的候选生成。实验表明MDM在标准文本风格转换任务中的应用，并展示了它们比自回归语言模型更具优势。此外，通过使用现成的预训练嵌入模型为MDM设置一个简单的软值验证者架构，即使是在现有的无分类器指导设定之上，也能显著提高生成质量。
URL
https://arxiv.org/abs/2508.10995
PDF
-
Neural Artistic Style and Color Transfer Using Deep Learning
2025-08-12 03:42:03Justin LondonAbstract
Neural artistic style transfers and blends the content and style representation of one image with the style of another. This enables artists to create unique innovative visuals and enhances artistic expression in various fields including art, design, and film. Color transfer algorithms are an important in digital image processing by adjusting the color information in a target image based on the colors in the source image. Color transfer enhances images and videos in film and photography, and can aid in image correction. We introduce a methodology that combines neural artistic style with color transfer. The method uses the Kullback-Leibler (KL) divergence to quantitatively evaluate color and luminance histogram matching algorithms including Reinhard global color transfer, iteration distribution transfer (IDT), IDT with regrain, Cholesky, and PCA between the original and neural artistic style transferred image using deep learning. We estimate the color channel kernel densities. Various experiments are performed to evaluate the KL of these algorithms and their color histograms for style to content transfer.
Abstract (translated)
神经艺术风格转移将一幅图像的内容和风格表示与另一幅图像的风格相结合，这使艺术家能够创造出独特的创新视觉效果，并在包括艺术、设计和电影在内的各个领域增强艺术表达。颜色传输算法是数字图像处理中的重要组成部分，通过基于源图像的颜色来调整目标图像中的颜色信息。颜色传输可以提升影视作品的质量，在摄影和视频制作中起到关键作用，并有助于图像校正。我们介绍了一种结合神经艺术风格转移与色彩转换的方法论。该方法使用Kullback-Leibler (KL) 散度，利用深度学习技术对原始图像和经神经艺术风格处理后的图像之间的颜色和亮度直方图匹配算法进行定量评估，包括Reinhard全局颜色传输、迭代分布传输(IDT)、IDT结合重粒化（regrain）、Cholesky分解以及主成分分析(PCA)。我们估计了色彩通道的核密度。通过各种实验来评估这些算法及其在风格到内容转换过程中的KL散度和颜色直方图。 这种方法论提供了一种量化和比较不同色彩传输技术与神经艺术风格转移结合效果的新途径，从而有助于改进图像处理技术和增强视觉创作能力。
URL
https://arxiv.org/abs/2508.08608
PDF
-
FantasyStyle: Controllable Stylized Distillation for 3D Gaussian Splatting
2025-08-11 16:11:08Yitong Yang, Yinglin Wang, Changshuo Wang, Huajie Wang, Shuting HeAbstract
The success of 3DGS in generative and editing applications has sparked growing interest in 3DGS-based style transfer. However, current methods still face two major challenges: (1) multi-view inconsistency often leads to style conflicts, resulting in appearance smoothing and distortion; and (2) heavy reliance on VGG features, which struggle to disentangle style and content from style images, often causing content leakage and excessive stylization. To tackle these issues, we introduce \textbf{FantasyStyle}, a 3DGS-based style transfer framework, and the first to rely entirely on diffusion model distillation. It comprises two key components: (1) \textbf{Multi-View Frequency Consistency}. We enhance cross-view consistency by applying a 3D filter to multi-view noisy latent, selectively reducing low-frequency components to mitigate stylized prior conflicts. (2) \textbf{Controllable Stylized Distillation}. To suppress content leakage from style images, we introduce negative guidance to exclude undesired content. In addition, we identify the limitations of Score Distillation Sampling and Delta Denoising Score in 3D style transfer and remove the reconstruction term accordingly. Building on these insights, we propose a controllable stylized distillation that leverages negative guidance to more effectively optimize the 3D Gaussians. Extensive experiments demonstrate that our method consistently outperforms state-of-the-art approaches, achieving higher stylization quality and visual realism across various scenes and styles.
Abstract (translated)
3DGS在生成和编辑应用中的成功激发了人们对基于3DGS的风格迁移的兴趣日益增长。然而，目前的方法仍然面临两大挑战：（1）多视角不一致性常常导致风格冲突，从而出现外观平滑化和扭曲；（2）过度依赖VGG特征，这使得从风格图像中区分风格和内容变得困难，通常会导致内容泄露和过度风格化。为了解决这些问题，我们引入了**FantasyStyle**，这是一个基于3DGS的风格迁移框架，并且是第一个完全依赖于扩散模型蒸馏的方法。该框架包含两个关键组件： （1）**多视角频率一致性**：通过在多视角噪声潜在变量上应用三维滤波器来增强跨视图的一致性，选择性地减少低频成分以减轻风格化的先验冲突。 （2）**可控的风格化蒸馏**：为了抑制从样式图像中泄露内容的问题，我们引入了负向指导来排除不需要的内容。此外，我们还指出了3D风格迁移中评分蒸馏采样和Delta去噪评分的局限性，并相应地移除了重构项。基于这些洞察，我们提出了一个可控的风格化蒸馏方法，利用负向指导更有效地优化三维高斯分布。 大量的实验表明，我们的方法在各种场景和风格下始终优于最先进的方法，在风格质量和视觉逼真度方面表现更好。
URL
https://arxiv.org/abs/2508.08136
PDF
-
GANime: Generating Anime and Manga Character Drawings from Sketches with Deep Learning
2025-08-10 02:20:19Tai Vu, Robert YangAbstract
The process of generating fully colorized drawings from sketches is a large, usually costly bottleneck in the manga and anime industry. In this study, we examine multiple models for image-to-image translation between anime characters and their sketches, including Neural Style Transfer, C-GAN, and CycleGAN. By assessing them qualitatively and quantitatively, we find that C-GAN is the most effective model that is able to produce high-quality and high-resolution images close to those created by humans.
Abstract (translated)
从草图生成全彩绘制的过程在漫画和动画行业中是一个庞大且通常成本高昂的瓶颈。在这项研究中，我们考察了多种用于在动漫角色与其草图之间进行图像到图像转换的模型，包括神经风格迁移（Neural Style Transfer）、条件生成对抗网络（C-GAN）和循环GAN（CycleGAN）。通过定性和定量评估，我们发现C-GAN是最有效的模型，能够产生接近人类创作的高度质量和高分辨率的图像。
URL
https://arxiv.org/abs/2508.09207
PDF
-
Optimization-Free Style Transfer for 3D Gaussian Splats
2025-08-07 19:35:01Raphael Du Sablon, David HartAbstract
The task of style transfer for 3D Gaussian splats has been explored in many previous works, but these require reconstructing or fine-tuning the splat while incorporating style information or optimizing a feature extraction network on the splat representation. We propose a reconstruction- and optimization-free approach to stylizing 3D Gaussian splats. This is done by generating a graph structure across the implicit surface of the splat representation. A feed-forward, surface-based stylization method is then used and interpolated back to the individual splats in the scene. This allows for any style image and 3D Gaussian splat to be used without any additional training or optimization. This also allows for fast stylization of splats, achieving speeds under 2 minutes even on consumer-grade hardware. We demonstrate the quality results this approach achieves and compare to other 3D Gaussian splat style transfer methods. Code is publicly available at this https URL.
Abstract (translated)
在许多先前的工作中，已经探讨了将风格转换应用于3D高斯点云的任务，但这些方法通常需要重构或微调点云，并在此过程中结合样式信息或优化表示点云的特征提取网络。我们提出了一种无需重建和优化的方法来为3D高斯点云添加风格效果。这通过在点云表示的隐式表面上生成图结构来实现。然后使用一种基于表面的前馈风格化方法，将结果插值回场景中的各个点云。这种方法使得可以使用任何样式图像和3D高斯点云，而无需进行额外的训练或优化。此外，这还允许快速地对点云进行风格化处理，在消费级硬件上甚至可以在不到两分钟的时间内完成。我们展示了这种技术的质量结果，并将其与其他3D高斯点云风格转换方法进行了比较。该代码可在以下网址公开获取：[https URL]。
URL
https://arxiv.org/abs/2508.05813
PDF
-
Improving Masked Style Transfer using Blended Partial Convolution
2025-08-07 18:35:44Seyed Hadi Seyed, Ayberk Cansever, David HartAbstract
Artistic style transfer has long been possible with the advancements of convolution- and transformer-based neural networks. Most algorithms apply the artistic style transfer to the whole image, but individual users may only need to apply a style transfer to a specific region in the image. The standard practice is to simply mask the image after the stylization. This work shows that this approach tends to improperly capture the style features in the region of interest. We propose a partial-convolution-based style transfer network that accurately applies the style features exclusively to the region of interest. Additionally, we present network-internal blending techniques that account for imperfections in the region selection. We show that this visually and quantitatively improves stylization using examples from the SA-1B dataset. Code is publicly available at this https URL.
Abstract (translated)
基于卷积和转换器神经网络的发展，艺术风格转移已经可以实现很长时间了。大多数算法将艺术风格转移应用于整个图像，但个别用户可能只需要对图像中的特定区域进行风格化处理。标准做法是在完成样式化后简单地用蒙版遮盖不需要的区域。这项研究表明，这种方法往往无法适当地捕捉到感兴趣区域内的风格特征。为此，我们提出了一种基于部分卷积的艺术风格转移网络，能够精确地将风格特征应用到感兴趣的特定区域内。此外，我们还介绍了网络内部融合技术来处理区域选择中的不完美之处。通过使用SA-1B数据集的例子，我们展示了这种做法在视觉和定量上都提高了样式化的质量。代码已公开发布在[提供的链接]。
URL
https://arxiv.org/abs/2508.05769
PDF
-
Coarse-to-Fine Joint Registration of MR and Ultrasound Images via Imaging Style Transfer
2025-08-07 10:27:50Junyi Wang, Xi Zhu, Yikun Guo, Zixi Wang, Haichuan Gao, Le Zhang, Fan ZhangAbstract
We developed a pipeline for registering pre-surgery Magnetic Resonance (MR) images and post-resection Ultrasound (US) images. Our approach leverages unpaired style transfer using 3D CycleGAN to generate synthetic T1 images, thereby enhancing registration performance. Additionally, our registration process employs both affine and local deformable transformations for a coarse-to-fine registration. The results demonstrate that our approach improves the consistency between MR and US image pairs in most cases.
Abstract (translated)
我们开发了一种管道，用于注册术前磁共振（MR）图像和术后超声（US）图像。我们的方法利用未配对的风格转换技术，通过3D CycleGAN生成合成T1图像，从而提高图像配准性能。此外，我们的配准过程采用了仿射变换和局部可变形变换相结合的方法，实现了从粗到精的配准。实验结果表明，在大多数情况下，我们的方法提高了MR与US图像对之间的匹配一致性。
URL
https://arxiv.org/abs/2508.05240
PDF
-
Learning Latent Representations for Image Translation using Frequency Distributed CycleGAN
2025-08-05 12:59:37Shivangi Nigam, Adarsh Prasad Behera, Shekhar Verma, P. NagabhushanAbstract
This paper presents Fd-CycleGAN, an image-to-image (I2I) translation framework that enhances latent representation learning to approximate real data distributions. Building upon the foundation of CycleGAN, our approach integrates Local Neighborhood Encoding (LNE) and frequency-aware supervision to capture fine-grained local pixel semantics while preserving structural coherence from the source domain. We employ distribution-based loss metrics, including KL/JS divergence and log-based similarity measures, to explicitly quantify the alignment between real and generated image distributions in both spatial and frequency domains. To validate the efficacy of Fd-CycleGAN, we conduct experiments on diverse datasets -- Horse2Zebra, Monet2Photo, and a synthetically augmented Strike-off dataset. Compared to baseline CycleGAN and other state-of-the-art methods, our approach demonstrates superior perceptual quality, faster convergence, and improved mode diversity, particularly in low-data regimes. By effectively capturing local and global distribution characteristics, Fd-CycleGAN achieves more visually coherent and semantically consistent translations. Our results suggest that frequency-guided latent learning significantly improves generalization in image translation tasks, with promising applications in document restoration, artistic style transfer, and medical image synthesis. We also provide comparative insights with diffusion-based generative models, highlighting the advantages of our lightweight adversarial approach in terms of training efficiency and qualitative output.
Abstract (translated)
本文提出了Fd-CycleGAN，这是一种图像到图像（I2I）转换框架，它通过增强潜在表示学习来逼近真实数据分布。基于CycleGAN的基础，我们的方法整合了局部邻域编码（LNE）和频率感知监督，以捕捉源领域的精细局部像素语义并保持结构一致性。我们采用基于分布的损失度量，包括KL/JS散度和对数相似性测量，明确量化真实图像和生成图像在空间和频域分布之间的对齐程度。 为了验证Fd-CycleGAN的有效性，我们在多种数据集上进行了实验——Horse2Zebra、Monet2Photo以及合成增强的Strike-off数据集。与基线CycleGAN和其他最先进的方法相比，我们的方法展示了更优的感知质量、更快的收敛速度和改进的模态多样性，尤其是在低数据环境下。 通过有效捕捉局部和全局分布特征，Fd-CycleGAN实现了更加视觉上一致且语义一致的转换。我们的结果显示，在图像翻译任务中频率引导的潜在学习显著提高了泛化能力，并在文档恢复、艺术风格转移以及医学图像合成等领域具有应用前景。此外，我们还提供了与基于扩散的生成模型比较分析的结果，突出了我们轻量级对抗方法在训练效率和定性输出方面的优势。
URL
https://arxiv.org/abs/2508.03415
PDF
-
Beyond Vulnerabilities: A Survey of Adversarial Attacks as Both Threats and Defenses in Computer Vision Systems
2025-08-03 17:02:05Zhongliang Guo, Yifei Qian, Yanli Li, Weiye Li, Chun Tong Lei, Shuai Zhao, Lei Fang, Ognjen Arandjelovi\'c, Chun Pong LauAbstract
Adversarial attacks against computer vision systems have emerged as a critical research area that challenges the fundamental assumptions about neural network robustness and security. This comprehensive survey examines the evolving landscape of adversarial techniques, revealing their dual nature as both sophisticated security threats and valuable defensive tools. We provide a systematic analysis of adversarial attack methodologies across three primary domains: pixel-space attacks, physically realizable attacks, and latent-space attacks. Our investigation traces the technical evolution from early gradient-based methods such as FGSM and PGD to sophisticated optimization techniques incorporating momentum, adaptive step sizes, and advanced transferability mechanisms. We examine how physically realizable attacks have successfully bridged the gap between digital vulnerabilities and real-world threats through adversarial patches, 3D textures, and dynamic optical perturbations. Additionally, we explore the emergence of latent-space attacks that leverage semantic structure in internal representations to create more transferable and meaningful adversarial examples. Beyond traditional offensive applications, we investigate the constructive use of adversarial techniques for vulnerability assessment in biometric authentication systems and protection against malicious generative models. Our analysis reveals critical research gaps, particularly in neural style transfer protection and computational efficiency requirements. This survey contributes a comprehensive taxonomy, evolution analysis, and identification of future research directions, aiming to advance understanding of adversarial vulnerabilities and inform the development of more robust and trustworthy computer vision systems.
Abstract (translated)
针对计算机视觉系统的对抗攻击已发展成为一项关键研究领域，挑战了关于神经网络鲁棒性和安全性的基本假设。这篇全面的综述考察了对抗技术不断演变的格局，并揭示了它们作为复杂的安全威胁和有价值的防御工具的双重性质。我们系统地分析了三种主要领域的对抗性攻击方法：像素空间攻击、物理上可实现的攻击以及潜在空间攻击。 我们的研究追溯了从早期基于梯度的方法（如FGSM和PGD）到包含动量、自适应步长和先进转移机制等更复杂优化技术的技术演变。我们还探讨了如何通过对抗性贴纸、3D纹理和动态光学扰动，物理上可实现的攻击成功地将数字漏洞与现实世界威胁联系起来。 此外，我们研究了潜在空间攻击的出现，这些攻击利用内部表示中的语义结构来创建更具转移性和意义的对抗样本。除了传统的进攻应用之外，我们还调查了如何利用对抗性技术进行生物特征认证系统中的脆弱性评估以及防范恶意生成模型。 我们的分析揭示了一些关键的研究空白，特别是在神经风格转换保护和计算效率要求方面。这份综述贡献了一个全面的技术分类、演进分析，并识别出未来研究的方向，旨在深化对对抗性漏洞的理解，并促进开发更加稳健且可信的计算机视觉系统。
URL
https://arxiv.org/abs/2508.01845
PDF
-
Automatic Melody Reduction via Shortest Path Finding
2025-08-03 03:42:08Ziyu Wang, Yuxuan Wu, Roger B. Dannenberg, Gus XiaAbstract
Melody reduction, as an abstract representation of musical compositions, serves not only as a tool for music analysis but also as an intermediate representation for structured music generation. Prior computational theories, such as the Generative Theory of Tonal Music, provide insightful interpretations of music, but they are not fully automatic and usually limited to the classical genre. In this paper, we propose a novel and conceptually simple computational method for melody reduction using a graph-based representation inspired by principles from computational music theories, where the reduction process is formulated as finding the shortest path. We evaluate our algorithm on pop, folk, and classical genres, and experimental results show that the algorithm produces melody reductions that are more faithful to the original melody and more musically coherent than other common melody downsampling methods. As a downstream task, we use melody reductions to generate symbolic music variations. Experiments show that our method achieves higher quality than state-of-the-art style transfer methods.
Abstract (translated)
旋律简化作为一种音乐作品的抽象表示，不仅作为音乐分析的工具使用，还作为结构化音乐生成的中间表达。此前的计算理论，如《调性音乐生成理论》，提供了对音乐深入的解读，但它们并非完全自动化，并且通常仅限于古典音乐领域。在本文中，我们提出了一种新颖而概念简单的基于图表示的旋律简化计算方法，该方法受计算音乐理论原理启发，将简化过程定义为寻找最短路径问题。我们在流行、民间和古典音乐等多种类型上评估了我们的算法，并通过实验结果证明，与常用的旋律降采样方法相比，该算法生成的旋律简化更忠实于原始旋律且更具音乐一致性。作为下游任务，我们利用简化后的旋律来生成符号音乐变体。实验表明，我们的方法比现有的最先进的风格转换方法产生了更高品质的结果。
URL
https://arxiv.org/abs/2508.01571
PDF
-
StyDeco: Unsupervised Style Transfer with Distilling Priors and Semantic Decoupling
2025-08-02 06:17:23Yuanlin Yang, Quanjian Song, Zhexian Gao, Ge Wang, Shanshan Li, Xiaoyan ZhangAbstract
Diffusion models have emerged as the dominant paradigm for style transfer, but their text-driven mechanism is hindered by a core limitation: it treats textual descriptions as uniform, monolithic guidance. This limitation overlooks the semantic gap between the non-spatial nature of textual descriptions and the spatially-aware attributes of visual style, often leading to the loss of semantic structure and fine-grained details during stylization. In this paper, we propose StyDeco, an unsupervised framework that resolves this limitation by learning text representations specifically tailored for the style transfer task. Our framework first employs Prior-Guided Data Distillation (PGD), a strategy designed to distill stylistic knowledge without human supervision. It leverages a powerful frozen generative model to automatically synthesize pseudo-paired data. Subsequently, we introduce Contrastive Semantic Decoupling (CSD), a task-specific objective that adapts a text encoder using domain-specific weights. CSD performs a two-class clustering in the semantic space, encouraging source and target representations to form distinct clusters. Extensive experiments on three classic benchmarks demonstrate that our framework outperforms several existing approaches in both stylistic fidelity and structural preservation, highlighting its effectiveness in style transfer with semantic preservation. In addition, our framework supports a unique de-stylization process, further demonstrating its extensibility. Our code is vailable at this https URL.
Abstract (translated)
扩散模型已经成为风格迁移的主导范式，但其基于文本驱动的机制受到一个核心限制：它将文字描述视为统一且单一的指导信息。这一限制忽视了文本描述非空间性质与视觉风格的空间感知属性之间的语义差距，常常导致在风格化过程中丧失语义结构和细粒度细节。 本文提出了一种名为StyDeco的新框架，该框架通过学习专门针对风格迁移任务设计的文本表示来解决上述问题。我们的框架首先采用Prior-Guided Data Distillation (PGD)策略，这是一种无需人类监督即可提取风格知识的方法。它利用一个强大的冻结生成模型自动合成伪配对数据。随后，我们引入了Contrastive Semantic Decoupling (CSD)，这是一个任务特定的目标函数，使用领域特有权重来调整文本编码器。CSD在语义空间中执行两类聚类操作，鼓励源表示和目标表示形成不同的集群。 通过三个经典基准测试的广泛实验表明，我们的框架在风格保真度和结构保留方面均优于多种现有方法，在具有语义保存能力的风格迁移方面表现出显著效果。此外，我们的框架还支持独特的去风格化过程，进一步证明了其可扩展性。代码可在以下链接获取：[此 URL]（请将 [此 URL] 替换为实际的代码库链接）。
URL
https://arxiv.org/abs/2508.01215
PDF
