Style_Transfer
-
Text Detoxification in isiXhosa and Yor`ub'a: A Cross-Lingual Machine Learning Approach for Low-Resource African Languages
2026-01-09 08:28:58Abayomi O. AgbeyangiAbstract
Toxic language is one of the major barrier to safe online participation, yet robust mitigation tools are scarce for African languages. This study addresses this critical gap by investigating automatic text detoxification (toxic to neutral rewriting) for two low-resource African languages, isiXhosa and Yorùbá. The work contributes a novel, pragmatic hybrid methodology: a lightweight, interpretable TF-IDF and Logistic Regression model for transparent toxicity detection, and a controlled lexicon- and token-guided rewriting component. A parallel corpus of toxic to neutral rewrites, which captures idiomatic usage, diacritics, and code switching, was developed to train and evaluate the model. The detection component achieved stratified K-fold accuracies of 61-72% (isiXhosa) and 72-86% (Yorùbá), with per-language ROC-AUCs up to 0.88. The rewriting component successfully detoxified all detected toxic sentences while preserving 100% of non-toxic sentences. These results demonstrate that scalable, interpretable machine learning detectors combined with rule-based edits offer a competitive and resource-efficient solution for culturally adaptive safety tooling, setting a new benchmark for low-resource Text Style Transfer (TST) in African languages.
Abstract (translated)
毒性语言是在线安全参与的主要障碍之一，但针对非洲语言的有效缓解工具却极为稀缺。本研究通过调查两种低资源非洲语言（isiXhosa 和 Yorùbá）的自动文本去毒化（将有毒内容转化为中立表述），来填补这一重要空白。 该研究提出了一种新颖且实用的混合方法：一个轻量级、可解释性高的 TF-IDF 和逻辑回归模型，用于透明度高的毒性检测；以及一个受控词典和标记引导重写组件。为了训练和评估模型，开发了一个平行语料库，其中包括有毒到中立表述转换，并捕捉了惯用表达、音标以及代码切换现象。 在isiXhosa语言上，该方法的检测部分实现了61-72%的分层K折准确率；而在Yorùbá语言上，则实现了72-86%的准确率。每种语言的最大ROC-AUC值达到了0.88。重写组件成功地去毒化了所有被检出的有毒句子，同时保持了非有毒句子100%的完整性。 这些结果表明，结合规则编辑和可解释性强、可扩展性好的机器学习检测器能够为文化适应性的安全工具提供竞争性和资源效率高的解决方案，并在非洲语言中的低资源文本风格转换（TST）领域树立新的标准。
URL
https://arxiv.org/abs/2601.05624
PDF
-
QwenStyle: Content-Preserving Style Transfer with Qwen-Image-Edit
2026-01-08 10:22:51Shiwen Zhang, Haibin Huang, Chi Zhang, Xuelong LiAbstract
Content-Preserving Style transfer, given content and style references, remains challenging for Diffusion Transformers (DiTs) due to its internal entangled content and style features. In this technical report, we propose the first content-preserving style transfer model trained on Qwen-Image-Edit, which activates Qwen-Image-Edit's strong content preservation and style customization capability. We collected and filtered high quality data of limited specific styles and synthesized triplets with thousands categories of style images in-the-wild. We introduce the Curriculum Continual Learning framework to train QwenStyle with such mixture of clean and noisy triplets, which enables QwenStyle to generalize to unseen styles without degradation of the precise content preservation capability. Our QwenStyle V1 achieves state-of-the-art performance in three core metrics: style similarity, content consistency, and aesthetic quality.
Abstract (translated)
基于给定内容和风格参考，对于扩散变换器（DiT）而言，进行保留内容的风格迁移仍然是一项挑战，因为其内部的内容和风格特征相互纠缠。在这份技术报告中，我们提出了第一个在Qwen-Image-Edit上训练的、能够保持内容并定制风格的能力的保留内容的风格迁移模型。我们收集并过滤了高质量且具有特定风格的数据，并合成含有数千种不同风格图片的三元组数据集。为了使用这种混合清洁与噪点三元组数据训练QwenStyle，我们引入了一个课程连续学习框架，使QwenStyle能够泛化到未见过的风格上，同时不会降低精准的内容保持能力。我们的QwenStyle V1在三个核心指标上取得了最先进的性能：风格相似度、内容一致性以及美学质量。
URL
https://arxiv.org/abs/2601.06202
PDF
-
FaceRefiner: High-Fidelity Facial Texture Refinement with Differentiable Rendering-based Style Transfer
2026-01-08 02:34:29Chengyang Li, Baoping Cheng, Yao Cheng, Haocheng Zhang, Renshuai Liu, Yinglin Zheng, Jing Liao, Xuan ChengAbstract
Recent facial texture generation methods prefer to use deep networks to synthesize image content and then fill in the UV map, thus generating a compelling full texture from a single image. Nevertheless, the synthesized texture UV map usually comes from a space constructed by the training data or the 2D face generator, which limits the methods' generalization ability for in-the-wild input images. Consequently, their facial details, structures and identity may not be consistent with the input. In this paper, we address this issue by proposing a style transfer-based facial texture refinement method named FaceRefiner. FaceRefiner treats the 3D sampled texture as style and the output of a texture generation method as content. The photo-realistic style is then expected to be transferred from the style image to the content image. Different from current style transfer methods that only transfer high and middle level information to the result, our style transfer method integrates differentiable rendering to also transfer low level (or pixel level) information in the visible face regions. The main benefit of such multi-level information transfer is that, the details, structures and semantics in the input can thus be well preserved. The extensive experiments on Multi-PIE, CelebA and FFHQ datasets demonstrate that our refinement method can improve the texture quality and the face identity preserving ability, compared with state-of-the-arts.
Abstract (translated)
最近的面部纹理生成方法倾向于使用深度网络合成图像内容，然后填充UV映射（一种在三维建模中用于描述物体表面几何和材质分布的方法），从而从单张图片生成逼真的完整纹理。然而，所合成的纹理UV映射通常源自于训练数据或二维人脸生成器构建的空间内，这限制了这些方法对野外输入图像的泛化能力。因此，产生的面部细节、结构及身份可能与输入不一致。 为了解决这个问题，本文提出了一种基于风格迁移的面部纹理精炼方法，名为FaceRefiner。FaceRefiner将从3D采样的纹理视作风格，而纹理生成方法的输出则被视为内容。期望通过这种方法将照片般真实的风格从风格图像转移到内容图像上。与当前仅转移高层和中层信息到结果的风格迁移方法不同的是，我们的风格迁移方法结合了可微分渲染技术，从而也能够将低层（或像素级别）的信息传递至可见面部区域。这种多层次信息传输的主要优点在于，可以更好地保留输入中的细节、结构及语义。 在Multi-PIE、CelebA和FFHQ数据集上进行的广泛实验表明，与现有先进方法相比，我们的精炼方法能够提高纹理质量和人脸身份保持能力。
URL
https://arxiv.org/abs/2601.04520
PDF
-
LAMS-Edit: Latent and Attention Mixing with Schedulers for Improved Content Preservation in Diffusion-Based Image and Style Editing
2026-01-06 12:57:05Wingwa Fu, Takayuki OkataniAbstract
Text-to-Image editing using diffusion models faces challenges in balancing content preservation with edit application and handling real-image editing. To address these, we propose LAMS-Edit, leveraging intermediate states from the inversion process--an essential step in real-image editing--during edited image generation. Specifically, latent representations and attention maps from both processes are combined at each step using weighted interpolation, controlled by a scheduler. This technique, Latent and Attention Mixing with Schedulers (LAMS), integrates with Prompt-to-Prompt (P2P) to form LAMS-Edit--an extensible framework that supports precise editing with region masks and enables style transfer via LoRA. Extensive experiments demonstrate that LAMS-Edit effectively balances content preservation and edit application.
Abstract (translated)
基于扩散模型的文本到图像编辑面临在内容保留和编辑应用之间取得平衡以及处理真实图像编辑的挑战。为了解决这些问题，我们提出了LAMS-Edit方法，该方法利用了反转过程中产生的中间状态——这是真实图像编辑中的一个关键步骤。具体而言，在生成编辑后的图像时，该方法结合了来自这两个过程（即原始编辑和反转过程）在每一步的潜在表示和注意力图，并通过加权插值进行组合，其权重由调度器控制。这一技术被称为Latent and Attention Mixing with Schedulers (LAMS)，并与Prompt-to-Prompt (P2P)集成以形成LAMS-Edit——一个支持区域掩码精确编辑并可通过LoRA实现风格转换的可扩展框架。广泛的实验表明，LAMS-Edit在内容保留和编辑应用之间实现了有效的平衡。
URL
https://arxiv.org/abs/2601.02987
PDF
-
Unsupervised Text Style Transfer for Controllable Intensity
2026-01-03 04:04:07Shuhuan Gu, Wenbiao Tao, Xinchen Ma, Kangkang He, Ye Guo, Xiang Li, Yunshi LanAbstract
Unsupervised Text Style Transfer (UTST) aims to build a system to transfer the stylistic properties of a given text without parallel text pairs. Compared with text transfer between style polarities, UTST for controllable intensity is more challenging due to the subtle differences in stylistic features across different intensity levels. Faced with the challenges posed by the lack of parallel data and the indistinguishability between adjacent intensity levels, we propose a SFT-then-PPO paradigm to fine-tune an LLM. We first fine-tune the LLM with synthesized parallel data. Then, we further train the LLM with PPO, where the rewards are elaborately designed for distinguishing the stylistic intensity in hierarchical levels. Both the global and local stylistic features are considered to formulate the reward functions. The experiments on two UTST benchmarks showcase that both rewards have their advantages and applying them to LLM fine-tuning can effectively improve the performance of an LLM backbone based on various evaluation metrics. Even for close levels of intensity, we can still observe the noticeable stylistic difference between the generated text.
Abstract (translated)
无监督文本风格迁移（UTST）的目标是构建一个系统，能够将给定文本的风格特征转移到其他文本中，而不需要平行文本对。与在不同风格极性之间进行文本转换相比，针对可控强度的UTST更具挑战性，因为不同强度级别的风格特征之间的差异非常细微。面对缺乏平行数据以及相邻强度级别难以区分的问题，我们提出了一种SFT-then-PPO（预训练后强化学习）范式来对大规模语言模型（LLM）进行微调。首先，使用合成的平行文本数据对LLM进行微调；然后，进一步利用PPO（Proximal Policy Optimization）方法训练该模型，其中奖励函数精心设计用于区分不同层次上的风格强度。在制定奖励函数时，既考虑了全局又考虑了局部的风格特征。 实验结果表明，在两个UTST基准测试上应用这两种类型的奖励均表现出各自的优点，并且将它们应用于LLM微调过程中能够有效提升基于多种评估指标下的模型性能。即使对于强度接近的情况下，仍然可以观察到生成文本之间的显著风格差异。
URL
https://arxiv.org/abs/2601.01060
PDF
-
MorphAny3D: Unleashing the Power of Structured Latent in 3D Morphing
2026-01-01 04:42:59Xiaokun Sun, Zeyu Cai, Hao Tang, Ying Tai, Jian Yang, Zhenyu ZhangAbstract
3D morphing remains challenging due to the difficulty of generating semantically consistent and temporally smooth deformations, especially across categories. We present MorphAny3D, a training-free framework that leverages Structured Latent (SLAT) representations for high-quality 3D morphing. Our key insight is that intelligently blending source and target SLAT features within the attention mechanisms of 3D generators naturally produces plausible morphing sequences. To this end, we introduce Morphing Cross-Attention (MCA), which fuses source and target information for structural coherence, and Temporal-Fused Self-Attention (TFSA), which enhances temporal consistency by incorporating features from preceding frames. An orientation correction strategy further mitigates the pose ambiguity within the morphing steps. Extensive experiments show that our method generates state-of-the-art morphing sequences, even for challenging cross-category cases. MorphAny3D further supports advanced applications such as decoupled morphing and 3D style transfer, and can be generalized to other SLAT-based generative models. Project page: this https URL.
Abstract (translated)
3D变形（morphing）仍然面临挑战，主要是因为生成具有语义一致性和时间上平滑的形变非常困难，尤其是在跨类别的情况下。我们提出了MorphAny3D，这是一个无需训练的框架，它利用结构化潜在（SLAT）表示来实现高质量的3D变形。我们的关键见解是，在3D生成器的注意力机制中智能地混合源和目标SLAT特征能够自然产生合理的形变序列。 为此，我们引入了形变交叉注意（MCA），该技术融合源和目标信息以确保结构的一致性，并且引入时间融合自关注（TFSA）来通过加入前一帧的特性增强时间上的连续性。一种方向校正策略进一步减轻变形步骤中的姿态模糊问题。 大量的实验表明，我们的方法可以生成业界领先的形变序列，甚至在具有挑战性的跨类别情况下也是如此。MorphAny3D还支持更高级的应用程序，如解耦变形和3D风格转换，并且可以扩展应用于其他基于SLAT的生成模型。项目页面：[此链接](https://this-url.com/)（请将占位符替换为实际URL）。
URL
https://arxiv.org/abs/2601.00204
PDF
-
Comparative Evaluation of CNN Architectures for Neural Style Transfer in Indonesian Batik Motif Generation: A Comprehensive Study
2025-12-31 11:06:09Happy Gery Pangestu, Andi Prademon Yunus, Siti KhomsahAbstract
Neural Style Transfer (NST) provides a computational framework for the digital preservation and generative exploration of Indonesian batik motifs; however, existing approaches remain largely centered on VGG-based architectures whose strong stylistic expressiveness comes at the cost of high computational and memory demands, that limits practical deployment in resource-limited environments. This study presents a systematic comparative analysis of five widely used CNN backbones, namely VGG16, VGG19, Inception V3, ResNet50, and ResNet101, based on 245 controlled experiments combining quantitative metrics, qualitative assessment, and statistical analysis to examine the trade-off between structural preservation, stylistic behavior, and computational efficiency. The results show that backbone selection does not yield statistically significant differences in structural similarity, as confirmed by ANOVA on SSIM (p= 0.83), indicating comparable levels of structural preservation rather than equivalent stylistic quality. Within this context, ResNet-based architectures achieve approximately 5-6x faster convergence than VGG models while maintaining similar perceptual similarity (LPIPS = 0.53) and requiring over 16x fewer FLOPs (0.63 vs 10.12 GFLOPs). Qualitative analysis reveals consistent stylistic trade-offs, with VGG producing denser painterly textures, ResNet favoring geometric stability and canting stroke preservation with milder stylization, and Inception V3 exhibiting intermediate but noisier behavior. These findings reposition architectural choice in NST from maximizing stylistic intensity toward efficiency-aware and structure-preserving deployment, highlighting ResNet-based backbones as a practical foundation for scalable, industry-oriented batik generation.
Abstract (translated)
神经风格迁移（NST）为印尼蜡染图案的数字保存和生成探索提供了一个计算框架；然而，现有的方法主要集中在基于VGG的架构上，虽然这种架构具有强大的表现力，但其高计算和内存需求限制了它在资源受限环境中的实际部署。这项研究对五种广泛使用的CNN骨干网络（即VGG16、VGG19、Inception V3、ResNet50和ResNet101）进行了系统的比较分析，基于245次有控制的实验结合定量指标、定性评估以及统计分析来考察结构保留、风格行为及计算效率之间的权衡。研究结果表明，在结构相似度方面（由SSIM的ANOVA测试得出p=0.83确认），骨干网络的选择并没有显示出显著差异，这说明不同模型之间具有相当接近的结构保存水平，并非同等的质量表现。 在这种背景下，ResNet架构比VGG模型快约5-6倍收敛速度，同时保持了相似的感知相似性（LPIPS = 0.53）和减少了16倍以上的浮点运算次数（FLOPs：分别为0.63 vs 10.12 Giga FLOPs）。定性分析表明存在一致性的风格权衡问题，VGG模型产生了更密集的绘画纹理，ResNet则倾向于几何稳定性及蜡笔线条保存，并且其风格化效果较为温和；而Inception V3展示了一种介于两者之间的但噪音更大的行为。 这些发现重新定位了NST中的架构选择，从最大化风格强度转向效率感知和结构保留的部署。研究结果强调基于ResNet的骨干网络作为具有规模性和行业导向性的蜡染生成实用基础的重要性。
URL
https://arxiv.org/abs/2601.00888
PDF
-
MiMo-Audio: Audio Language Models are Few-Shot Learners
2025-12-29 19:06:05Core Team, Dong Zhang, Gang Wang, Jinlong Xue, Kai Fang, Liang Zhao, Rui Ma, Shuhuai Ren, Shuo Liu, Tao Guo, Weiji Zhuang, Xin Zhang, Xingchen Song, Yihan Yan, Yongzhe He, Cici, Bowen Shen, Chengxuan Zhu, Chong Ma, Chun Chen, Heyu Chen, Jiawei Li, Lei Li, Menghang Zhu, Peidian Li, Qiying Wang, Sirui Deng, Weimin Xiong, Wenshan Huang, Wenyu Yang, Yilin Jiang, Yixin Yang, Yuanyuan Tian, Yue Ma, Yue Yu, Zihan Zhang, Zihao Yue, Bangjun Xiao, Bingquan Xia, Bofei Gao, Bowen Ye, Can Cai, Chang Liu, Chenhong He, Chunan Li, Dawei Zhu, Duo Zhang, Fengyuan Shi, Guoan Wang, Hailin Zhang, Hanglong Lv, Hanyu Li, Hao Tian, Heng Qu, Hongshen Xu, Houbin Zhang, Huaqiu Liu, Jiangshan Duo, Jianguang Zuo, Jianyu Wei, Jiebao Xiao, Jinhao Dong, Jun Shi, Junhao Hu, Kainan Bao, Kang Zhou, Linghao Zhang, Meng Chen, Nuo Chen, Peng Zhang, Qianli Chen, Qiantong Wang, Rang Li, Shaohui Liu, Shengfan Wang, Shicheng Li, Shihua Yu, Shijie Cao, Shimao Chen, Shuhao Gu, Weikun Wang, Wenhan Ma, Xiangwei Deng, Xing Yong, Xing Zhang, Xu Wang, Yifan Song, Yihao Zhao, Yingbo Zhao, Yizhao Gao, Yu Cheng, Yu Tu, Yudong Wang, Zhaojun Huang, Zhengju Tang, Zhenru Lin, Zhichao Song, Zhipeng Xu, Zhixian Zheng, Zihan JiangAbstract
Existing audio language models typically rely on task-specific fine-tuning to accomplish particular audio tasks. In contrast, humans are able to generalize to new audio tasks with only a few examples or simple instructions. GPT-3 has shown that scaling next-token prediction pretraining enables strong generalization capabilities in text, and we believe this paradigm is equally applicable to the audio domain. By scaling MiMo-Audio's pretraining data to over one hundred million of hours, we observe the emergence of few-shot learning capabilities across a diverse set of audio tasks. We develop a systematic evaluation of these capabilities and find that MiMo-Audio-7B-Base achieves SOTA performance on both speech intelligence and audio understanding benchmarks among open-source models. Beyond standard metrics, MiMo-Audio-7B-Base generalizes to tasks absent from its training data, such as voice conversion, style transfer, and speech editing. MiMo-Audio-7B-Base also demonstrates powerful speech continuation capabilities, capable of generating highly realistic talk shows, recitations, livestreaming and debates. At the post-training stage, we curate a diverse instruction-tuning corpus and introduce thinking mechanisms into both audio understanding and generation. MiMo-Audio-7B-Instruct achieves open-source SOTA on audio understanding benchmarks (MMSU, MMAU, MMAR, MMAU-Pro), spoken dialogue benchmarks (Big Bench Audio, MultiChallenge Audio) and instruct-TTS evaluations, approaching or surpassing closed-source models. Model checkpoints and full evaluation suite are available at this https URL.
Abstract (translated)
现有的音频语言模型通常依赖于特定任务的微调来完成具体的音频任务。相比之下，人类能够在仅有少量示例或简单指令的情况下掌握新的音频任务。GPT-3 显示出扩大下一个标记预测预训练可以增强文本中的强大泛化能力，我们认为这种范式同样适用于音频领域。通过将 MiMo-Audio 的预训练数据扩展到超过一亿小时的规模，我们观察到了跨多种音频任务出现少量样本学习能力的现象。我们系统地评估了这些能力，并发现 MiMo-Audio-7B-Base 在开源模型中于语音智能和音频理解基准测试上达到了最先进的性能（SOTA）。除了标准度量之外，MiMo-Audio-7B-Base 还可以泛化到其训练数据中不存在的任务，例如声音转换、风格转移以及语音编辑。此外，MiMo-Audio-7B-Base 展现了强大的语音持续生成能力，能够产生高度逼真的脱口秀节目、朗诵、直播和辩论内容。 在预训练之后的阶段，我们整理了一个多样化的指令微调语料库，并将思考机制引入到音频理解和生成中。MiMo-Audio-7B-Instruct 在音频理解基准测试（MMSU, MMAU, MMAR, MMAU-Pro）、语音对话基准测试（Big Bench Audio, MultiChallenge Audio）以及指令TTS评估方面达到了开源模型的最高性能，甚至接近或超过了封闭源代码模型的表现。该模型的检查点和完整评测套件可以在提供的链接中找到：[此处插入链接]。
URL
https://arxiv.org/abs/2512.23808
PDF
-
UTDesign: A Unified Framework for Stylized Text Editing and Generation in Graphic Design Images
2025-12-23 16:13:55Yiming Zhao, Yuanpeng Gao, Yuxuan Luo, Jiwei Duan, Shisong Lin, Longfei Xiong, Zhouhui LianAbstract
AI-assisted graphic design has emerged as a powerful tool for automating the creation and editing of design elements such as posters, banners, and advertisements. While diffusion-based text-to-image models have demonstrated strong capabilities in visual content generation, their text rendering performance, particularly for small-scale typography and non-Latin scripts, remains limited. In this paper, we propose UTDesign, a unified framework for high-precision stylized text editing and conditional text generation in design images, supporting both English and Chinese scripts. Our framework introduces a novel DiT-based text style transfer model trained from scratch on a synthetic dataset, capable of generating transparent RGBA text foregrounds that preserve the style of reference glyphs. We further extend this model into a conditional text generation framework by training a multi-modal condition encoder on a curated dataset with detailed text annotations, enabling accurate, style-consistent text synthesis conditioned on background images, prompts, and layout specifications. Finally, we integrate our approach into a fully automated text-to-design (T2D) pipeline by incorporating pre-trained text-to-image (T2I) models and an MLLM-based layout planner. Extensive experiments demonstrate that UTDesign achieves state-of-the-art performance among open-source methods in terms of stylistic consistency and text accuracy, and also exhibits unique advantages compared to proprietary commercial approaches. Code and data for this paper are available at this https URL.
Abstract (translated)
AI辅助的图形设计已作为自动化创建和编辑如海报、横幅和广告等设计元素的强大工具兴起。尽管基于扩散模型的文本到图像生成方法在视觉内容生成方面展示了强大的能力，但它们在小规模排版和非拉丁文字渲染上的性能仍然有限。本文中，我们提出了UTDesign框架，这是一个用于高精度风格化文本编辑及条件文本生成的设计图象统一框架，支持英文和中文脚本。我们的框架引入了一个基于DiT的新型文本样式迁移模型，该模型从合成数据集上训练得出，能够生成保持参考字符样式的透明RGBA文本前景。我们进一步通过在精心策划并带有详细文本注释的数据集上训练一个多模态条件编码器，将此模型扩展为一个条件文本生成框架，使基于背景图像、提示和布局规范的准确且风格一致的文字合成成为可能。最后，我们将方法整合到完全自动化的文本到设计（Text-to-Design, T2D）工作流中，通过融合预训练的文本到图像（T2I）模型及MLLM布局规划器实现这一目标。广泛的实验表明，UTDesign在开源方法中达到了风格一致性和文字准确性方面的顶尖水平，并且相比专有的商业方法也展现出独特的优势。本文代码和数据可在提供的链接获取。
URL
https://arxiv.org/abs/2512.20479
PDF
-
Generative Latent Coding for Ultra-Low Bitrate Image Compression
2025-12-23 09:35:40Zhaoyang Jia, Jiahao Li, Bin Li, Houqiang Li, Yan LuAbstract
Most existing image compression approaches perform transform coding in the pixel space to reduce its spatial redundancy. However, they encounter difficulties in achieving both high-realism and high-fidelity at low bitrate, as the pixel-space distortion may not align with human perception. To address this issue, we introduce a Generative Latent Coding (GLC) architecture, which performs transform coding in the latent space of a generative vector-quantized variational auto-encoder (VQ-VAE), instead of in the pixel space. The generative latent space is characterized by greater sparsity, richer semantic and better alignment with human perception, rendering it advantageous for achieving high-realism and high-fidelity compression. Additionally, we introduce a categorical hyper module to reduce the bit cost of hyper-information, and a code-prediction-based supervision to enhance the semantic consistency. Experiments demonstrate that our GLC maintains high visual quality with less than 0.04 bpp on natural images and less than 0.01 bpp on facial images. On the CLIC2020 test set, we achieve the same FID as MS-ILLM with 45% fewer bits. Furthermore, the powerful generative latent space enables various applications built on our GLC pipeline, such as image restoration and style transfer. The code is available at this https URL.
Abstract (translated)
大多数现有的图像压缩方法在像素空间中进行变换编码，以减少其空间冗余。然而，在低比特率下实现高真实感和高保真度方面遇到了困难，因为像素空间中的失真可能与人类感知不一致。为了解决这个问题，我们引入了一种生成式潜在编码（GLC）架构，该架构在生成向量量化变分自编码器（VQ-VAE）的潜在空间中进行变换编码，而不是在像素空间中。生成式潜在空间以其更高的稀疏性、更丰富的语义以及更好的人类感知一致性为特点，这使得它更适合实现高真实感和高保真的压缩效果。 此外，我们还引入了一种类别超模块以减少超信息的比特成本，并通过基于代码预测的监督来增强语义一致性。实验表明，在自然图像上我们的GLC在小于0.04 bpp的情况下能够保持高质量视觉效果，在面部图像上则是在小于0.01 bpp的情况下实现这一点。在CLIC2020测试集上，我们与MS-ILLM相比，实现了相同的FID分数但比特率减少了45%。 此外，强大的生成式潜在空间使我们的GLC管道能够支持多种应用，例如图像修复和风格转换。代码可在以下链接获取：[此URL]（请将“this https URL”替换为实际的GitHub或相关存储库链接）。
URL
https://arxiv.org/abs/2512.20194
PDF
-
LouvreSAE: Sparse Autoencoders for Interpretable and Controllable Style Transfer
2025-12-22 00:36:22Raina Panda, Daniel Fein, Arpita Singhal, Mark Fiore, Maneesh Agrawala, Matyas BohacekAbstract
Artistic style transfer in generative models remains a significant challenge, as existing methods often introduce style only via model fine-tuning, additional adapters, or prompt engineering, all of which can be computationally expensive and may still entangle style with subject matter. In this paper, we introduce a training- and inference-light, interpretable method for representing and transferring artistic style. Our approach leverages an art-specific Sparse Autoencoder (SAE) on top of latent embeddings of generative image models. Trained on artistic data, our SAE learns an emergent, largely disentangled set of stylistic and compositional concepts, corresponding to style-related elements pertaining brushwork, texture, and color palette, as well as semantic and structural concepts. We call it LouvreSAE and use it to construct style profiles: compact, decomposable steering vectors that enable style transfer without any model updates or optimization. Unlike prior concept-based style transfer methods, our method requires no fine-tuning, no LoRA training, and no additional inference passes, enabling direct steering of artistic styles from only a few reference images. We validate our method on ArtBench10, achieving or surpassing existing methods on style evaluations (VGG Style Loss and CLIP Score Style) while being 1.7-20x faster and, critically, interpretable.
Abstract (translated)
在生成模型中进行艺术风格转换仍然是一项重要挑战，现有方法通常通过微调模型、添加适配器或提示工程等方式引入风格，这些方式可能会消耗大量计算资源，并且仍可能导致风格与主题内容相互纠缠。在这篇论文中，我们提出了一种轻量级的训练和推理方法，用于表示并传输艺术风格。我们的方法利用了一个特定于艺术领域的稀疏自动编码器（SAE），该编码器基于生成图像模型的潜在嵌入之上构建。通过使用艺术数据进行训练，我们的SAE学习到了一组具有高度独立性的风格化与构图概念，这些概念对应着与笔触、纹理和色彩调板相关的元素以及语义和结构的概念。我们将这种方法命名为LouvreSAE，并利用它来构造风格配置文件：即紧凑且可分解的导向向量，这使得无需对模型进行任何更新或优化就能实现风格转换。 不同于之前的基于概念的方法，我们的方法不需要微调、LoRA训练或者额外的推理步骤，可以直接从少量参考图像中引导艺术风格。我们在ArtBench10数据集上验证了该方法的有效性，在评估风格时（包括VGG Style Loss和CLIP Score Style）达到了或超过了现有方法的表现水平，并且我们的方法速度提高了1.7到20倍，最为关键的是它是可解释的。
URL
https://arxiv.org/abs/2512.18930
PDF
-
Stylized Synthetic Augmentation further improves Corruption Robustness
2025-12-17 18:28:04Georg Siedel, Rojan Regmi, Abhirami Anand, Weijia Shao, Silvia Vock, Andrey MorozovAbstract
This paper proposes a training data augmentation pipeline that combines synthetic image data with neural style transfer in order to address the vulnerability of deep vision models to common corruptions. We show that although applying style transfer on synthetic images degrades their quality with respect to the common FID metric, these images are surprisingly beneficial for model training. We conduct a systematic empirical analysis of the effects of both augmentations and their key hyperparameters on the performance of image classifiers. Our results demonstrate that stylization and synthetic data complement each other well and can be combined with popular rule-based data augmentation techniques such as TrivialAugment, while not working with others. Our method achieves state-of-the-art corruption robustness on several small-scale image classification benchmarks, reaching 93.54%, 74.9% and 50.86% robust accuracy on CIFAR-10-C, CIFAR-100-C and TinyImageNet-C, respectively
Abstract (translated)
本文提出了一种训练数据增强管道，该管道结合了合成图像数据与神经风格转换技术，旨在解决深度视觉模型在面对常见干扰时的脆弱性问题。我们发现，尽管在合成图像上应用风格迁移会降低它们的质量（根据常见的FID度量标准），但这些图像对模型训练来说却意外地非常有益。 本文系统地分析了这两种增强方法及其关键超参数对图像分类器性能的影响。研究结果表明，风格化与合成数据可以很好地相互补充，并且能够与流行的基于规则的数据增强技术（如TrivialAugment）结合使用，但不能与其他某些方法一起使用。 我们的方法在几个小规模的图像分类基准测试中达到了最先进的抗干扰性，在CIFAR-10-C、CIFAR-100-C和TinyImageNet-C上的鲁棒准确率分别达到93.54%、74.9%和50.86%。
URL
https://arxiv.org/abs/2512.15675
PDF
-
Reasoning-Style Poisoning of LLM Agents via Stealthy Style Transfer: Process-Level Attacks and Runtime Monitoring in RSV Space
2025-12-16 14:34:10Xingfu Zhou, Pengfei WangAbstract
Large Language Model (LLM) agents relying on external retrieval are increasingly deployed in high-stakes environments. While existing adversarial attacks primarily focus on content falsification or instruction injection, we identify a novel, process-oriented attack surface: the agent's reasoning style. We propose Reasoning-Style Poisoning (RSP), a paradigm that manipulates how agents process information rather than what they process. We introduce Generative Style Injection (GSI), an attack method that rewrites retrieved documents into pathological tones--specifically "analysis paralysis" or "cognitive haste"--without altering underlying facts or using explicit triggers. To quantify these shifts, we develop the Reasoning Style Vector (RSV), a metric tracking Verification depth, Self-confidence, and Attention focus. Experiments on HotpotQA and FEVER using ReAct, Reflection, and Tree of Thoughts (ToT) architectures reveal that GSI significantly degrades performance. It increases reasoning steps by up to 4.4 times or induces premature errors, successfully bypassing state-of-the-art content filters. Finally, we propose RSP-M, a lightweight runtime monitor that calculates RSV metrics in real-time and triggers alerts when values exceed safety thresholds. Our work demonstrates that reasoning style is a distinct, exploitable vulnerability, necessitating process-level defenses beyond static content analysis.
Abstract (translated)
大型语言模型（LLM）代理越来越多地依赖外部检索，并被部署在高风险环境中。虽然现有的对抗性攻击主要集中在内容伪造或指令注入上，我们发现了一个新型的、以过程为导向的攻击面：即代理的推理风格。我们提出了推理样式投毒（Reasoning-Style Poisoning, RSP），这是一种通过改变代理处理信息的方式而非其所处理的信息本身来进行操纵的方法。我们引入了一种名为生成式风格注入（Generative Style Injection, GSI）的攻击方法，该方法将检索到的文档重写为病态语气——具体来说是“分析瘫痪”或“认知急躁”，而不会改变事实真相或使用显式的触发器。为了量化这些变化，我们开发了推理样式向量（Reasoning Style Vector, RSV），这是一个追踪验证深度、自我信心和注意力焦点的指标。 在HotpotQA和FEVER数据集上采用ReAct、反思及思维树（ToT）架构进行的实验表明，GSI显著降低了性能。它增加了多达4.4倍的推理步骤或导致过早出现错误，成功绕过了最先进的内容过滤器。最后，我们提出了RSP-M，这是一种轻量级的运行时监控工具，可在实时计算RSV指标，并在值超过安全阈限时触发警报。 我们的研究表明，推理风格是一个独特的、可被利用的安全漏洞，需要超越静态内容分析的过程级别的防御措施。
URL
https://arxiv.org/abs/2512.14448
PDF
-
Coarse-to-Fine Hierarchical Alignment for UAV-based Human Detection using Diffusion Models
2025-12-15 19:57:36Wenda Li, Meng Wu, Sungmin Eum, Heesung Kwon, Qing QuAbstract
Training object detectors demands extensive, task-specific annotations, yet this requirement becomes impractical in UAV-based human detection due to constantly shifting target distributions and the scarcity of labeled images. As a remedy, synthetic simulators are adopted to generate annotated data, with a low annotation cost. However, the domain gap between synthetic and real images hinders the model from being effectively applied to the target domain. Accordingly, we introduce Coarse-to-Fine Hierarchical Alignment (CFHA), a three-stage diffusion-based framework designed to transform synthetic data for UAV-based human detection, narrowing the domain gap while preserving the original synthetic labels. CFHA explicitly decouples global style and local content domain discrepancies and bridges those gaps using three modules: (1) Global Style Transfer -- a diffusion model aligns color, illumination, and texture statistics of synthetic images to the realistic style, using only a small real reference set; (2) Local Refinement -- a super-resolution diffusion model is used to facilitate fine-grained and photorealistic details for the small objects, such as human instances, preserving shape and boundary integrity; (3) Hallucination Removal -- a module that filters out human instances whose visual attributes do not align with real-world data to make the human appearance closer to the target distribution. Extensive experiments on public UAV Sim2Real detection benchmarks demonstrate that our methods significantly improve the detection accuracy compared to the non-transformed baselines. Specifically, our method achieves up to $+14.1$ improvement of mAP50 on Semantic-Drone benchmark. Ablation studies confirm the complementary roles of the global and local stages and highlight the importance of hierarchical alignment. The code is released at \href{this https URL}{this url}.
Abstract (translated)
训练目标检测器需要大量的、特定任务的标注数据，但在基于无人机的人类检测中，由于目标分布不断变化以及标记图像稀少，这一要求变得不切实际。作为解决方案，合成模拟器被用来生成有注释的数据，这大大降低了标注成本。然而，合成与真实图像之间的领域差距阻碍了模型在目标领域的有效应用。为此，我们引入了一种名为“粗到细层次对齐”（Coarse-to-Fine Hierarchical Alignment, CFHA）的三阶段扩散框架，旨在将合成数据转换为适用于基于无人机的人类检测，并缩小域间差异，同时保持原始标签信息不变。 CFHA明确地解耦了全局风格与局部内容领域的不一致，并通过三个模块来弥补这些差距：（1）全局样式迁移——一个扩散模型仅使用少量的真实参考集，就能将合成图像的颜色、光照和纹理统计特性对齐到真实样式的水平；（2）局部细化——一种超分辨率扩散模型被用来促进小物体（如人类实例）的细粒度和逼真细节，同时保持形状和边界的完整性；（3）幻觉移除——一个过滤模块将视觉属性与现实世界数据不一致的人类实例排除出去，使人类外观更接近目标分布。 在公共无人机Sim2Real检测基准上的广泛实验表明，我们的方法相比于未转换的基线显著提高了检测精度。具体而言，在Semantic-Drone基准上，我们的方法实现了高达+14.1 mAP50的改进。消融研究表明全局和局部阶段之间的互补作用，并强调了层次对齐的重要性。代码可在[此处](this https URL)获取。
URL
https://arxiv.org/abs/2512.13869
PDF
-
SCAdapter: Content-Style Disentanglement for Diffusion Style Transfer
2025-12-15 04:02:14Luan Thanh Trinh, Kenji Doi, Atsuki OsanaiAbstract
Diffusion models have emerged as the leading approach for style transfer, yet they struggle with photo-realistic transfers, often producing painting-like results or missing detailed stylistic elements. Current methods inadequately address unwanted influence from original content styles and style reference content features. We introduce SCAdapter, a novel technique leveraging CLIP image space to effectively separate and integrate content and style features. Our key innovation systematically extracts pure content from content images and style elements from style references, ensuring authentic transfers. This approach is enhanced through three components: Controllable Style Adaptive Instance Normalization (CSAdaIN) for precise multi-style blending, KVS Injection for targeted style integration, and a style transfer consistency objective maintaining process coherence. Comprehensive experiments demonstrate SCAdapter significantly outperforms state-of-the-art methods in both conventional and diffusion-based baselines. By eliminating DDIM inversion and inference-stage optimization, our method achieves at least $2\times$ faster inference than other diffusion-based approaches, making it both more effective and efficient for practical applications.
Abstract (translated)
扩散模型已经成为风格转换的领先方法，但它们在进行照片级写实的风格转换时存在困难，常常产生类似绘画的结果或遗漏细微的风格元素。目前的方法未能充分解决原始内容风格和风格参考内容特征之间不必要的相互影响问题。我们引入了SCAdapter，这是一种新颖的技术，利用CLIP图像空间有效地分离并整合内容和风格特征。我们的关键技术创新系统地从内容图片中提取纯净的内容，并从风格参考中提取风格元素，确保进行真实的转换。这种技术通过三个组件得到增强：可控风格自适应实例归一化（CSAdaIN）用于精确的多风格融合、KVS注入以实现目标风格集成以及一种保持过程一致性的风格传输一致性目标。全面的实验表明，在传统方法和基于扩散的方法基线上，SCAdapter显著优于现有的最先进的方法。通过消除DDIM反转和推理阶段优化，我们的方法至少比其他基于扩散的方法快两倍的推断速度，使其在实际应用中既更有效又更高效。
URL
https://arxiv.org/abs/2512.12963
PDF
-
Reducing Domain Gap with Diffusion-Based Domain Adaptation for Cell Counting
2025-12-12 18:19:41Mohammad Dehghanmanshadi, Wallapak TavanapongAbstract
Generating realistic synthetic microscopy images is critical for training deep learning models in label-scarce environments, such as cell counting with many cells per image. However, traditional domain adaptation methods often struggle to bridge the domain gap when synthetic images lack the complex textures and visual patterns of real samples. In this work, we adapt the Inversion-Based Style Transfer (InST) framework originally designed for artistic style transfer to biomedical microscopy images. Our method combines latent-space Adaptive Instance Normalization with stochastic inversion in a diffusion model to transfer the style from real fluorescence microscopy images to synthetic ones, while weakly preserving content structure. We evaluate the effectiveness of our InST-based synthetic dataset for downstream cell counting by pre-training and fine-tuning EfficientNet-B0 models on various data sources, including real data, hard-coded synthetic data, and the public Cell200-s dataset. Models trained with our InST-synthesized images achieve up to 37\% lower Mean Absolute Error (MAE) compared to models trained on hard-coded synthetic data, and a 52\% reduction in MAE compared to models trained on Cell200-s (from 53.70 to 25.95 MAE). Notably, our approach also outperforms models trained on real data alone (25.95 vs. 27.74 MAE). Further improvements are achieved when combining InST-synthesized data with lightweight domain adaptation techniques such as DACS with CutMix. These findings demonstrate that InST-based style transfer most effectively reduces the domain gap between synthetic and real microscopy data. Our approach offers a scalable path for enhancing cell counting performance while minimizing manual labeling effort. The source code and resources are publicly available at: this https URL.
Abstract (translated)
生成逼真的合成显微图像对于在标签稀缺的环境下训练深度学习模型（例如，每张图像中有大量细胞的情况下的计数任务）至关重要。然而，传统的领域适应方法往往难以跨越合成图像缺乏真实样本中的复杂纹理和视觉模式的领域差距。在这项工作中，我们将原本用于艺术风格迁移的基于逆向的样式转换(Inversion-Based Style Transfer, InST)框架应用于生物医学显微镜图像。我们的方法结合了潜在空间自适应实例归一化与扩散模型中的随机反转，以从真实荧光显微图像中传递风格到合成图像上，并弱化地保持内容结构。 我们通过在不同数据源（包括真实数据、硬编码的合成数据以及公共Cell200-s数据集）上对EfficientNet-B0模型进行预训练和微调来评估基于InST的合成数据集用于下游细胞计数的有效性。使用我们的InST生成图像训练得到的模型相比单纯使用硬编码合成数据训练的模型，平均绝对误差(Mean Absolute Error, MAE)最多可减少37%，与Cell200-s上训练的模型相比则能降低52%（MAE从53.70降至25.95）。值得注意的是，我们的方法甚至超过了单纯基于真实数据训练得到的结果（25.95 vs. 27.74 MAE）。 通过将InST合成的数据与轻量级领域适应技术相结合（如DACS结合CutMix），还可以进一步提高性能。这些发现表明，基于InST的风格转换最有效地减少了合成和真实显微数据之间的域差距。我们的方法为在减少人工标注工作的同时提升细胞计数性能提供了一条可扩展的途径。源代码和资源可在以下网址获取：[this https URL] (请将 [ ] 替换为实际链接地址)。
URL
https://arxiv.org/abs/2512.11763
PDF
-
EditMGT: Unleashing Potentials of Masked Generative Transformers in Image Editing
2025-12-12 16:51:19Wei Chow, Linfeng Li, Lingdong Kong, Zefeng Li, Qi Xu, Hang Song, Tian Ye, Xian Wang, Jinbin Bai, Shilin Xu, Xiangtai Li, Junting Pan, Shaoteng Liu, Ran Zhou, Tianshu Yang, Songhua LiuAbstract
Recent advances in diffusion models (DMs) have achieved exceptional visual quality in image editing tasks. However, the global denoising dynamics of DMs inherently conflate local editing targets with the full-image context, leading to unintended modifications in non-target regions. In this paper, we shift our attention beyond DMs and turn to Masked Generative Transformers (MGTs) as an alternative approach to tackle this challenge. By predicting multiple masked tokens rather than holistic refinement, MGTs exhibit a localized decoding paradigm that endows them with the inherent capacity to explicitly preserve non-relevant regions during the editing process. Building upon this insight, we introduce the first MGT-based image editing framework, termed EditMGT. We first demonstrate that MGT's cross-attention maps provide informative localization signals for localizing edit-relevant regions and devise a multi-layer attention consolidation scheme that refines these maps to achieve fine-grained and precise localization. On top of these adaptive localization results, we introduce region-hold sampling, which restricts token flipping within low-attention areas to suppress spurious edits, thereby confining modifications to the intended target regions and preserving the integrity of surrounding non-target areas. To train EditMGT, we construct CrispEdit-2M, a high-resolution dataset spanning seven diverse editing categories. Without introducing additional parameters, we adapt a pre-trained text-to-image MGT into an image editing model through attention injection. Extensive experiments across four standard benchmarks demonstrate that, with fewer than 1B parameters, our model achieves similarity performance while enabling 6 times faster editing. Moreover, it delivers comparable or superior editing quality, with improvements of 3.6% and 17.6% on style change and style transfer tasks, respectively.
Abstract (translated)
近期，扩散模型（DMs）在图像编辑任务中取得了卓越的视觉效果。然而，DMs 的全局去噪动态特性会将局部编辑目标与整幅图像上下文混淆在一起，导致非目标区域出现未预期的修改。为此，本文转向探讨遮罩生成变换器（MGTs），作为一种替代方法来应对这一挑战。通过预测多个被屏蔽的标记而非整体细化，MGT 展现了本地化解码范式，并因此具备在编辑过程中固有的能力，能够显式地保留非相关区域。 基于此见解，我们引入首个基于 MGT 的图像编辑框架——EditMGT。首先，我们证明了 MGT 的交叉注意力图提供了有关编辑相关区域定位的丰富信息信号，并设计了一种多层注意力整合方案以细化这些地图，从而实现精细化和精准化定位。在此基础上，我们提出了区域保持采样策略，在低关注度区域内限制标记翻转操作，抑制意外修改，使修改局限于目标区域并保留周围非目标区域的完整性。 为了训练 EditMGT，我们构建了 CrispEdit-2M 数据集，这是一个涵盖七类多样化编辑任务的高分辨率数据集。在不增加额外参数的情况下，通过注意力注入的方式将预训练的文字到图像 MGT 适配为图像编辑模型。跨四个标准基准进行的广泛实验表明，在少于10亿个参数的情况下，我们的模型实现了相似的表现水平，并使编辑速度提高了六倍。此外，它提供了相当或更好的编辑质量，风格变化任务和风格迁移任务分别提高了3.6% 和 17.6% 的性能。
URL
https://arxiv.org/abs/2512.11715
PDF
-
Lang2Motion: Bridging Language and Motion through Joint Embedding Spaces
2025-12-11 13:14:18Bishoy Galoaa, Xiangyu Bai, Sarah OstadabbasAbstract
We present Lang2Motion, a framework for language-guided point trajectory generation by aligning motion manifolds with joint embedding spaces. Unlike prior work focusing on human motion or video synthesis, we generate explicit trajectories for arbitrary objects using motion extracted from real-world videos via point tracking. Our transformer-based auto-encoder learns trajectory representations through dual supervision: textual motion descriptions and rendered trajectory visualizations, both mapped through CLIP's frozen encoders. Lang2Motion achieves 34.2% Recall@1 on text-to-trajectory retrieval, outperforming video-based methods by 12.5 points, and improves motion accuracy by 33-52% (12.4 ADE vs 18.3-25.3) compared to video generation baselines. We demonstrate 88.3% Top-1 accuracy on human action recognition despite training only on diverse object motions, showing effective transfer across motion domains. Lang2Motion supports style transfer, semantic interpolation, and latent-space editing through CLIP-aligned trajectory representations.
Abstract (translated)
我们提出了Lang2Motion，这是一个通过将运动流形与联合嵌入空间对齐来生成语言引导的点轨迹的框架。不同于先前专注于人体动作或视频合成的工作，我们利用从现实世界视频中提取并通过点跟踪获得的运动数据，为任意对象生成显式的轨迹。我们的基于变压器的自编码器通过双重监督学习轨迹表示：文本运动描述和渲染后的轨迹可视化，两者都通过CLIP的冻结编码器进行映射。 Lang2Motion在文本到轨迹检索上达到了34.2%的Recall@1，比视频基线方法高出12.5个百分点，并将动作准确性提高了33-52%，与视频生成基准相比（ADE值为12.4 vs 18.3-25.3）。尽管训练数据仅为多样化对象的动作，但我们在人类动作识别上达到了88.3%的Top-1准确率，证明了在不同运动领域之间的有效迁移。Lang2Motion支持风格转移、语义插值以及通过与CLIP对齐的轨迹表示进行潜在空间编辑。
URL
https://arxiv.org/abs/2512.10617
PDF
-
Topology-Agnostic Animal Motion Generation from Text Prompt
2025-12-11 07:08:29Keyi Chen, Mingze Sun, Zhenyu Liu, Zhangquan Chen, Ruqi HuangAbstract
Motion generation is fundamental to computer animation and widely used across entertainment, robotics, and virtual environments. While recent methods achieve impressive results, most rely on fixed skeletal templates, which prevent them from generalizing to skeletons with different or perturbed topologies. We address the core limitation of current motion generation methods - the combined lack of large-scale heterogeneous animal motion data and unified generative frameworks capable of jointly modeling arbitrary skeletal topologies and textual conditions. To this end, we introduce OmniZoo, a large-scale animal motion dataset spanning 140 species and 32,979 sequences, enriched with multimodal annotations. Building on OmniZoo, we propose a generalized autoregressive motion generation framework capable of producing text-driven motions for arbitrary skeletal topologies. Central to our model is a Topology-aware Skeleton Embedding Module that encodes geometric and structural properties of any skeleton into a shared token space, enabling seamless fusion with textual semantics. Given a text prompt and a target skeleton, our method generates temporally coherent, physically plausible, and semantically aligned motions, and further enables cross-species motion style transfer.
Abstract (translated)
运动生成是计算机动画的基础，并广泛应用于娱乐、机器人技术和虚拟环境等领域。尽管最近的方法取得了令人印象深刻的结果，但大多数方法依赖于固定的骨骼模板，这限制了它们对不同或受到干扰的拓扑结构的适应能力。我们解决了当前运动生成方法的核心局限性——缺乏大规模异构动物运动数据和统一的生成框架，这些框架能够同时建模任意骨骼拓扑和文本条件。为此，我们引入了OmniZoo，这是一个包含140种物种、共32,979个序列的大规模动物运动数据集，并且包含了多模式注释。基于OmniZoo，我们提出了一种通用的自回归运动生成框架，能够为任意骨骼拓扑生成文本驱动的动作。我们的模型核心是一个感知拓扑结构的骨架嵌入模块，该模块将任何骨骼的几何和结构性质编码到共享令牌空间中，从而能够无缝融合文本语义。在给定一个文本提示和目标骨骼后，我们的方法可以生成时间上连贯、物理上合理且语义对齐的动作，并进一步支持跨物种动作风格转换。
URL
https://arxiv.org/abs/2512.10352
PDF
-
Stylized Meta-Album: Group-bias injection with style transfer to study robustness against distribution shifts
2025-12-10 15:55:32Romain Mussard (UNIROUEN), Aur\'elien Gauffre (UGA), Ihsan Ullah (TAU, LISN), Thanh Gia Hieu Khuong (TAU, LISN), Massih-Reza Amini (UGA), Isabelle Guyon (TAU, LISN), Lisheng Sun-Hosoya (TAU, LISN)Abstract
We introduce Stylized Meta-Album (SMA), a new image classification meta-dataset comprising 24 datasets (12 content datasets, and 12 stylized datasets), designed to advance studies on out-of-distribution (OOD) generalization and related topics. Created using style transfer techniques from 12 subject classification datasets, SMA provides a diverse and extensive set of 4800 groups, combining various subjects (objects, plants, animals, human actions, textures) with multiple styles. SMA enables flexible control over groups and classes, allowing us to configure datasets to reflect diverse benchmark scenarios. While ideally, data collection would capture extensive group diversity, practical constraints often make this infeasible. SMA addresses this by enabling large and configurable group structures through flexible control over styles, subject classes, and domains-allowing datasets to reflect a wide range of real-world benchmark scenarios. This design not only expands group and class diversity, but also opens new methodological directions for evaluating model performance across diverse group and domain configurations-including scenarios with many minority groups, varying group imbalance, and complex domain shifts-and for studying fairness, robustness, and adaptation under a broader range of realistic conditions. To demonstrate SMA's effectiveness, we implemented two benchmarks: (1) a novel OOD generalization and group fairness benchmark leveraging SMA's domain, class, and group diversity to evaluate existing benchmarks. Our findings reveal that while simple balancing and algorithms utilizing group information remain competitive as claimed in previous benchmarks, increasing group diversity significantly impacts fairness, altering the superiority and relative rankings of algorithms. We also propose to use \textit{Top-M worst group accuracy} as a new hyperparameter tuning metric, demonstrating broader fairness during optimization and delivering better final worst-group accuracy for larger group diversity. (2) An unsupervised domain adaptation (UDA) benchmark utilizing SMA's group diversity to evaluate UDA algorithms across more scenarios, offering a more comprehensive benchmark with lower error bars (reduced by 73\% and 28\% in closed-set setting and UniDA setting, respectively) compared to existing efforts. These use cases highlight SMA's potential to significantly impact the outcomes of conventional benchmarks.
Abstract (translated)
我们介绍了Stylized Meta-Album（SMA），这是一个新的图像分类元数据集，包含24个数据集（12个内容数据集和12个风格化数据集），旨在推进出界分布（OOD）泛化的研究及相关话题。通过使用来自12个主题分类数据集的样式转换技术创建而成，SMA提供了一套多样且全面的4800组组合，将各种主题（对象、植物、动物、人类动作、纹理）与多种风格相结合。SMA允许对组和类进行灵活控制，使我们能够配置数据集以反映多样化基准测试场景。 虽然理想的数据收集会捕捉广泛的群体多样性，但实际限制通常使得这一目标难以实现。SMA通过对其风格、主题类别及领域进行灵活控制来解决此问题，从而建立大型且可配置的群体结构，让数据集能反映多种现实世界中的基准测试情景。这种设计不仅扩大了组和类别的多样性，还为评估模型在各种多样化的群组和领域设置下的表现打开了新的方法论方向，并可用于研究更多真实条件下公平性、鲁棒性和适应性的问题。 为了展示SMA的有效性，我们实施了两个基准测试：(1) 一个利用SMA的领域、类别及群体多样性进行OOD泛化和群体公平性评估的新基准；我们的发现表明，尽管简单的平衡以及使用群体信息的算法在现有基准中仍然具有竞争力，但增加群体多样性显著影响了公平性，改变了算法的优势及其相对排名。(2) 一项无监督域适应（UDA）基准测试，利用SMA的群组多样性来评价UDA算法跨更多场景的表现。与现有的努力相比，在封闭集设置和UniDA设置下，该基准测试提供了一个更全面的评估，误差范围降低了73% 和 28%，展示了SMA在扩大群体多样性的优化过程中实现更广泛公平性，并最终提高了最差群体准确性。 这些应用场景突显了SMA对传统基准结果具有显著影响的潜力。
URL
https://arxiv.org/abs/2512.09773
PDF
