Style_Transfer
-
How to Guide Your Flow: Few-Step Alignment via Flow Map Reward Guidance
2026-04-29 19:56:53Jerry Y. Huang, Justin Lin, Sheel Shah, Kartik Nair, Nicholas M. BoffiAbstract
In generative modeling, we often wish to produce samples that maximize a user-specified reward such as aesthetic quality or alignment with human preferences, a problem known as guidance. Despite their widespread use, existing guidance methods either require expensive multi-particle, many-step schemes or rely on poorly understood approximations. We reformulate guidance as a deterministic optimal control problem, yielding a hierarchy of algorithms that subsumes existing approaches at the coarsest level. We show that the flow map, an object of significant recent interest for its role in fast inference, arises naturally in the optimal solution. Based on this observation, we propose Flow Map Reward Guidance (FMRG): a training-free, single-trajectory framework that uses the flow map to both integrate and guide the flow. At text-to-image scale, FMRG matches or surpasses baselines across inverse problems, style transfer, human preferences, and VLM rewards with as few as 3 NFEs, giving at least an order-of-magnitude speedup in comparison to prior state of the art.
Abstract (translated)
URL
https://arxiv.org/abs/2604.27147
PDF
-
Text Style Transfer with Machine Translation for Graphic Designs
2026-04-29 07:16:36Deergh Singh Budhauria, Sanyam Jain, Rishav Agarwal, Tracy KingAbstract
Globalization of graphic designs such as those used in marketing materials and magazines is increasingly important for communication to broad audiences. To accomplish this, the textual content in the graphic designs needs to be accurately translated and have the text styling preserved in order to fit visually into the design. Preserving text styling requires high accuracy word alignment between the original and the translated text. The problem of word alignment between source and translated text is long known. The industry standards for extracting word alignments are defined by Giza++ and attention probabilities from neural machine translation (NMT) models. In this paper, we explore three new methods to tackle the word alignment problem for transferring text styles from the source to the translated text. The proposed methods are developed on top of commercially available NMT and LLM translation technologies. They include: NMT with custom input and output tags for text styling; LLM with custom input and output tags; a hybrid with NMT for translation followed by an LLM with use of unigram mappings. To analyze the performance of these solutions, their alignment results are compared with the results of an attention head approach to gauge their usability in graphic design applications. Interestingly, the attention head strong baseline proves more accurate than the LLM or NMT approach and on par with the hybrid NMT+LLM approach.
Abstract (translated)
URL
https://arxiv.org/abs/2604.26361
PDF
-
Why Domain Matters: A Preliminary Study of Domain Effects in Underwater Object Detection
2026-04-28 23:28:14Melanie Wille, Dimity Miller, Tobias Fischer, Scarlett RaineAbstract
Domain shift, where deviations between training and deployment data distributions degrade model performance, is a key challenge in underwater environments. Existing benchmarks testing performance for underwater domain shift simulate variability through synthetic style transfer. This fails to capture intrinsic scene factors such as visibility, illumination, scene composition, or acquisition factors, limiting analysis of real-world effects. We propose a labeling framework that defines underwater domains using measurable image, scene, and acquisition characteristics. Unlike prior benchmarks, it captures physically meaningful factors, enabling semantically consistent image grouping and supporting domain-specific evaluation of detection performance including failure analysis. We validate this on public datasets, showing systematic variations across domain factors and revealing hidden failure modes.
Abstract (translated)
URL
https://arxiv.org/abs/2604.26174
PDF
-
StyleVAR: Controllable Image Style Transfer via Visual Autoregressive Modeling
2026-04-22 19:52:35Liqi Jing, Dingming Zhang, Peinian Li, Lichen ZhuAbstract
We build on the Visual Autoregressive Modeling (VAR) framework and formulate style transfer as conditional discrete sequence modeling in a learned latent space. Images are decomposed into multi-scale representations and tokenized into discrete codes by a VQ-VAE; a transformer then autoregressively models the distribution of target tokens conditioned on style and content tokens. To inject style and content information, we introduce a blended cross-attention mechanism in which the evolving target representation attends to its own history, while style and content features act as queries that decide which aspects of this history to emphasize. A scale-dependent blending coefficient controls the relative influence of style and content at each stage, encouraging the synthesized representation to align with both the content structure and the style texture without breaking the autoregressive continuity of VAR. We train StyleVAR in two stages from a pretrained VAR checkpoint: supervised fine-tuning on a large triplet dataset of content--style--target images, followed by reinforcement fine-tuning with Group Relative Policy Optimization (GRPO) against a DreamSim-based perceptual reward, with per-action normalization weighting to rebalance credit across VAR's multi-scale hierarchy. Across three benchmarks spanning in-, near-, and out-of-distribution regimes, StyleVAR consistently outperforms an AdaIN baseline on Style Loss, Content Loss, LPIPS, SSIM, DreamSim, and CLIP similarity, and the GRPO stage yields further gains over the SFT checkpoint, most notably on the reward-aligned perceptual metrics. Qualitatively, the method transfers texture while maintaining semantic structure, especially for landscapes and architectural scenes, while a generalization gap on internet images and difficulty with human faces highlight the need for better content diversity and stronger structural priors.
Abstract (translated)
URL
https://arxiv.org/abs/2604.21052
PDF
-
Style-Based Neural Architectures for Real-Time Weather Classification
2026-04-20 13:28:24Hamed Ouattara, Pascal Houssam Salmane, Pierre Duthon, Fr\'ed\'eric Bernardin, Omar Ait AiderAbstract
In this paper, we present three neural network architectures designed for real-time classification of weather conditions (sunny, rain, snow, fog) from images. These models, inspired by recent advances in style transfer, aim to capture the stylistic elements present in images. One model, called "Multi-PatchGAN", is based on PatchGANs used in well-known architectures such as Pix2Pix and CycleGAN, but here adapted with multiple patch sizes for detection tasks. The second model, "Truncated ResNet50", is a simplified version of ResNet50 retaining only its first nine layers. This truncation, determined by an evolutionary algorithm, facilitates the extraction of high-frequency features essential for capturing subtle stylistic details. Finally, we propose "Truncated ResNet50 with Gram Matrix and Attention", which computes Gram matrices for each layer during training and automatically weights them via an attention mechanism, thus optimizing the extraction of the most relevant stylistic expressions for classification. These last two models outperform the state of the art and demonstrate remarkable generalization capability on several public databases. Although developed for weather detection, these architectures are also suitable for other appearance-based classification tasks, such as animal species recognition, texture classification, disease detection in medical imaging, or industrial defect identification.
Abstract (translated)
URL
https://arxiv.org/abs/2604.18251
PDF
-
UniCSG: Unified High-Fidelity Content-Constrained Style-Driven Generation via Staged Semantic and Frequency Disentanglement
2026-04-20 05:59:20Jingwei Yang, Ruoxi Wu, Wei Shen, Meng Li, Yulong Liu, Huimin She, Lunxi YuanAbstract
Style transfer must match a target style while preserving content semantics. DiT-based diffusion models often suffer from content-style entanglement, leading to reference-content leakage and unstable generation. We present UniCSG, a unified framework for content-constrained, style-driven generation in both text-guided and reference-guided settings. UniCSG employs staged training: (i) a latent-space semantic disentanglement stage that combines low-frequency preprocessing with conditioning corruption to encourage content-style separation, and (ii) a latent-space frequency-aware detail reconstruction stage that refines details via multi-scale frequency supervision. We further incorporate pixel-space reward learning to align latent objectives with perceptual quality after decoding. Experiments demonstrate improved content faithfulness, style alignment, and robustness in both settings.
Abstract (translated)
URL
https://arxiv.org/abs/2604.17850
PDF
-
Towards In-Context Tone Style Transfer with A Large-Scale Triplet Dataset
2026-04-17 14:49:51Yuhai Deng, Huimin She, Wei Shen, Meng Li, Ruoxi Wu, Lunxi Yuan, Xiang LiAbstract
Tone style transfer for photo retouching aims to adapt the stylistic tone of the reference image to a given content image. However, the lack of high-quality large-scale triplet datasets with stylized ground truth forces existing methods to rely on self-supervised or proxy objectives, which limits model capability. To mitigate this gap, we design a data construction pipeline to build TST100K, a large-scale dataset of 100,000 content-reference-stylized triplets. At the core of this pipeline, we train a tone style scorer to ensure strict stylistic consistency for each triplet. In addition, existing methods typically extract content and reference features independently and then fuse them in a decoder, which may cause semantic loss and lead to inappropriate color transfer and degraded visual aesthetics. Instead, we propose ICTone, a diffusion-based framework that performs tone transfer in an in-context manner by jointly conditioning on both images, leveraging the semantic priors of generative models for semantic-aware transfer. Reward feedback learning using the tone style scorer is further incorporated to improve stylistic fidelity and visual quality. Experiments demonstrate the effectiveness of TST100K, and ICTone achieves state-of-the-art performance on both quantitative metrics and human evaluations.
Abstract (translated)
URL
https://arxiv.org/abs/2604.16114
PDF
-
Beyond Prompts: Unconditional 3D Inversion for Out-of-Distribution Shapes
2026-04-16 11:55:29Victoria Yue Chen, Emery Pierson, L\'eopold Maillard, Maks OvsjanikovAbstract
Text-driven inversion of generative models is a core paradigm for manipulating 2D or 3D content, unlocking numerous applications such as text-based editing, style transfer, or inverse problems. However, it relies on the assumption that generative models remain sensitive to natural language prompts. We demonstrate that for state-of-the-art native text-to-3D generative models, this assumption often collapses. We identify a critical failure mode where generation trajectories are drawn into latent ``sink traps'': regions where the model becomes insensitive to prompt modifications. In these regimes, changes to the input text fail to alter internal representations in a way that alters the output geometry. Crucially, we observe that this is not a limitation of the model's \textit{geometric} expressivity; the same generative models possess the ability to produce a vast diversity of shapes but, as we demonstrate, become insensitive to out-of-distribution \textit{text} guidance. We investigate this behavior by analyzing the sampling trajectories of the generative model, and find that complex geometries can still be represented and produced by leveraging the model's unconditional generative prior. This leads to a more robust framework for text-based 3D shape editing that bypasses latent sinks by decoupling a model's geometric representation power from its linguistic sensitivity. Our approach addresses the limitations of current 3D pipelines and enables high-fidelity semantic manipulation of out-of-distribution 3D shapes. Project webpage: this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2604.14914
PDF
-
Heuristic Style Transfer for Real-Time, Efficient Weather Attribute Detection
2026-04-15 15:01:29Hamed Ouattara, Pierre Duthon, Pascal Houssam Salmane, Fr\'ed\'eric Bernardin, Omar Ait AiderAbstract
We present lightweight and efficient architectures to detect weather conditions from RGB images, predicting the weather type (sunny, rain, snow, fog) and 11 complementary attributes such as intensity, visibility, and ground condition, for a total of 53 classes across the tasks. This work examines to what extent weather conditions manifest as variations in visual style. We investigate style-inspired techniques, including Gram matrices, a truncated ResNet-50 targeting lower and intermediate layers, and PatchGAN-style architectures, within a multi-task framework with attention mechanisms. Two families are introduced: RTM (ResNet50-Truncated-MultiTasks) and PMG (PatchGAN-MultiTasks-Gram), together with their variants. Our contributions include automation of Gram-matrix computation, integration of PatchGAN into supervised multi-task learning, and local style capture through local Gram for improved spatial coherence. We also release a dataset of 503,875 images annotated with 12 weather attributes under a Creative Commons Attribution (CC-BY) license. The models achieve F1 scores above 96 percent on our internal test set and above 78 percent in zero-shot evaluation on several external datasets, confirming their generalization ability. The PMG architecture, with fewer than 5 million parameters, runs in real time with a small memory footprint, making it suitable for embedded systems. The modular design of the models also allows style-related or weather-related tasks to be added or removed as needed.
Abstract (translated)
URL
https://arxiv.org/abs/2604.13947
PDF
-
MAST: Mask-Guided Attention Mass Allocation for Training-Free Multi-Style Transfer
2026-04-14 04:47:09Dongkyung Kang, Jaeyeon Hwang, Junseo Park, Minji Kang, Yeryeong Lee, Beomseok Ko, Hanyoung Roh, Jeongmin Shin, Hyeryung JangAbstract
Style transfer aims to render a content image with the visual characteristics of a reference style while preserving its underlying semantic layout and structural geometry. While recent diffusion-based models demonstrate strong stylization capabilities by leveraging powerful generative priors and controllable internal representations, they typically assume a single global style. Extending them to multi-style scenarios often leads to boundary artifacts, unstable stylization, and structural inconsistency due to interference between multiple style representations. To overcome these limitations, we propose MAST (Mask-Guided Attention Mass Allocation for Training-Free Multi-Style Transfer), a novel training-free framework that explicitly controls content-style interactions within the diffusion attention mechanism. To achieve artifact-free and structure-preserving stylization, MAST integrates four connected modules. First, Layout-preserving Query Anchoring prevents global layout collapse by firmly anchoring the semantic structure using content queries. Second, Logit-level Attention Mass Allocation deterministically distributes attention probability mass across spatial regions, seamlessly fusing multiple styles without boundary artifacts. Third, Sharpness-aware Temperature Scaling restores the attention sharpness degraded by multi-style expansion. Finally, Discrepancy-aware Detail Injection adaptively compensates for localized high-frequency detail losses by measuring structural discrepancies. Extensive experiments demonstrate that MAST effectively mitigates boundary artifacts and maintains structural consistency, preserving texture fidelity and spatial coherence even as the number of applied styles increases.
Abstract (translated)
URL
https://arxiv.org/abs/2604.12281
PDF
-
Please Make it Sound like Human: Encoder-Decoder vs. Decoder-Only Transformers for AI-to-Human Text Style Transfer
Abstract
AI-generated text has become common in academic and professional writing, prompting research into detection methods. Less studied is the reverse: systematically rewriting AI-generated prose to read as genuinely human-authored. We build a parallel corpus of 25,140 paired AI-input and human-reference text chunks, identify 11 measurable stylistic markers separating the two registers, and fine-tune three models: BART-base, BART-large, and Mistral-7B-Instruct with QLoRA. BART-large achieves the highest reference similarity -- BERTScore F1 of 0.924, ROUGE-L of 0.566, and chrF++ of 55.92 -- with 17x fewer parameters than Mistral-7B. We show that Mistral-7B's higher marker shift score reflects overshoot rather than accuracy, and argue that shift accuracy is a meaningful blind spot in current style transfer evaluation.
Abstract (translated)
URL
https://arxiv.org/abs/2604.11687
PDF
-
FREE-Switch: Frequency-based Dynamic LoRA Switch for Style Transfer
2026-04-11 04:32:59Shenghe Zheng, Minyu Zhang, Tianhao Liu, Hongzhi WangAbstract
With the growing availability of open-sourced adapters trained on the same diffusion backbone for diverse scenes and objects, combining these pretrained weights enables low-cost customized generation. However, most existing model merging methods are designed for classification or text generation, and when applied to image generation, they suffer from content drift due to error accumulation across multiple diffusion steps. For image-oriented methods, training-based approaches are computationally expensive and unsuitable for edge deployment, while training-free ones use uniform fusion strategies that ignore inter-adapter differences, leading to detail degradation. We find that since different adapters are specialized for generating different types of content, the contribution of each diffusion step carries different significance for each adapter. Accordingly, we propose a frequency-domain importance-driven dynamic LoRA switch method. Furthermore, we observe that maintaining semantic consistency across adapters effectively mitigates detail loss; thus, we design an automatic Generation Alignment mechanism to align generation intents at the semantic level. Experiments demonstrate that our FREE-Switch (Frequency-based Efficient and Dynamic LoRA Switch) framework efficiently combines adapters for different objects and styles, substantially reducing the training cost of high-quality customized generation.
Abstract (translated)
URL
https://arxiv.org/abs/2604.10023
PDF
-
MegaStyle: Constructing Diverse and Scalable Style Dataset via Consistent Text-to-Image Style Mapping
2026-04-09 15:29:08Junyao Gao, Sibo Liu, Jiaxing Li, Yanan Sun, Yuanpeng Tu, Fei Shen, Weidong Zhang, Cairong Zhao, Jun ZhangAbstract
In this paper, we introduce MegaStyle, a novel and scalable data curation pipeline that constructs an intra-style consistent, inter-style diverse and high-quality style dataset. We achieve this by leveraging the consistent text-to-image style mapping capability of current large generative models, which can generate images in the same style from a given style description. Building on this foundation, we curate a diverse and balanced prompt gallery with 170K style prompts and 400K content prompts, and generate a large-scale style dataset MegaStyle-1.4M via content-style prompt combinations. With MegaStyle-1.4M, we propose style-supervised contrastive learning to fine-tune a style encoder MegaStyle-Encoder for extracting expressive, style-specific representations, and we also train a FLUX-based style transfer model MegaStyle-FLUX. Extensive experiments demonstrate the importance of maintaining intra-style consistency, inter-style diversity and high-quality for style dataset, as well as the effectiveness of the proposed MegaStyle-1.4M. Moreover, when trained on MegaStyle-1.4M, MegaStyle-Encoder and MegaStyle-FLUX provide reliable style similarity measurement and generalizable style transfer, making a significant contribution to the style transfer community. More results are available at our project website this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2604.08364
PDF
-
Evaluation of Randomization through Style Transfer for Enhanced Domain Generalization
2026-04-07 09:05:45Dustin Eisenhardt, Timothy Schauml\"offel, Alperen Kantarci, Gemma RoigAbstract
Deep learning models for computer vision often suffer from poor generalization when deployed in real-world settings, especially when trained on synthetic data due to the well-known Sim2Real gap. Despite the growing popularity of style transfer as a data augmentation strategy for domain generalization, the literature contains unresolved contradictions regarding three key design axes: the diversity of the style pool, the role of texture complexity, and the choice of style source. We present a systematic empirical study that isolates and evaluates each of these factors for driving scene understanding, resolving inconsistencies in prior work. Our findings show that (i) expanding the style pool yields larger gains than repeated augmentation with few styles, (ii) texture complexity has no significant effect when the pool is sufficiently large, and (iii) diverse artistic styles outperform domain-aligned alternatives. Guided by these insights, we derive StyleMixDG (Style-Mixing for Domain Generalization), a lightweight, model-agnostic augmentation recipe that requires no architectural modifications or additional losses. Evaluated on the GTAV $\rightarrow$ {BDD100k, Cityscapes, Mapillary Vistas} benchmark, StyleMixDG demonstrates consistent improvements over strong baselines, confirming that the empirically identified design principles translate into practical gains. The code will be released on GitHub.
Abstract (translated)
计算机视觉深度学习模型在真实世界场景中部署时，常因Sim2Real差距（合成数据与真实数据间的分布差异）而面临泛化能力不足的问题。尽管风格迁移作为领域泛化的数据增强策略日益流行，但关于三个关键设计维度——风格池多样性、纹理复杂度的作用以及风格源的选择——学界仍存在未解决的矛盾。本研究通过系统性实证分析，分离并评估了这些因素对驾驶场景理解任务的影响，解决了先前工作中的矛盾。研究发现：(i) 扩大风格池比重复使用少量风格带来更大提升；(ii) 当风格池足够大时，纹理复杂度的影响不显著；(iii) 多样化艺术风格优于领域对齐的替代方案。基于这些洞见，我们提出了StyleMixDG（面向领域泛化的风格混合），一种轻量级、模型无关的数据增强方案，无需修改网络结构或增加损失函数。在GTAV→{BDD100k, Cityscapes, Mapillary Vistas}基准测试中，StyleMixDG相较于强基线持续取得改进，验证了实证设计原则可转化为实际性能提升。代码将在GitHub发布。
URL
https://arxiv.org/abs/2604.05616
PDF
-
M2StyleGS: Multi-Modality 3D Style Transfer with Gaussian Splatting
2026-04-04 15:51:56Xingyu Miao, Xueqi Qiu, Haoran Duan, Yawen Huang, Xian Wu, Jingjing Deng, Yang LongAbstract
Conventional 3D style transfer methods rely on a fixed reference image to apply artistic patterns to 3D scenes. However, in practical applications such as virtual or augmented reality, users often prefer more flexible inputs, including textual descriptions and diverse imagery. In this work, we introduce a novel real-time styling technique M2StyleGS to generate a sequence of precisely color-mapped views. It utilizes 3D Gaussian Splatting (3DGS) as a 3D presentation and multi-modality knowledge refined by CLIP as a reference style. M2StyleGS resolves the abnormal transformation issue by employing a precise feature alignment, namely subdivisive flow, it strengthens the projection of the mapped CLIP text-visual combination feature to the VGG style feature. In addition, we introduce observation loss, which assists in the stylized scene better matching the reference style during the generation, and suppression loss, which suppresses the offset of reference color information throughout the decoding process. By integrating these approaches, M2StyleGS can employ text or images as references to generate a set of style-enhanced novel views. Our experiments show that M2StyleGS achieves better visual quality and surpasses the previous work by up to 32.92% in terms of consistency.
Abstract (translated)
传统3D风格迁移方法依赖固定的参考图像将艺术图案应用于3D场景。然而，在虚拟现实或增强现实等实际应用中，用户往往更倾向于使用更灵活的输入方式，包括文本描述和多样化的图像。本研究提出了一种新型实时风格化技术M2StyleGS，用于生成一系列精确着色的视图。该方法以3D高斯泼溅（3DGS）作为三维表征，并利用经CLIP模型提炼的多模态知识作为参考风格。M2StyleGS通过采用精确的特征对齐技术——细分流（subdivisive flow），解决了异常变换问题，同时强化了映射后的CLIP文本-视觉组合特征向VGG风格特征的投影。此外，我们引入了观察损失（observation loss），辅助风格化场景在生成过程中更好地匹配参考风格；并采用抑制损失（suppression loss），在解码全程抑制参考色彩信息的偏移。通过整合这些方法，M2StyleGS能够以文本或图像作为参考，生成一组风格增强的新视角视图。实验表明，M2StyleGS在视觉质量上表现更优，并在一致性指标上较先前工作最高提升32.92%。
URL
https://arxiv.org/abs/2604.03773
PDF
-
Abstraction in Style
2026-03-31 15:59:55Min Lu, Yuanfeng He, Anthony Chen, Jianhuang He, Pu Wang, Daniel Cohen-Or, Hui HuangAbstract
Artistic styles often embed abstraction beyond surface appearance, involving deliberate reinterpretation of structure rather than mere changes in texture or color. Conventional style transfer methods typically preserve the input geometry and therefore struggle to capture this deeper abstraction behavior, especially for illustrative and nonphotorealistic styles. In this work, we introduce Abstraction in Style (AiS), a generative framework that separates structural abstraction from visual stylization. Given a target image and a small set of style exemplars, AiS first derives an intermediate abstraction proxy that reinterprets the target's structure in accordance with the abstraction logic exhibited by the style. The proxy captures semantic structure while relaxing geometric fidelity, enabling subsequent stylization to operate on an abstracted representation rather than the original image. In a second stage, the abstraction proxy is rendered to produce the final stylized output, preserving visual coherence with the reference style. Both stages are implemented using a shared image space analogy, enabling transformations to be learned from visual exemplars without explicit geometric supervision. By decoupling abstraction from appearance and treating abstraction as an explicit, transferable process, AiS supports a wider range of stylistic transformations, improves controllability, and enables more expressive stylization.
Abstract (translated)
艺术风格往往蕴含超越表层的抽象，涉及对结构的有意重构，而非简单的纹理或色彩变化。传统风格迁移方法通常保留输入图像的几何结构，因此难以捕捉这种深层的抽象特性，尤其对于插画与非真实感风格。本研究提出一种名为“风格中的抽象”（Abstraction in Style, AiS）的生成框架，该框架将结构抽象与视觉风格化分离。给定目标图像与少量风格范例后，AiS首先生成一个中间抽象代理，该代理依据风格所展现的抽象逻辑对目标结构进行重构。此代理在保留语义结构的同时放宽几何保真度，使后续风格化操作得以在抽象化表示而非原始图像上进行。第二阶段中，抽象代理被渲染以生成最终风格化输出，同时保持与参考风格的视觉一致性。两个阶段均通过共享图像空间类比实现，使变换能够从视觉范例中学习，而无需显式的几何监督。通过将抽象与外观解耦，并将抽象视为一个可显式转移的过程，AiS支持更广泛的风格转换，提升可控性，并实现更具表现力的风格化效果。
URL
https://arxiv.org/abs/2603.29924
PDF
-
SceneExpander: Expanding 3D Scenes with Free-Form Inserted Views
2026-03-28 02:04:48Zijian He, enjie Liu, Yihao Wang, Weizhi Zhong, Huan Yuan, Kun Gai, Guangrun Wang, Guanbin LiAbstract
World building with 3D scene representations is increasingly important for content creation, simulation, and interactive experiences, yet real workflows are inherently iterative: creators must repeatedly extend an existing scene under user control. Motivated by this research gap, we study 3D scene expansion in a user-centric workflow: starting from a real scene captured by multi-view images, we extend its coverage by inserting an additional view synthesized by a generative model. Unlike simple object editing or style transfer in a fixed scene, the inserted view is often 3D-misaligned with the original reconstruction, introducing geometry shifts, hallucinated content, or view-dependent artifacts that break global multi-view consistency. To address the challenge, we propose SceneExpander, which applies test-time adaptation to a parametric feed-forward 3D reconstruction model with two complementary distillation signals: anchor distillation stabilizes the original scene by distilling geometric cues from the captured views, while inserted-view self-distillation preserves observation-supported predictions yet adapts latent geometry and appearance to accommodate the misaligned inserted view. Experiments on ETH scenes and online data demonstrate improved expansion behavior and reconstruction quality under misalignment.
Abstract (translated)
基于3D场景表征的世界构建对于内容创作、仿真和交互体验愈发重要，但实际工作流本质上是迭代的：创作者必须在用户控制下反复扩展现有场景。受此研究空白驱动，我们在以用户为中心的工作流中研究3D场景扩展：从多视角图像捕捉的真实场景出发，通过插入由生成模型合成的额外视图来扩展其覆盖范围。与固定场景中的简单对象编辑或风格迁移不同，插入的视图通常与原始重建存在3D错位，引入几何偏移、幻觉内容或视角相关伪影，从而破坏全局多视角一致性。为应对这一挑战，我们提出SceneExpander，该方法对参数化前馈3D重建模型应用测试时适应，并利用两种互补的蒸馏信号：锚点蒸馏通过从捕捉的视图中蒸馏几何线索来稳定原始场景，而插入视图自蒸馏则保留有观测支持的预测，同时适应潜在几何和外观以容纳错位的插入视图。在ETH场景和在线数据上的实验表明，该方法在错位条件下改善了扩展行为和重建质量。
URL
https://arxiv.org/abs/2603.27084
PDF
-
Adapting Frozen Mono-modal Backbones for Multi-modal Registration via Contrast-Agnostic Instance Optimization
2026-03-27 13:21:19Yi Zhang, Yidong Zhao, Qian TaoAbstract
Deformable image registration remains a central challenge in medical image analysis, particularly under multi-modal scenarios where intensity distributions vary significantly across scans. While deep learning methods provide efficient feed-forward predictions, they often fail to generalize robustly under distribution shifts at test time. A straightforward remedy is full network fine-tuning, yet for modern architectures such as Transformers or deep U-Nets, this adaptation is prohibitively expensive in both memory and runtime when operating in 3D. Meanwhile, the naive fine-tuning struggles more with potential degradation in performance in the existence of drastic domain shifts. In this work, we propose a registration framework that integrates a frozen pretrained \textbf{mono-modal} registration model with a lightweight adaptation pipeline for \textbf{multi-modal} image registration. Specifically, we employ style transfer based on contrast-agnostic representation generation and refinement modules to bridge modality and domain gaps with instance optimization at test time. This design is orthogonal to the choice of backbone mono-modal model, thus avoids the computational burden of full fine-tuning while retaining the flexibility to adapt to unseen domains. We evaluate our approach on the Learn2Reg 2025 LUMIR validation set and observe consistent improvements over the pretrained state-of-the-art mono-modal backbone. In particular, the method ranks second on the multi-modal subset, third on the out-of-domain subset, and achieves fourth place overall in Dice score. These results demonstrate that combining frozen mono-modal models with modality adaptation and lightweight instance optimization offers an effective and practical pathway toward robust multi-modal registration.
Abstract (translated)
URL
https://arxiv.org/abs/2603.26393
PDF
-
HAM: A Training-Free Style Transfer Approach via Heterogeneous Attention Modulation for Diffusion Models
2026-03-25 07:52:30Yeqi He, Liang Li, Zhiwen Yang, Xichun Sheng, Zhidong Zhao, Chenggang YanAbstract
Diffusion models have demonstrated remarkable performance in image generation, particularly within the domain of style transfer. Prevailing style transfer approaches typically leverage pre-trained diffusion models' robust feature extraction capabilities alongside external modular control pathways to explicitly impose style guidance signals. However, these methods often fail to capture complex style reference or retain the identity of user-provided content images, thus falling into the trap of style-content balance. Thus, we propose a training-free style transfer approach via $\textbf{h}$eterogeneous $\textbf{a}$ttention $\textbf{m}$odulation ($\textbf{HAM}$) to protect identity information during image/text-guided style reference transfer, thereby addressing the style-content trade-off challenge. Specifically, we first introduces style noise initialization to initialize latent noise for diffusion. Then, during the diffusion process, it innovatively employs HAM for different attention mechanisms, including Global Attention Regulation (GAR) and Local Attention Transplantation (LAT), which better preserving the details of the content image while capturing complex style references. Our approach is validated through a series of qualitative and quantitative experiments, achieving state-of-the-art performance on multiple quantitative metrics.
Abstract (translated)
扩散模型在图像生成领域展现出卓越性能，尤其在风格迁移任务中表现突出。现有风格迁移方法通常利用预训练扩散模型强大的特征提取能力，结合外部模块化控制路径来显式施加风格引导信号。然而，这些方法往往难以捕捉复杂风格参考或保留用户提供内容图像的身份特征，从而陷入风格-内容平衡的困境。因此，我们提出一种无需训练的异质注意力调制（HAM）风格迁移方法，以在图像/文本引导的风格参考迁移过程中保护身份信息，从而解决风格-内容权衡挑战。具体而言，我们首先引入风格噪声初始化来为扩散过程初始化潜在噪声。随后在扩散过程中，创新性地对不同的注意力机制采用HAM，包括全局注意力调控（GAR）与局部注意力移植（LAT），在捕捉复杂风格参考的同时更好地保留内容图像的细节。通过一系列定性与定量实验验证了本方法，在多项定量指标上达到了最先进性能。
URL
https://arxiv.org/abs/2603.24043
PDF
-
FHAvatar: Fast and High-Fidelity Reconstruction of Face-and-Hair Composable 3D Head Avatar from Few Casual Captures
2026-03-24 15:43:54Yujie Sun, Zhuoqiang Cai, Chaoyue Niu, Jianchuan Chen, Zhiwen Chen, Chengfei Lv, Fan WuAbstract
We present FHAvatar, a novel framework for reconstructing 3D Gaussian avatars with composable face and hair components from an arbitrary number of views. Unlike previous approaches that couple facial and hair representations within a unified modeling process, we explicitly decouple two components in texture space by representing the face with planar Gaussians and the hair with strand-based Gaussians. To overcome the limitations of existing methods that rely on dense multi-view captures or costly per-identity optimization, we propose an aggregated transformer backbone to learn geometry-aware cross-view priors and head-hair structural coherence from multi-view datasets, enabling effective and efficient feature extraction and fusion from few casual captures. Extensive quantitative and qualitative experiments demonstrate that FHAvatar achieves state-of-the-art reconstruction quality from only a few observations of new identities within minutes, while supporting real-time animation, convenient hairstyle transfer, and stylized editing, broadening the accessibility and applicability of digital avatar creation.
Abstract (translated)
我们提出了FHAvatar，一种新颖的框架，可从任意数量的视角中重建具有可组合面部与发丝组件的三维高斯虚拟形象。与以往在统一建模过程中耦合面部与发丝表征的方法不同，我们通过在纹理空间中用平面高斯表示面部、用基于发丝的高斯表示发丝，显式解耦这两个组件。为克服现有方法依赖密集多视角采集或昂贵逐身份优化的局限，我们提出聚合Transformer主干网络，从多视角数据集中学习几何感知的跨视角先验与头-发结构一致性，从而实现对少量 casually captured 数据的高效特征提取与融合。大量定量与定性实验表明，FHAvatar在数分钟内即可从新身份的少量观测中实现最先进的重建质量，同时支持实时动画、便捷发型迁移与风格化编辑，拓宽了数字虚拟形象创建的普及性与应用范围。
URL
https://arxiv.org/abs/2603.23345
PDF
