Inpainting
-
How Do Inpainting Artifacts Propagate to Language?
2026-02-24 03:46:33Pratham Yashwante, Davit Abrahamyan, Shresth Grover, Sukruth RaoAbstract
We study how visual artifacts introduced by diffusion-based inpainting affect language generation in vision-language models. We use a two-stage diagnostic setup in which masked image regions are reconstructed and then provided to captioning models, enabling controlled comparisons between captions generated from original and reconstructed inputs. Across multiple datasets, we analyze the relationship between reconstruction fidelity and downstream caption quality. We observe consistent associations between pixel-level and perceptual reconstruction metrics and both lexical and semantic captioning performance. Additional analysis of intermediate visual representations and attention patterns shows that inpainting artifacts lead to systematic, layer-dependent changes in model behavior. Together, these results provide a practical diagnostic framework for examining how visual reconstruction quality influences language generation in multimodal systems.
Abstract (translated)
我们研究了基于扩散的图像修复过程中引入的视觉伪影如何影响视觉-语言模型中的语言生成。为此，我们采用了一个两阶段诊断设置，在该设置中，被遮挡的图像区域会被重建并提供给描述模型，从而可以对从原始和重构输入生成的描述进行有控制的比较。在多个数据集上，我们分析了重建保真度与下游描述质量之间的关系。我们观察到像素级和感知重建指标与词汇和语义描述性能之间存在一致的相关性。对于中间视觉表示和注意力模式的进一步分析表明，修复伪影会导致模型行为发生系统性和层级相关的改变。综合起来，这些结果提供了一个实用的诊断框架，用于考察视觉重构质量如何影响多模态系统中的语言生成。
URL
https://arxiv.org/abs/2602.20520
PDF
-
HDR Reconstruction Boosting with Training-Free and Exposure-Consistent Diffusion
2026-02-23 10:57:22Yo-Tin Lin, Su-Kai Chen, Hou-Ning Hu, Yen-Yu Lin, Yu-Lun LiuAbstract
Single LDR to HDR reconstruction remains challenging for over-exposed regions where traditional methods often fail due to complete information loss. We present a training-free approach that enhances existing indirect and direct HDR reconstruction methods through diffusion-based inpainting. Our method combines text-guided diffusion models with SDEdit refinement to generate plausible content in over-exposed areas while maintaining consistency across multi-exposure LDR images. Unlike previous approaches requiring extensive training, our method seamlessly integrates with existing HDR reconstruction techniques through an iterative compensation mechanism that ensures luminance coherence across multiple exposures. We demonstrate significant improvements in both perceptual quality and quantitative metrics on standard HDR datasets and in-the-wild captures. Results show that our method effectively recovers natural details in challenging scenarios while preserving the advantages of existing HDR reconstruction pipelines. Project page: this https URL
Abstract (translated)
单次LDR到HDR的重建对于过度曝光区域而言仍然具有挑战性，因为传统方法往往由于信息完全丢失而失败。我们提出了一种无需训练的方法，通过基于扩散的修复技术增强现有的间接和直接HDR重建方法。我们的方法结合了文本引导的扩散模型与SDEdit细化技术，在保持多曝光LDR图像之间一致性的同时，生成过度曝光区域中的合理内容。不同于以往需要大量训练的方法，我们的方法通过迭代补偿机制无缝地整合到现有HDR重建技术中，并确保不同曝光之间的亮度连贯性。 我们在标准HDR数据集和现实场景拍摄的照片上展示了感知质量和量化指标的显著改进。结果显示，该方法在具有挑战性的场景下有效恢复自然细节的同时，还保留了现有HDR重建流程的优势。项目页面：[请参阅链接以获取更多信息](https://this https URL)
URL
https://arxiv.org/abs/2602.19706
PDF
-
Vinedresser3D: Agentic Text-guided 3D Editing
2026-02-23 06:30:36Yankuan Chi, Xiang Li, Zixuan Huang, James M. RehgAbstract
Text-guided 3D editing aims to modify existing 3D assets using natural-language instructions. Current methods struggle to jointly understand complex prompts, automatically localize edits in 3D, and preserve unedited content. We introduce Vinedresser3D, an agentic framework for high-quality text-guided 3D editing that operates directly in the latent space of a native 3D generative model. Given a 3D asset and an editing prompt, Vinedresser3D uses a multimodal large language model to infer rich descriptions of the original asset, identify the edit region and edit type (addition, modification, deletion), and generate decomposed structural and appearance-level text guidance. The agent then selects an informative view and applies an image editing model to obtain visual guidance. Finally, an inversion-based rectified-flow inpainting pipeline with an interleaved sampling module performs editing in the 3D latent space, enforcing prompt alignment while maintaining 3D coherence and unedited regions. Experiments on diverse 3D edits demonstrate that Vinedresser3D outperforms prior baselines in both automatic metrics and human preference studies, while enabling precise, coherent, and mask-free 3D editing.
Abstract (translated)
文本引导的3D编辑旨在使用自然语言指令修改现有的3D资产。当前方法在理解复杂的提示、自动定位3D中的编辑以及保留未被编辑的内容方面存在困难。我们介绍了Vinedresser3D，这是一个代理框架，用于高质量的文本指导下的3D编辑，它直接在原生3D生成模型的潜在空间中操作。给定一个3D资产和一个编辑提示后，Vinedresser3D使用多模态大型语言模型来推断原始资产的丰富描述，识别编辑区域以及添加、修改或删除类型的编辑，并生成分解后的结构层次与外观层次的文本指导。然后，该代理选择信息丰富的视图并应用图像编辑模型以获取视觉指引。最后，一个基于逆向修正流修补管道（带有交错采样模块）在3D潜在空间中执行编辑操作，在确保提示对齐的同时保持3D的一致性和未修改区域。 多样化的实验表明，Vinedresser3D在自动度量标准和人类偏好研究中均优于先前的基线方法，同时支持精确、一致且无需遮罩的3D编辑。
URL
https://arxiv.org/abs/2602.19542
PDF
-
Detecting AI-Generated Forgeries via Iterative Manifold Deviation Amplification
2026-02-21 14:14:17Jiangling Zhang, Shuxuan Gao, Bofan Liu, Siqiang Feng, Jirui Huang, Yaxiong Chen, Ziyu ChenAbstract
The proliferation of highly realistic AI-generated images poses critical challenges for digital forensics, demanding precise pixel-level localization of manipulated regions. Existing methods predominantly learn discriminative patterns of specific forgeries and often struggle with novel manipulations as editing techniques continue to evolve. We propose the Iterative Forgery Amplifier Network (IFA-Net), which shifts from learning "what is fake" to modeling "what is real". Grounded in the principle that all manipulations deviate from the natural image manifold, IFA-Net leverages a frozen Masked Autoencoder (MAE) pretrained on real images as a universal realness prior. Our framework operates through a two-stage closed-loop process: an initial Dual-Stream Segmentation Network (DSSN) fuses the original image with MAE reconstruction residuals for coarse localization, followed by a Task-Adaptive Prior Injection (TAPI) module that converts this coarse prediction into guiding prompts to steer the MAE decoder and amplify reconstruction failures in suspicious regions for precise refinement. Extensive experiments on four diffusion-based inpainting benchmarks show that IFA-Net achieves an average improvement of 6.5% in IoU and 8.1% in F1-score over the second-best method, while demonstrating strong generalization to traditional manipulation types.
Abstract (translated)
高度逼真的AI生成图像的泛滥对数字取证提出了严峻挑战，需要精确地定位被篡改区域。现有的方法主要侧重于学习特定伪造品的鉴别模式，并且往往难以应对不断演进的编辑技术所带来的新类型操纵。我们提出了一种迭代伪造放大网络（Iterative Forgery Amplifier Network, IFA-Net），该网络从“识别假图像”转向了“建模真实图像”。基于所有篡改都会偏离自然图像流形的原则，IFA-Net利用一个预训练于真实图像的冻结Masked Autoencoder (MAE) 作为普遍的真实度先验。 我们的框架通过一个两阶段闭合循环过程运行：首先是一个初始的双通道分割网络（Dual-Stream Segmentation Network, DSSN），它将原始图像与MAE重建残差融合，以进行粗略定位；随后是一个任务适应性先验注入（Task-Adaptive Prior Injection, TAPI）模块，该模块将初步预测转化为指导提示，引导MAE解码器并在可疑区域放大重建失败，从而实现精确的细化。 在四个基于扩散的方法填充基准数据集上的广泛实验表明，IFA-Net相较于次佳方法平均提升了6.5%的交并比（IoU）和8.1%的F1分数，并展示了对传统篡改类型的强大泛化能力。
URL
https://arxiv.org/abs/2602.18842
PDF
-
EditCtrl: Disentangled Local and Global Control for Real-Time Generative Video Editing
2026-02-16 18:59:58Yehonathan Litman, Shikun Liu, Dario Seyb, Nicholas Milef, Yang Zhou, Carl Marshall, Shubham Tulsiani, Caleb LeakAbstract
High-fidelity generative video editing has seen significant quality improvements by leveraging pre-trained video foundation models. However, their computational cost is a major bottleneck, as they are often designed to inefficiently process the full video context regardless of the inpainting mask's size, even for sparse, localized edits. In this paper, we introduce EditCtrl, an efficient video inpainting control framework that focuses computation only where it is needed. Our approach features a novel local video context module that operates solely on masked tokens, yielding a computational cost proportional to the edit size. This local-first generation is then guided by a lightweight temporal global context embedder that ensures video-wide context consistency with minimal overhead. Not only is EditCtrl 10 times more compute efficient than state-of-the-art generative editing methods, it even improves editing quality compared to methods designed with full-attention. Finally, we showcase how EditCtrl unlocks new capabilities, including multi-region editing with text prompts and autoregressive content propagation.
Abstract (translated)
URL
https://arxiv.org/abs/2602.15031
PDF
-
Adapting VACE for Real-Time Autoregressive Video Diffusion
Abstract
We describe an adaptation of VACE (Video All-in-one Creation and Editing) for real-time autoregressive video generation. VACE provides unified video control (reference guidance, structural conditioning, inpainting, and temporal extension) but assumes bidirectional attention over full sequences, making it incompatible with streaming pipelines that require fixed chunk sizes and causal attention. The key modification moves reference frames from the diffusion latent space into a parallel conditioning pathway, preserving the fixed chunk sizes and KV caching that autoregressive models require. This adaptation reuses existing pretrained VACE weights without additional training. Across 1.3B and 14B model scales, VACE adds 20-30% latency overhead for structural control and inpainting, with negligible VRAM cost relative to the base model. Reference-to-video fidelity is severely degraded compared to batch VACE due to causal attention constraints. A reference implementation is available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2602.14381
PDF
-
When Test-Time Guidance Is Enough: Fast Image and Video Editing with Diffusion Guidance
2026-02-15 14:13:25Ahmed Ghorbel, Badr Moufad, Navid Bagheri Shouraki, Alain Oliviero Durmus, Thomas Hirtz, Eric Moulines, Jimmy Olsson, Yazid JanatiAbstract
Text-driven image and video editing can be naturally cast as inpainting problems, where masked regions are reconstructed to remain consistent with both the observed content and the editing prompt. Recent advances in test-time guidance for diffusion and flow models provide a principled framework for this task; however, existing methods rely on costly vector--Jacobian product (VJP) computations to approximate the intractable guidance term, limiting their practical applicability. Building upon the recent work of Moufad et al. (2025), we provide theoretical insights into their VJP-free approximation and substantially extend their empirical evaluation to large-scale image and video editing benchmarks. Our results demonstrate that test-time guidance alone can achieve performance comparable to, and in some cases surpass, training-based methods.
Abstract (translated)
URL
https://arxiv.org/abs/2602.14157
PDF
-
Synthetic Craquelure Generation for Unsupervised Painting Restoration
2026-02-13 09:13:46Jana Cuch-Guill\'en, Antonio Agudo, Ra\"ul P\'erez-GonzaloAbstract
Cultural heritage preservation increasingly demands non-invasive digital methods for painting restoration, yet identifying and restoring fine craquelure patterns from complex brushstrokes remains challenging due to scarce pixel-level annotations. We propose a fully annotation-free framework driven by a domain-specific synthetic craquelure generator, which simulates realistic branching and tapered fissure geometry using Bézier trajectories. Our approach couples a classical morphological detector with a learning-based refinement module: a SegFormer backbone adapted via Low-Rank Adaptation (LoRA). Uniquely, we employ a detector-guided strategy, injecting the morphological map as an input spatial prior, while a masked hybrid loss and logit adjustment constrain the training to focus specifically on refining candidate crack regions. The refined masks subsequently guide an Anisotropic Diffusion inpainting stage to reconstruct missing content. Experimental results demonstrate that our pipeline significantly outperforms state-of-the-art photographic restoration models in zero-shot settings, while faithfully preserving the original paint brushwork.
Abstract (translated)
URL
https://arxiv.org/abs/2602.12742
PDF
-
Best of Both Worlds: Multimodal Reasoning and Generation via Unified Discrete Flow Matching
2026-02-12 17:59:08Onkar Susladkar, Tushar Prakash, Gayatri Deshmukh, Kiet A. Nguyen, Jiaxun Zhang, Adheesh Juvekar, Tianshu Bao, Lin Chai, Sparsh Mittal, Inderjit S Dhillon, Ismini LourentzouAbstract
We propose UniDFlow, a unified discrete flow-matching framework for multimodal understanding, generation, and editing. It decouples understanding and generation via task-specific low-rank adapters, avoiding objective interference and representation entanglement, while a novel reference-based multimodal preference alignment optimizes relative outcomes under identical conditioning, improving faithfulness and controllability without large-scale retraining. UniDFlpw achieves SOTA performance across eight benchmarks and exhibits strong zero-shot generalization to tasks including inpainting, in-context image generation, reference-based editing, and compositional generation, despite no explicit task-specific training.
Abstract (translated)
我们提出了一种名为UniDFlow的统一离散流匹配框架，用于多模态理解、生成和编辑。该框架通过特定任务的低秩适配器分离了理解和生成过程，从而避免了目标干扰和表示纠缠。此外，一种基于参考的多模态偏好对齐技术在相同的条件下优化相对结果，提高了准确性和可控性，并且无需大规模重新训练即可实现这些改进。尽管没有进行特定任务的专门训练，UniDFlow在八个基准测试中取得了最先进的性能，并展示了强大的零样本泛化能力，适用于包括补全、上下文图像生成、基于参考的编辑和组合生成等任务。
URL
https://arxiv.org/abs/2602.12221
PDF
-
Projected Representation Conditioning for High-fidelity Novel View Synthesis
2026-02-12 14:35:30Min-Seop Kwak, Minkyung Kwon, Jinhyeok Choi, Jiho Park, Seungryong KimAbstract
We propose a novel framework for diffusion-based novel view synthesis in which we leverage external representations as conditions, harnessing their geometric and semantic correspondence properties for enhanced geometric consistency in generated novel viewpoints. First, we provide a detailed analysis exploring the correspondence capabilities emergent in the spatial attention of external visual representations. Building from these insights, we propose a representation-guided novel view synthesis through dedicated representation projection modules that inject external representations into the diffusion process, a methodology named ReNoV, short for representation-guided novel view synthesis. Our experiments show that this design yields marked improvements in both reconstruction fidelity and inpainting quality, outperforming prior diffusion-based novel-view methods on standard benchmarks and enabling robust synthesis from sparse, unposed image collections.
Abstract (translated)
我们提出了一种基于扩散的新视角合成的新型框架，该框架利用外部表示作为条件，并借助它们的几何和语义对应特性来增强生成的新视点的几何一致性。首先，我们提供了详细的分析，探讨了空间注意力中出现于外部视觉表示之间的对应能力。在此基础上，我们提出了通过专门的表示投影模块进行表示引导的新视角合成方法，将外部表示注入扩散过程之中，这种方法被命名为ReNoV（代表“表示引导的新视图合成”）。我们的实验表明，这种设计在重建保真度和图像修复质量方面都有显著改进，在标准基准测试中超越了先前基于扩散的新视图方法，并且能够从稀疏、未配准的图像集合中进行稳健的合成。
URL
https://arxiv.org/abs/2602.12003
PDF
-
Ctrl&Shift: High-Quality Geometry-Aware Object Manipulation in Visual Generation
2026-02-11 23:36:30Penghui Ruan, Bojia Zi, Xianbiao Qi, Youze Huang, Rong Xiao, Pichao Wang, Jiannong Cao, Yuhui ShiAbstract
Object-level manipulation, relocating or reorienting objects in images or videos while preserving scene realism, is central to film post-production, AR, and creative editing. Yet existing methods struggle to jointly achieve three core goals: background preservation, geometric consistency under viewpoint shifts, and user-controllable transformations. Geometry-based approaches offer precise control but require explicit 3D reconstruction and generalize poorly; diffusion-based methods generalize better but lack fine-grained geometric control. We present Ctrl&Shift, an end-to-end diffusion framework to achieve geometry-consistent object manipulation without explicit 3D representations. Our key insight is to decompose manipulation into two stages, object removal and reference-guided inpainting under explicit camera pose control, and encode both within a unified diffusion process. To enable precise, disentangled control, we design a multi-task, multi-stage training strategy that separates background, identity, and pose signals across tasks. To improve generalization, we introduce a scalable real-world dataset construction pipeline that generates paired image and video samples with estimated relative camera poses. Extensive experiments demonstrate that Ctrl&Shift achieves state-of-the-art results in fidelity, viewpoint consistency, and controllability. To our knowledge, this is the first framework to unify fine-grained geometric control and real-world generalization for object manipulation, without relying on any explicit 3D modeling.
Abstract (translated)
对象级操作，即在图像或视频中重新定位或重新定向物体以保持场景真实感，是电影后期制作、增强现实（AR）和创意编辑的核心。然而，现有方法难以同时实现三个核心目标：背景保留、视角变化下的几何一致性以及用户可控的变换。基于几何的方法提供了精确控制但需要显式的三维重建且泛化能力差；扩散基础的方法具有更好的泛化能力，但在细粒度几何控制方面表现不佳。 我们提出了Ctrl&Shift框架，这是一个端到端的扩散框架，可以在不使用明确三维表示的情况下实现几何一致的对象操作。我们的关键见解是将对象处理分解为两个阶段：对象去除和基于参考引导的修复（inpainting），同时在显式的相机姿态控制下进行。我们将这两个过程编码在一个统一的扩散过程中。 为了实现精确、解耦的控制，我们设计了一个多任务、多阶段训练策略，在不同任务中分离背景信号、身份信号和姿势信号。为了提高泛化能力，我们引入了一种可扩展的真实世界数据集构建流水线，该流程生成具有估计相对相机姿态配对图像和视频样本。 广泛的实验表明，Ctrl&Shift在保真度、视角一致性以及可控性方面均达到了最先进的成果。据我们所知，这是第一个框架，在不依赖任何显式三维建模的情况下统一了细粒度几何控制与现实世界的泛化能力，实现了对象操作的突破。
URL
https://arxiv.org/abs/2602.11440
PDF
-
APEX: Learning Adaptive High-Platform Traversal for Humanoid Robots
2026-02-11 18:55:11Yikai Wang, Tingxuan Leng, Changyi Lin, Shiqi Liu, Shir Simon, Bingqing Chen, Jonathan Francis, Ding ZhaoAbstract
Humanoid locomotion has advanced rapidly with deep reinforcement learning (DRL), enabling robust feet-based traversal over uneven terrain. Yet platforms beyond leg length remain largely out of reach because current RL training paradigms often converge to jumping-like solutions that are high-impact, torque-limited, and unsafe for real-world deployment. To address this gap, we propose APEX, a system for perceptive, climbing-based high-platform traversal that composes terrain-conditioned behaviors: climb-up and climb-down at vertical edges, walking or crawling on the platform, and stand-up and lie-down for posture reconfiguration. Central to our approach is a generalized ratchet progress reward for learning contact-rich, goal-reaching maneuvers. It tracks the best-so-far task progress and penalizes non-improving steps, providing dense yet velocity-free supervision that enables efficient exploration under strong safety regularization. Based on this formulation, we train LiDAR-based full-body maneuver policies and reduce the sim-to-real perception gap through a dual strategy: modeling mapping artifacts during training and applying filtering and inpainting to elevation maps during deployment. Finally, we distill all six skills into a single policy that autonomously selects behaviors and transitions based on local geometry and commands. Experiments on a 29-DoF Unitree G1 humanoid demonstrate zero-shot sim-to-real traversal of 0.8 meter platforms (approximately 114% of leg length), with robust adaptation to platform height and initial pose, as well as smooth and stable multi-skill transitions.
Abstract (translated)
基于深度强化学习（DRL）的人形机器人行走技术已经取得了快速进展，实现了在不平坦地形上稳健的脚部穿越。然而，越过高于腿部长度的平台仍然是一个挑战，因为目前的强化学习训练模式往往会收敛于跳跃式的解决方案，这种方案高冲击、扭矩受限且不适合实际部署。为了解决这个问题，我们提出了APEX系统——一种感知驱动、基于攀爬的高空平台穿越系统，该系统能够组合地形条件行为：在垂直边缘处向上和向下攀爬，在平台上行走或爬行以及站立和躺下以重新配置姿势。我们的方法的核心是一种通用的棘轮进度奖励机制，用于学习接触密集型、目标导向的动作。这种机制追踪迄今为止的最佳任务进展，并对非改进步骤进行惩罚，提供了稠密而无需考虑速度的监督，这有助于在强烈的安全约束条件下有效探索。 基于这一理论框架，我们使用LiDAR数据训练全身操控策略，并通过双管齐下的方法来减少仿真与现实之间的感知差距：在训练过程中模拟映射误差，在部署时对高程地图进行过滤和填补。最后，我们将所有六项技能整合到一个单一的策略中，该策略能够根据局部几何形状和指令自主选择行为并实现多技能转换。 实验结果表明，基于29自由度的Unitree G1人形机器人能够在不进行额外训练的情况下实现在0.8米高的平台上（大约为腿部长度的114%）自如穿越。此外，该系统还展示了对平台高度和初始姿势变化的强大适应性，并能够实现平稳、稳定的多技能转换。
URL
https://arxiv.org/abs/2602.11143
PDF
-
RealHD: A High-Quality Dataset for Robust Detection of State-of-the-Art AI-Generated Images
2026-02-11 05:38:40Hanzhe Yu, Yun Ye, Jintao Rong, Qi Xuan, Chen MaAbstract
The rapid advancement of generative AI has raised concerns about the authenticity of digital images, as highly realistic fake images can now be generated at low cost, potentially increasing societal risks. In response, several datasets have been established to train detection models aimed at distinguishing AI-generated images from real ones. However, existing datasets suffer from limited generalization, low image quality, overly simple prompts, and insufficient image diversity. To address these limitations, we propose a high-quality, large-scale dataset comprising over 730,000 images across multiple categories, including both real and AI-generated images. The generated images are synthesized via state-of-the-art methods, including text-to-image generation (guided by over 10,000 carefully designed prompts), image inpainting, image refinement, and face swapping. Each generated image is annotated with its generation method and category. Inpainting images further include binary masks to indicate inpainted regions, providing rich metadata for analysis. Compared to existing datasets, detection models trained on our dataset demonstrate superior generalization capabilities. Our dataset not only serves as a strong benchmark for evaluating detection methods but also contributes to advancing the robustness of AI-generated image detection techniques. Building upon this, we propose a lightweight detection method based on image noise entropy, which transforms the original image into an entropy tensor of Non-Local Means (NLM) noise before classification. Extensive experiments demonstrate that models trained on our dataset achieve strong generalization, and our method delivers competitive performance, establishing a solid baseline for future research. The dataset and source code are publicly available at this https URL.
Abstract (translated)
生成式人工智能的快速发展引发了人们对数字图像真实性问题的关注，因为现在可以低成本地生成极其逼真的假图片，这可能增加社会风险。为应对这一挑战，已建立了几个数据集来训练检测模型以区分AI生成的图像和真实图像。然而，现有的数据集存在泛化能力有限、图像质量低、提示过于简单以及缺乏图像多样性等问题。为了克服这些限制，我们提出了一项高质量的大规模数据集，该数据集包含超过73万张跨多个类别的图像，包括真实的和AI生成的图像。所生成的图像是通过最先进的方法合成的，包括文本到图像生成（由10,000多个精心设计的提示指导）、图像修复、图像改进和面部替换。每一张生成的图像都标注了其生成的方法和类别。在经过修复处理的图片中还包括二进制掩模以指示修复区域，这为分析提供了丰富的元数据。 与现有数据集相比，在我们的数据集上训练的检测模型展示了更出色的泛化能力。除了作为一个强大的基准来评估检测方法外，该数据集还促进了AI生成图像检测技术的鲁棒性的进步。基于此，我们提出了一种轻量级的检测方法，它通过将原始图片转换为非局部均值（NLM）噪声的熵张量来进行分类前处理。广泛的实验表明，在我们的数据集上训练的模型能够实现强大的泛化能力，并且我们的方法提供了具有竞争力的表现，从而为未来的研究奠定了坚实的基础。 该数据集和源代码可在以下网址公开获取：[此URL]
URL
https://arxiv.org/abs/2602.10546
PDF
-
Geometric Image Editing via Effects-Sensitive In-Context Inpainting with Diffusion Transformers
2026-02-09 08:39:47Shuo Zhang, Wenzhuo Wu, Huayu Zhang, Jiarong Cheng, Xianghao Zang, Chao Ban, Hao Sun, Zhongjiang He, Tianwei Cao, Kongming Liang, Zhanyu MaAbstract
Recent advances in diffusion models have significantly improved image editing. However, challenges persist in handling geometric transformations, such as translation, rotation, and scaling, particularly in complex scenes. Existing approaches suffer from two main limitations: (1) difficulty in achieving accurate geometric editing of object translation, rotation, and scaling; (2) inadequate modeling of intricate lighting and shadow effects, leading to unrealistic results. To address these issues, we propose GeoEdit, a framework that leverages in-context generation through a diffusion transformer module, which integrates geometric transformations for precise object edits. Moreover, we introduce Effects-Sensitive Attention, which enhances the modeling of intricate lighting and shadow effects for improved realism. To further support training, we construct RS-Objects, a large-scale geometric editing dataset containing over 120,000 high-quality image pairs, enabling the model to learn precise geometric editing while generating realistic lighting and shadows. Extensive experiments on public benchmarks demonstrate that GeoEdit consistently outperforms state-of-the-art methods in terms of visual quality, geometric accuracy, and realism.
Abstract (translated)
最近在扩散模型方面取得的进展显著提高了图像编辑的质量。然而，在处理几何变换（如平移、旋转和缩放）时，特别是在复杂场景中，仍然面临挑战。现有方法存在两大主要局限性：(1) 难以实现对象平移、旋转和缩放的精确几何编辑；(2) 不充分地模拟复杂的光照和阴影效果，导致生成的结果不够真实。 为了解决这些问题，我们提出了GeoEdit框架，该框架利用扩散变换器模块通过上下文生成来整合几何变换，从而实现对物体的精准编辑。此外，我们引入了敏感效应注意力机制（Effects-Sensitive Attention），以增强复杂照明和阴影效果的建模，提高现实感。 为了进一步支持训练，我们构建了一个大规模的几何编辑数据集RS-Objects，包含超过120,000对高质量图像对，使模型能够学习精准的几何编辑并生成逼真的光照与阴影。在公共基准测试中的广泛实验表明，GeoEdit在视觉质量、几何精度和现实感方面始终优于当前最先进的方法。
URL
https://arxiv.org/abs/2602.08388
PDF
-
PISCO: Precise Video Instance Insertion with Sparse Control
2026-02-09 05:15:39Xiangbo Gao, Renjie Li, Xinghao Chen, Yuheng Wu, Suofei Feng, Qing Yin, Zhengzhong TuAbstract
The landscape of AI video generation is undergoing a pivotal shift: moving beyond general generation - which relies on exhaustive prompt-engineering and "cherry-picking" - towards fine-grained, controllable generation and high-fidelity post-processing. In professional AI-assisted filmmaking, it is crucial to perform precise, targeted modifications. A cornerstone of this transition is video instance insertion, which requires inserting a specific instance into existing footage while maintaining scene integrity. Unlike traditional video editing, this task demands several requirements: precise spatial-temporal placement, physically consistent scene interaction, and the faithful preservation of original dynamics - all achieved under minimal user effort. In this paper, we propose PISCO, a video diffusion model for precise video instance insertion with arbitrary sparse keyframe control. PISCO allows users to specify a single keyframe, start-and-end keyframes, or sparse keyframes at arbitrary timestamps, and automatically propagates object appearance, motion, and interaction. To address the severe distribution shift induced by sparse conditioning in pretrained video diffusion models, we introduce Variable-Information Guidance for robust conditioning and Distribution-Preserving Temporal Masking to stabilize temporal generation, together with geometry-aware conditioning for realistic scene adaptation. We further construct PISCO-Bench, a benchmark with verified instance annotations and paired clean background videos, and evaluate performance using both reference-based and reference-free perceptual metrics. Experiments demonstrate that PISCO consistently outperforms strong inpainting and video editing baselines under sparse control, and exhibits clear, monotonic performance improvements as additional control signals are provided. Project page: this http URL.
Abstract (translated)
人工智能视频生成领域的格局正经历着关键的转变：从依赖详尽提示工程和“挑选”的通用生成，转向精细、可控的生成以及高保真的后期处理。在专业的人工智能辅助电影制作中，进行精确、有针对性的修改至关重要。这一转型的核心是视频实例插入技术，该技术要求将特定实例精确定位到现有镜头中，并保持场景完整性。与传统的视频编辑不同，这项任务需要满足多种需求：包括精确的时间和空间定位、物理上一致的场景交互以及真实动态的忠实保存——所有这些都应在用户最小化努力的情况下实现。 本文提出了一种名为PISCO（Precise Instance Insertion with Sparse Keyframe Control）的视频扩散模型，用于在任意稀疏关键帧控制下进行准确的视频实例插入。PISCO允许用户指定单一关键帧、开始和结束的关键帧或任意时间戳上的稀疏关键帧，并自动传播对象外观、运动及交互情况。 为了应对预训练视频扩散模型中由稀疏条件引起的严重分布偏移问题，我们引入了变量信息引导以实现稳健的条件设置以及保持分布一致的时间掩码技术来稳定时序生成。此外还结合几何感知条件以实现实景场景适应性。 为评估性能，我们构建了一个基准测试PISCO-Bench，该基准包含有验证过的实例注释和配对干净背景视频，并使用参考依赖性和无参考的感知度量进行评价。实验结果表明，在稀疏控制下，PISCO始终超越强大的图像修复及视频编辑基线模型，随着提供更多的控制信号，其性能表现出明显的单调改善趋势。 项目页面：[此HTTP URL](http://this%20URL)（请将“this http URL”替换为实际的链接）。
URL
https://arxiv.org/abs/2602.08277
PDF
-
A Unified Framework for Multimodal Image Reconstruction and Synthesis using Denoising Diffusion Models
2026-02-09 03:54:24Weijie Gan, Xucheng Wang, Tongyao Wang, Wenshang Wang, Chunwei Ying, Yuyang Hu, Yasheng Chen, Hongyu An, Ulugbek S. KamilovAbstract
Image reconstruction and image synthesis are important for handling incomplete multimodal imaging data, but existing methods require various task-specific models, complicating training and deployment workflows. We introduce Any2all, a unified framework that addresses this limitation by formulating these disparate tasks as a single virtual inpainting problem. We train a single, unconditional diffusion model on the complete multimodal data stack. This model is then adapted at inference time to ``inpaint'' all target modalities from any combination of inputs of available clean images or noisy measurements. We validated Any2all on a PET/MR/CT brain dataset. Our results show that Any2all can achieve excellent performance on both multimodal reconstruction and synthesis tasks, consistently yielding images with competitive distortion-based performance and superior perceptual quality over specialized methods.
Abstract (translated)
图像重建和图像合成对于处理不完整的多模态成像数据非常重要，但现有的方法需要各种特定任务的模型，从而增加了训练和部署流程的复杂性。我们介绍了Any2all，这是一个统一框架，通过将这些不同的任务表述为单一虚拟修复问题来解决这一局限性。我们在完整的多模态数据堆栈上训练一个无条件扩散模型。在推理时，这个模型被调整以从任何组合的可用清晰图像或噪声测量值中“修补”所有目标模式。我们使用PET/MR/CT脑部数据集验证了Any2all的有效性。我们的结果显示，Any2all在多模态重建和合成任务上均能取得出色性能，在基于失真的性能方面与专业方法相当，并且感知质量更优。
URL
https://arxiv.org/abs/2602.08249
PDF
-
ForeHOI: Feed-forward 3D Object Reconstruction from Daily Hand-Object Interaction Videos
2026-02-05 22:05:57Yuantao Chen, Jiahao Chang, Chongjie Ye, Chaoran Zhang, Zhaojie Fang, Chenghong Li, Xiaoguang HanAbstract
The ubiquity of monocular videos capturing daily hand-object interactions presents a valuable resource for embodied intelligence. While 3D hand reconstruction from in-the-wild videos has seen significant progress, reconstructing the involved objects remains challenging due to severe occlusions and the complex, coupled motion of the camera, hands, and object. In this paper, we introduce ForeHOI, a novel feed-forward model that directly reconstructs 3D object geometry from monocular hand-object interaction videos within one minute of inference time, eliminating the need for any pre-processing steps. Our key insight is that, the joint prediction of 2D mask inpainting and 3D shape completion in a feed-forward framework can effectively address the problem of severe occlusion in monocular hand-held object videos, thereby achieving results that outperform the performance of optimization-based methods. The information exchanges between the 2D and 3D shape completion boosts the overall reconstruction quality, enabling the framework to effectively handle severe hand-object occlusion. Furthermore, to support the training of our model, we contribute the first large-scale, high-fidelity synthetic dataset of hand-object interactions with comprehensive annotations. Extensive experiments demonstrate that ForeHOI achieves state-of-the-art performance in object reconstruction, significantly outperforming previous methods with around a 100x speedup. Code and data are available at: this https URL.
Abstract (translated)
单目视频捕捉日常手部与物体互动的现象普遍存在，为具身智能提供了宝贵的资源。尽管从野外视频中重建3D手部已经取得了显著的进步，但由于严重的遮挡以及摄像头、手部和物体的复杂耦合运动，重构涉及的对象仍然具有挑战性。在本文中，我们介绍了ForeHOI，这是一种新颖的前馈模型，可以直接从单目手物交互视频中重建3D对象几何形状，并且仅需一分钟的推理时间即可完成，无需任何预处理步骤。我们的关键见解是，在一个前馈框架内同时预测2D掩码填充和3D形状补全可以有效地解决单目手持物体视频中的严重遮挡问题，从而实现优于基于优化方法的表现。2D与3D形状补全之间的信息交换提升了整体重建质量，使该模型能够有效处理严重的手物遮挡情况。 为了支持我们模型的训练，我们还贡献了首个大规模、高保真合成的手部和物体互动数据集，并且提供了全面的注释。广泛的实验表明，ForeHOI在对象重建方面达到了最先进的性能，显著超越了先前的方法，并且速度提高了大约100倍。代码和数据可在以下网址获取：this https URL。
URL
https://arxiv.org/abs/2602.06226
PDF
-
LocateEdit-Bench: A Benchmark for Instruction-Based Editing Localization
2026-02-05 12:01:09Shiyu Wu, Shuyan Li, Jing Li, Jing Liu, Yequan WangAbstract
Recent advancements in image editing have enabled highly controllable and semantically-aware alteration of visual content, posing unprecedented challenges to manipulation localization. However, existing AI-generated forgery localization methods primarily focus on inpainting-based manipulations, making them ineffective against the latest instruction-based editing paradigms. To bridge this critical gap, we propose LocateEdit-Bench, a large-scale dataset comprising $231$K edited images, designed specifically to benchmark localization methods against instruction-driven image editing. Our dataset incorporates four cutting-edge editing models and covers three common edit types. We conduct a detailed analysis of the dataset and develop two multi-metric evaluation protocols to assess existing localization methods. Our work establishes a foundation to keep pace with the evolving landscape of image editing, thereby facilitating the development of effective methods for future forgery localization. Dataset will be open-sourced upon acceptance.
Abstract (translated)
最近的图像编辑技术进步使得对视觉内容进行高度可控且语义感知的修改成为可能，这给篡改定位带来了前所未有的挑战。然而，现有的AI生成伪造物检测方法主要针对基于修复（inpainting）的操纵，因此对于最新的指令驱动型编辑范式无效。为了填补这一关键空白，我们提出了LocateEdit-Bench，这是一个包含231,000张编辑图像的大规模数据集，专门用于评估面向指令驱动图像编辑的定位方法。我们的数据集整合了四种先进的编辑模型，并涵盖了三种常见的编辑类型。我们对数据集进行了详细的分析，并开发了两个多度量评估协议来评价现有的定位方法。我们的工作为跟上不断演变的图像编辑格局奠定了基础，从而促进未来伪造物检测方法的发展。该数据集将在被接受后开源。
URL
https://arxiv.org/abs/2602.05577
PDF
-
Artifact Removal and Image Restoration in AFM:A Structured Mask-Guided Directional Inpainting Approach
2026-02-03 22:37:22Juntao Zhang, Angona Biswas, Jaydeep Rade, Charchit Shukla, Juan Ren, Anwesha Sarkar, Adarsh Krishnamurthy, Aditya BaluAbstract
Atomic Force Microscopy (AFM) enables high-resolution surface imaging at the nanoscale, yet the output is often degraded by artifacts introduced by environmental noise, scanning imperfections, and tip-sample interactions. To address this challenge, a lightweight and fully automated framework for artifact detection and restoration in AFM image analysis is presented. The pipeline begins with a classification model that determines whether an AFM image contains artifacts. If necessary, a lightweight semantic segmentation network, custom-designed and trained on AFM data, is applied to generate precise artifact masks. These masks are adaptively expanded based on their structural orientation and then inpainted using a directional neighbor-based interpolation strategy to preserve 3D surface continuity. A localized Gaussian smoothing operation is then applied for seamless restoration. The system is integrated into a user-friendly GUI that supports real-time parameter adjustments and batch processing. Experimental results demonstrate the effective artifact removal while preserving nanoscale structural details, providing a robust, geometry-aware solution for high-fidelity AFM data interpretation.
Abstract (translated)
原子力显微镜（AFM）能够在纳米尺度上实现高分辨率表面成像，然而输出结果常因环境噪声、扫描缺陷和探针-样品相互作用等因素而退化。为解决这一挑战，本文提出了一种轻量级且完全自动化的框架，用于在AFM图像分析中检测并修复这些伪影。该流程首先使用分类模型判断一张AFM图是否含有伪影。如果需要，会应用一种基于AFM数据定制设计和训练的轻量化语义分割网络，生成精确的伪影掩模。根据其结构方向动态扩展这些掩模后，采用基于方向邻居插值策略进行修复以保持三维表面连续性，并通过局部高斯平滑操作实现无缝修复。 该系统集成于一个用户友好的图形界面（GUI）中，支持实时参数调整和批量处理功能。实验结果表明，此方法能够有效地移除伪影同时保留纳米级结构细节，为AFM数据的高保真解读提供了一种稳健且具有几何感知能力的解决方案。
URL
https://arxiv.org/abs/2602.04051
PDF
-
Thinking inside the Convolution for Image Inpainting: Reconstructing Texture via Structure under Global and Local Side
2026-02-03 02:30:07Haipeng Liu, Yang Wang, Biao Qian, Yong Rui, Meng WangAbstract
Image inpainting has earned substantial progress, owing to the encoder-and-decoder pipeline, which is benefited from the Convolutional Neural Networks (CNNs) with convolutional downsampling to inpaint the masked regions semantically from the known regions within the encoder, coupled with an upsampling process from the decoder for final inpainting output. Recent studies intuitively identify the high-frequency structure and low-frequency texture to be extracted by CNNs from the encoder, and subsequently for a desirable upsampling recovery. However, the existing arts inevitably overlook the information loss for both structure and texture feature maps during the convolutional downsampling process, hence suffer from a non-ideal upsampling output. In this paper, we systematically answer whether and how the structure and texture feature map can mutually help to alleviate the information loss during the convolutional downsampling. Given the structure and texture feature maps, we adopt the statistical normalization and denormalization strategy for the reconstruction guidance during the convolutional downsampling process. The extensive experimental results validate its advantages to the state-of-the-arts over the images from low-to-high resolutions including 256*256 and 512*512, especially holds by substituting all the encoders by ours. Our code is available at this https URL
Abstract (translated)
图像修复技术取得了显著进步，这主要归功于编码器-解码器管道的使用，该管道利用了具有卷积下采样功能的卷积神经网络（CNN），以从编码器中已知区域语义化地修复被遮罩的区域，并通过解码器中的上采样过程生成最终的图像修复输出。最近的研究直观地发现，CNN在编码过程中能够分别提取高频结构和低频纹理信息，并进一步进行所需的上采样恢复。然而，现有的技术不可避免地忽视了卷积下采样过程中对于结构和纹理特征图的信息丢失问题，从而导致非理想的上采样结果。在这篇文章中，我们系统性地探讨了如何通过结构和纹理特征图相互帮助来减轻在卷积下采样过程中的信息损失。基于这种策略，我们在给定的结构和纹理特征图基础上采用了统计归一化和反归一化的重建引导方法来进行卷积下采样的处理。广泛的实验结果验证了这种方法相对于现有的图像修复技术（包括从低分辨率到高分辨率如256*256和512*512的各种图像）具有优势，特别是在全部使用我们提出的编码器时尤为明显。我们的代码可以在这个链接上找到：[https URL]
URL
https://arxiv.org/abs/2602.03013
PDF
