Inpainting
-
Free-Lunch Augmentation by Revisiting Diffusion-Based Data Generation for Cross-Domain Few-Shot Object Detection
2026-08-05 02:56:32Zijian Zhuang, Yixiong Zou, Yuhua Li, Ruixuan LiAbstract
Cross-Domain Few-Shot Object Detection (CDFSOD) aims to transfer knowledge from data-rich upstream generic domains to downstream expert domains using scarce training data, where the significant domain gap and data scarcity make it an unsolved challenge. To address this problem, we revisit a natural yet underexplored approach in CDFSOD: data augmentation, by directly synthesizing data through diffusion models to supplement limited training samples. However, due to large domain gaps, we find that current diffusion methods cannot produce good results, leading to performance even lower than using the original images. To address these limitations, we divide the domain gaps into visual gaps and semantic gaps for separate analysis. For the visual gap, we find that the diffusion model cannot distinguish noise from useful information on expert domains, which can be mitigated by adding weakened noise. For the semantic gap, we find that the background semantics shows much smaller gaps between domains than foreground semantics, and we can bridge this gap by background inpainting. Based on the above analysis, we propose a method (Selective Inpainting with Tailored Noise, SITN) to dynamically take different strategies for downstream data synthesis based on their different gaps from the general domain, including a Generation Module for adding tailored noise and a Selection Module to dynamically select the inpainting regions. Extensive experiments on 6 datasets of CDFSOD and 4 datasets of cross-domain few-shot segmentation (CDFSS) validate that we can synthesize helpful data, achieving new state-of-the-art performance. Our codes is available at this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2608.04394
PDF
-
Helping Music Co-Creation Agents 'Listen' Well: Hierarchical Self-Supervised World Models for Understanding and Generation
2026-08-05 02:38:20Scott H. HawleyAbstract
Collaborative music agents need internal representations rich enough to support both understanding and generation, yet flexible enough for a workflow where the human retains agency. We present a hierarchical self-supervised ``world model'' for symbolic music: a 2.55M-parameter Swin V2 encoder trained on MIDI piano-roll images with JEPA-style objectives (pitch- and time-shift equivariance, masked embedding prediction, and a distributional regularizer), using no labels and no music-theory vocabulary. Probing the frozen embeddings shows that the level at which a musical property becomes decodable tracks its musical time scale: phrase boundaries are read off the coarsest levels, note density and harmonic detail off the finest. Temporal and phrase structure emerge from the self-supervised objectives alone, while harmonic content must be asked for; a small chord-supervision head raises joint chord recovery from .18 to .54, and key detection, which is never supervised, from .16 to .70. Following the Representation AutoEncoder paradigm, a conditional flow-matching model stands in for a trained decoder, flowing in pixel space from PCA-reduced conditioning: it reproduces a target window at pixel F1 $0.996$, and the same per-level conditioning dropout that controls how far variations stray also enables graphical prompting for masked inpainting with no inpainting-specific sampler. The pipeline runs on CPU producing a suggestion in $2.8$ s, or $0.6$ s on Apple MPS, which we demonstrate in a live interactive demo. In concert with an LLM-based brain, these capabilities supply the core of a collaborative music creation agent in service of, rather than in place of, human agency.
Abstract (translated)
URL
https://arxiv.org/abs/2608.04378
PDF
-
Progressive Learning of a Diffusion-based Inpainting Model for Separating Overlapped Fingerprints
2026-08-04 17:08:46Noor Hussein, Anil K. Jain, Karthik NandakumarAbstract
Overlapped friction ridge patterns are a recurring problem in latent fingerprints recovered from crime scenes and in live-scan scenarios where residual fingerprints on the sensor may corrupt subsequent acquisitions. Existing approaches for separating overlapped fingerprints either rely on rule-based orientation field completion that requires strong domain knowledge or train end-to-end deep neural networks that do not account for domain-specific considerations. This work introduces a diffusion-based pipeline for separating component fingerprints from an image containing overlapping friction ridge patterns. We formulate the separation problem as an inpainting task and progressively learn a diffusion model for this task in multiple stages. Starting from a pre-trained Stable Diffusion model, we progressively incorporate a fingerprint prior, add the ability to complete partial fingerprints, and finally propose \textbf{overlap-aware inpainting} that reconstructs each component print using a diffusion inpainting model based on multi-channel conditioning. Experiments on two public datasets demonstrate that component fingerprints reconstructed using the proposed diffusion-based inpainting method can match with their mated counterparts with very high probability.
Abstract (translated)
URL
https://arxiv.org/abs/2608.03937
PDF
-
Localize, Don't Beautify: Client-Side Control of Image-Editing APIs for Cosmetic Surgery Previews
Abstract
Ask a commercial image editor to preview a cosmetic procedure and it will often change more of the face than the request names: a nose edit can also smooth skin or alter lighting. Existing methods for confining an edit to one region require access to the model's internals, which a public editing API does not expose. We ask how much control is possible from the client side alone. In a pilot benchmark, six commercial editing configurations and one mask-based inpainting model perform facelift-style jaw-neck and rhinoplasty edits at three levels of client-side control: the prompt alone; cutting the edited region out of the response and pasting it back onto the original photograph through a landmark-derived mask (a masked composite); and asking the model itself to inpaint inside the mask where supported. Of 210 attempted edits, 196 could be scored. ArcFace cosine measures identity preservation; a CIELAB pixel-change ratio measures how much change lands inside the requested region rather than a protected facial zone. On the 12 frontal faces the regional metric could score, the masked composite improved localization over the paired prompt-only output by a median of 0.446 (95% face-clustered bootstrap interval 0.421-0.457) while changing the requested region about as much. Editors differed in edit strength versus identity retention, and the one inpainting model we tested did not beat the simple composite. Against each face's input-to-postoperative baseline, no editor moved its outputs closer to the postoperative photograph in identity-embedding terms. This is a study of control, not clinical accuracy: no surgeons rated the outputs, and each condition was generated once. Within that scope, keeping a surgical preview inside its intended region needs no access to the model; a mask and composite on the client enforce it across every editor tested, at low provider cost.
Abstract (translated)
URL
https://arxiv.org/abs/2608.02841
PDF
-
SatEdit: Mask-Conditioned Image Editing via VLM-Guided Segment Annotation
2026-07-31 12:52:32Muhammad Talha, Muhammad Ahmed AmerAbstract
Satellite image editing requires spatially precise object-level control, but supervised editing datasets for overhead imagery are costly to build because object masks, semantic labels, and paired edits are rarely available at scale. We introduce SatEdit, a mask-conditioned satellite image editing framework that constructs training supervision from unlabeled imagery. SatEdit proposes object masks with a seg- mentation foundation model, assigns semantic la- bels to sampled segments with a Vision-Language Model, and applies lightweight human verification before generating paired addition and removal exam- ples through mask-guided inpainting. We fine-tune a high-resolution image editing backbone with LoRA on a SODA-A-derived dataset containing 1,014 im- ages and 852 verified object annotations across 91 classes. In controlled comparisons with open- source and proprietary image editing models, SatE- dit achieves the highest aggregate masked-region se- mantic alignment, with a CLIP score of 0.6322 and CLIP delta of 0.0726, while preserving the surround- ing scene qualitatively. These results suggest that VLM-assisted segment annotation is a practical route to data-efficient, spatially controllable satellite image editing.
Abstract (translated)
URL
https://arxiv.org/abs/2607.29367
PDF
-
Beyond Visual Ambiguity: Guiding Robust Monocular Depth Estimation in Challenging Scenarios via Detailed Long Captions
2026-07-30 14:32:02Junrui Zhang, Jiaqi Li, Yiran Wang, Liao Shen, Zhiguo CaoAbstract
Monocular depth estimation (MDE) faces challenges with non-Lambertian surfaces and adverse weather conditions due to the visual ambiguities inherent in single-image limited information. Existing works address them in isolation via image inpainting or augmentation, yielding limited robustness gains. Language, as a powerful complementary modality to vision, is demonstrated to enhance the visual perception capabilities of vision-language models (VLMs) via detailed long captions. However, prior language-integrated MDE methods fail to fully harness this potential due to short text input with limited information, coarse global text feature learning, and limited language guidance during depth decoding. To address these limitations, we propose CapDepth, a novel framework for robust MDE that leverages guidance from detailed long captions to alleviate visual ambiguities in both challenging scenarios. First, we design a detailed long caption input template that explicitly conveys rich spatial relationships among multiple atom sentences. Second, a dynamic caption encoder is introduced to extract fine-grained depth-relevant text features via progressive masked attention. Finally, we propose a text-adaptive decoder that guides enhanced depth decoding with text features via stable adaptive layer normalization. Extensive experiments validate the efficacy of CapDepth, which outperforms state-of-the-art methods, achieving depth error reductions of 25.0% on non-Lambertian surfaces and 22.0% under adverse weather conditions.
Abstract (translated)
URL
https://arxiv.org/abs/2607.28285
PDF
-
Now You Have My Healthy Attention: A U-DiT for Brain-MRI Inpainting
2026-07-30 10:20:01Danilo Danese, Angela Lombardi, Tommaso Di NoiaAbstract
The ASNR-MICCAI BraTS Local Synthesis (Inpainting) task asks for the anatomically plausible completion of healthy brain tissue within a masked region of a T1-weighted MRI, providing a tumor-free anatomical reference for downstream analysis. As the task is scored by distortion metrics (SSIM, PSNR, MSE), we build a deterministic regression model and focus on giving it inductive biases tailored to inpainting. Our network follows the U-DiT principle of performing self-attention on a downsampled token grid: a volumetric encoder-decoder imports long-range context through a downsampled global self-attention block with three-dimensional rotary position embeddings, while convolutions and skip connections preserve high-frequency detail. Two ideas drive our results. First, we constrain the attention so that occluded ("void") tokens attend only to known-healthy tokens of the same volume, with a learned bias toward each query's contralateral homologue, forcing the completion to be inferred from observed anatomy rather than from other unknown regions. Second, we add a contralateral-symmetry input that supplies the mirrored healthy hemisphere as a patient-specific prior; since the brain is approximately bilaterally symmetric and lesions are typically unilateral, this prior improves the distortion metrics at matched structural similarity. On the official BraTS-2026 validation leaderboard our submission reaches a mean healthy-region SSIM of $0.864$, PSNR of $24.7$\,dB and MSE of $4.6{\times}10^{-3}$ over $219$ cases. We further analyse the residual smoothness inherent to distortion-optimal regression and discuss its implications for anatomical realism.
Abstract (translated)
URL
https://arxiv.org/abs/2607.27974
PDF
-
FakeIDet3-DB: Refining Digital Attacks and Patch Extraction for Secure ID Benchmarking
2026-07-29 09:03:29Mu\~noz-Haro Javier, Teruel Andres, Tolosana Ruben, DeAlcala Daniel, Vera-Rodriguez Ruben, Morales Aythami, Fierrez JulianAbstract
Identity document (ID) authentication relies on the structural integrity of complex, high-frequency security patterns. However, advanced Generative AI models can now inject localized, high-fidelity manipulations, creating deceptive attacks that bypass standard verification. Training robust image forensic models to detect these anomalies is hindered by privacy regulations, forcing reliance on synthetic templates lacking the intricate visual patterns of real IDs. To bridge this domain gap, we introduce FakeIDet3-DB, the first comprehensive database of digital manipulations on real, government-issued IDs. FakeIDet3-DB encompasses classical (e.g., copy-move) and Generative AI-driven manipulations (e.g., face-swapping, inpainting) enhanced with advanced image refinement procedures to suppress visual artifacts. In addition, to comply with strict data protection regulations (e.g., GDPR), we adopt a recently-proposed framework based on patches. In order to maximize forensic utility, we formulate privacy-aware patch extraction from a real ID as a geometrically constrained image processing problem. We propose PACE, a Pseudo-Anonymized Contextual patch Extraction algorithm, which leverages Integral Image mapping and distance-driven Non-Maximum Suppression (NMS). PACE efficiently contours anonymization masks that prevent Personally Identifiable Information (PII) leakage while maximizing semantic density in peri-censorship regions, yielding almost 5.2M patches extracted from more than 6.4K images from real/fake IDs. Furthermore, an extensive evaluation of the proposed FakeIDet3-DB is performed using state-of-the-art models, showcasing they all struggle to detect and locate attacks coming from generative and classic techniques (32.45\% EER in detection and 83.48\% AUC-ROC in localization).
Abstract (translated)
URL
https://arxiv.org/abs/2607.26641
PDF
-
$pimathbf{R}^2$: Reactive Real-time Flow Policies
2026-07-28 17:59:31Sungjae Park, Shubham TulsianiAbstract
Generalist manipulation policies increasingly take the form of action-chunking flow policies built on large pretrained backbones. Such chunks run open-loop, so the policy cannot react to sensory input arriving mid-execution, sacrificing \emph{reactivity}. Replanning more often would restore it, but the perception-to-action pipeline (a large backbone plus multiple denoising steps) is too slow: this \emph{latency} forbids frequent replanning and leaves committed actions stale, making such policies ill-suited for dynamic, closed-loop control. We present $\pi\mathbf{R}^2$, which makes these policies reactive and real-time while retaining large backbones, expressive multi-modal policies, and multi-action prediction. Built on the per-position noise schedule of diffusion forcing, $\pi\mathbf{R}^2$ contributes two ideas. First, it splits conditioning into a fast channel (proprioception, fresh every tick) and an asynchronously updated slow channel (vision-language features), so the policy reacts to proprioception within a chunk while tolerating stale vision. Second, a latency-adaptive flow schedule treats in-flight actions as inpainting conditioning and emits actions in one denoising step per call, letting one trained model adapt to varying hardware latency. Requiring minimal modification to existing architectures, $\pi\mathbf{R}^2$ can be finetuned from a pretrained policy: applied to GR00T-N1.7 on a real xArm6+XHand platform, it replans closed-loop roughly $4\times$ faster than the base policy (~$25$Hz on an A5000 GPU), acting on a fresh observation every $40$ms. Across simulation and real-world manipulation tasks, $\pi\mathbf{R}^2$ improves the success rate by up to $23\%$ in simulation and $30\%$ in the real world over the strongest baseline. Project page: this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2607.26055
PDF
-
Inferring Missing Trajectory Data with Temporal Convolutional Networks
Abstract
Trajectory data collected in real-world settings is frequently incomplete due to sensor failure, communication loss, or occlusion. We address the task of \emph{trajectory inpainting}: reconstructing contiguous missing segments from observed context. We propose a Temporal Convolutional Network (TCN) with symmetric dilation that relaxes the standard causality constraint, allowing each time step to draw on both past and future observations, a property that is essential for inpainting, but absent from forecasting-oriented architectures. The model is trained with a composite loss that combines weighted mean squared error, boundary--continuity penalties, and a smoothness regularizer. Trained on a synthetic dataset of $1,000$ (train), $200$ (validation), and $300$ (test) two-dimensional trajectories with randomly placed 20% masked segments, the model achieves good R$^{2}$, MSE and MAE metrics.
Abstract (translated)
URL
https://arxiv.org/abs/2607.25147
PDF
-
NEO: NeRF It Once, Edit It Many Times for Continuous Object Manipulation
2026-07-27 15:16:46Miko{\l}aj Zieli\'nski, David Hall, Dominik Belter, Peyman MoghadamAbstract
In this paper, we present NEO, a unified framework providing language-guided NeRF editing for robotic manipulation. Our paper introduces (i) a language-guided object removal that combines neural field resampling with multiview-consistent progressive inpainting, (ii) a direct NeRF weight editing method utilizing knowledge distillation, composing original and edited NeRFs via a teacher-student model, enabling coherent modeling of future scene states before a robot executes an action, and (iii) the first benchmark (NEO-Dataset) for quantitatively evaluating NeRF scene editing methods suitable for robot manipulation. We show that our approach outperforms state-of-the-art baselines in scene editing tasks, including object removal and pick-and-place robotic experiments, yielding visually coherent and geometrically consistent edits that reduce artifacts commonly introduced by prior methods.
Abstract (translated)
URL
https://arxiv.org/abs/2607.24538
PDF
-
Image Inpainting via Stochastic Dynamics
2026-07-27 08:20:23Jiaqi Kuang, Zihao Guo, Zhongmin QianAbstract
Image inpainting aims to recover missing regions while preserving structural consistency. We propose a non-parametric method without network training based on data-guided stochastic dynamics. Starting from a masked image, the missing pixels are evolved through a reverse-time stochastic differential equation with a kernel-weighted correction estimated directly from a reference dataset. This empirical correction guides the reconstruction toward high-density regions of the data distribution without training a neural network or fitting a parametric density model. Experiments on MNIST, Fashion-MNIST, and MVTec show that the proposed method outperforms Mean Fill, Telea, and Navier-Stokes inpainting in PSNR, SSIM, and visual quality. On CelebA, it remains competitive and produces plausible completions for structure-sensitive occlusions. These results demonstrate the effectiveness of empirical reference statistics as a non-parametric prior for image inpainting.
Abstract (translated)
URL
https://arxiv.org/abs/2607.24140
PDF
-
Layering Virtual Try-On
2026-07-24 21:25:58Chun Feng, Bowei Chen, Mengyi Shan, Ira Kemelmacher-ShlizermanAbstract
In the real world, fashion is about layering: adding a jacket over a shirt, or a sequence of adding and removing layers, rather than just a single-layer swap. This fundamental real-world task remains a challenge in existing Virtual Try-On (VTON) methods, which excel at single-layer replacement but are not designed to layer or de-layer an existing outfit. This paper proposes Layering Virtual Try-On (LVTON), a layering benchmark and method that preserves an existing outfit while enabling sequential layering. We find that current VTON paradigms are fundamentally ill-equipped for LVTON, as their reliance on cloth-agnostic representations and single-item datasets discards essential layering context. Our key insight is that the LVTON challenge must be disentangled into two distinct competencies: (1) General VTON Priors (e.g., deformation, identity preservation) and (2) Specific Layering Knowledge (e.g., layering order and occlusion reasoning). First, our model obtains general VTON priors by being trained on data produced by an automatic data generation pipeline that synthesizes samples from fashion videos via segmentation and inpainting. Second, the model is fine-tuned on a small, dedicated LVTON dataset to learn the layering logic. Our method achieves state-of-the-art results on our LVTON benchmark and demonstrates superior generalizability on traditional VTON benchmarks, setting new state-of-the-art results when fine-tuned and exhibiting zero-shot capabilities.
Abstract (translated)
URL
https://arxiv.org/abs/2607.22924
PDF
-
Small, Bias-Free, Blind and Convolutional Denoiser: A compact ConvNeXt U-Net for blind Gaussian color-image denoising
2026-07-24 13:31:22Nikolas MarkouAbstract
We describe and evaluate BF-ConvUNeXt, a compact bias-free ConvNeXt U-Net for blind additive-white-Gaussian-noise color image denoising, combining four existing ingredients so a single property survives end to end: a frozen depthwise Gabor stem (oriented band-pass, zero trainable parameters), a Laplacian-pyramid encoder routing the high-frequency residual into each skip connection, a ConvNeXt-V1 U-Net body, and bias-free construction throughout (no additive bias, linear head, LeakyReLU, variance-only batch norm). Together these make the 0.82M-parameter network exactly degree-1 homogeneous at inference, D(alpha y) = alpha D(y), licensing a Miyasawa/Tweedie score reading of the residual and blind generalization across noise levels from one model. We train a single blind model on a noise-sigma curriculum (sigma approximately 6.4 to 64, 0-255 scale); it extrapolates past that ceiling without a cliff, degrading smoothly to 22.8 dB at sigma=150 and 20.0 dB at sigma=200. Evaluated unchanged on four standard color sets (CBSD68, Kodak24, McMaster, Urban100) at sigma in {15,25,50}, it matches or beats DnCNN and FFDNet on every set and level, averaging about +0.7 dB over DnCNN. Against heavyweight CNN/transformer state of the art it trails by a small margin (roughly 0.3-1.7 dB depending on set) at 1/15 to 1/39 of their parameters. The homogeneity is inference-only and checkpoint-specific, and the learned residual is a local, not global, score (non-conservative Jacobian), so plug-and-play/RED guarantees do not transfer; it still drives stochastic sampling and linear inverse problems (inpainting, super-resolution, deblurring, compressive sensing).
Abstract (translated)
URL
https://arxiv.org/abs/2607.22793
PDF
-
Diffusion Models in Medical Image Inpainting: Challenges, Solution Taxonomy, and Future Directions
2026-07-24 02:19:52Arthur Dantas Mangussi, Joana Cristo Santos, Ricardo Cardoso Pereira, Ana Carolina Lorena, M\'ario A. T. Figueiredo, Pedro Henriques AbreuAbstract
Image inpainting aims to reconstruct missing or corrupted regions of an image while preserving as much as possible, visual and semantic consistency. In medical imaging, this task is particularly important because artifacts, missing information, and pathological alterations can compromise diagnostic reliability and downstream clinical applications. Recently, diffusion models have emerged as state-of-the-art generative approaches for medical image inpainting due to their ability to generate anatomically consistent reconstructions. This survey presents a systematic review of diffusion-based methods for medical image inpainting, covering the main architectures, applications, datasets, and evaluation strategies reported across 60 studies. In addition, we propose a taxonomy for diffusion-based approaches. The analysis reveals a rapid growth of research interest in diffusion-based medical image inpainting, with denoising diffusion probabilistic models and latent diffusion models emerging as the dominant architectures. The reviewed studies mainly focus on artifact removal, data augmentation, pseudo-healthy tissue reconstruction, and anomaly detection, particularly in magnetic resonance imaging and computed tomography imaging. Overall, diffusion models demonstrate strong performance in producing anatomically plausible reconstructions and aiding downstream clinical tasks. However, the review also highlights important challenges, including the lack of standardized benchmarks, limited dataset diversity, and restricted validation procedures across diverse clinical applications and imaging scenarios.
Abstract (translated)
URL
https://arxiv.org/abs/2607.21904
PDF
-
Oxygen-TryOn: Fashion-Native Foundation Model for Any-item Virtual Try-On
2026-07-23 17:45:56Yong Liu, Xiaolong Fu, Zihang Xu, Wen Xue, Xueheng Li, Lin Song, Yuan Zhang, Chuyang Zhao, Haoyang Huang, Nan Duan, Yipeng Sun, Yan Li, Simiu GuAbstract
We present Oxygen-TryOn, a unified foundation model for any-item virtual try-on. Rather than repurposing a general-purpose image editor, Oxygen-TryOn is fashion-native, built for try-on through a dedicated data engine and try-on-specific training. Given one or more reference items (clean product shots or in-the-wild worn-on photos) and a single target subject image, it synthesizes a photorealistic image of the subject wearing the items across virtually any fashion category. Prior systems handle a single garment category in a studio setting, and recent multi-reference methods remain garment-centric; in contrast, Oxygen-TryOn supports diverse items and scenarios, including full- and half-body views, a variable number of references, and free multi-item composition, while faithfully preserving both subject identity and item appearance. Instead of mask-based inpainting, we reformulate try-on as a multi-reference, understanding-driven generation task. We build a data engine that collects, manufactures, annotates, and filters high-quality try-on data at scale, and design a three-stage recipe of continued pre-training (CPT), supervised fine-tuning (SFT), and reinforcement learning (RL). The RL stage uses a hybrid reward combining an in-house try-on reward model with a proprietary, rubric-guided general-purpose model, jointly supervising fine-grained consistency and instruction-level quality. It also follows general editing instructions (e.g., pose changes) in the same pass. Across public benchmarks and our in-house Oxygen-TryOn Bench, it achieves state-of-the-art consistency and realism on single-item try-on and leads on multi-item try-on, matching or surpassing both leading proprietary systems (Nano Banana Pro, GPT-Image-2, Seedream5 Lite) and open-source models (FLUX.2).
Abstract (translated)
URL
https://arxiv.org/abs/2607.21694
PDF
-
3D-GIMP: When 3D Gaussian Inpainting Meets PatchMatch
2026-07-22 23:27:20Xuening Tian, Dieter Schmalstieg, Shohei MoriAbstract
Recent advances in 3D scene editing have leveraged iterative diffusion models to update input views. However, this process is computationally expensive and struggles to produce sharp details. Meanwhile, ``hallucination drift'' frequently introduces multi-view inconsistencies, leading to structural artifacts when rendering novel viewpoints. To address this problem, we present 3D-GIMP (3D Gaussian Inpainting Meets Patch Matching), a novel hybrid paradigm designed for high-fidelity object removal in 3D Gaussian Splatting. Instead of diffusing every view, 3D-GIMP performs a single generative inpainting on a key reference view, which serves as an appearance prior. We then introduce a 3D-aware PatchMatch algorithm to propagate these reference textures across all remaining views via correspondence matching, effectively bypassing the stochastic nature of frame-by-frame diffusion. By prioritizing reconstructive consistency over iterative generation, 3D-GIMP maintains high-frequency details across arbitrary resolutions while ensuring a mathematically consistent 3D reconstruction. Our experiments demonstrate that 3D-GIMP not only achieves competitive inpainting quality as previous methods using diffusion in multiple views, but also outperforms these methods in rendering speed and view consistency.
Abstract (translated)
URL
https://arxiv.org/abs/2607.20789
PDF
-
Diffusion ReRoll: Revisable Denoising for Robotic Sequential Prediction
2026-07-22 08:50:40Seonsoo Kim, Seongil Hong, Jun-Gill KangAbstract
We propose Diffusion ReRoll, a diffusion-based framework for robotic sequential prediction that enables revisable denoising over horizons. Existing diffusion-based sequence predictors typically perform a single monotonic denoising process. In contrast, Diffusion ReRoll selectively re-noises regions that have become locally stable while the remaining regions continue denoising, so the re-noised regions can be refined again using context from the rest of the horizon. This structured re-noising enables iterative cross-horizon revision, allowing earlier and later segments to revise one another, while maintaining local consistency. We evaluate Diffusion ReRoll against full-sequence diffusion and causal denoising based on Diffusion Forcing across long-horizon planning, policy learning, and unified video-action modeling. On OGBench PointMaze and AntMaze, Diffusion ReRoll achieves relative gains in average success rate of 21% over Diffusion Forcing in matched guidance-based planning and 23% over Diffuser in matched goal-inpainting. In diffusion-policy-style action prediction, Diffusion ReRoll improves average success by 56.5% relative to Diffusion Policy across different prediction horizons and history lengths on the LIBERO-10 multi-task benchmark. In unified video-action prediction, Diffusion ReRoll improves policy and inverse dynamics performance, especially under out-of-distribution evaluation, and achieves the best action-video consistency. These results support structured re-noising as an effective mechanism for revisable robotic sequence generation.
Abstract (translated)
URL
https://arxiv.org/abs/2607.19919
PDF
-
Aligned Stable Inpainting: Mitigating Unwanted Object Insertion and Preserving Color Consistency
2026-07-21 15:09:52Yikai Wang, Junqiu Yu, Chenjie Cao, Xiangyang Xue, Yanwei FuAbstract
Generative image inpainting can produce realistic results even with large, irregular masks, but existing methods still suffer from two common problems: (1) Unwanted object insertion: hallucinate artifacts that do not match the surrounding context. (2) Color inconsistency: noticeable color shifts that lead to smeared textures. We analyze the causes of these issues and propose Aligned Stable inpainting with UnKnown Areas prior (ASUKA), a post-hoc framework for pre-trained inpainting models. To reduce unwanted object insertion, we use reconstruction-based priors to guide the generative model, suppressing hallucinated objects while preserving generative flexibility. To address color inconsistency, we design a specialized VAE decoder that formulates latent-to-image decoding as a local harmonization task. We implement ASUKA on both U-Net-based and DiT-based inpainting models with lightweight modifications. Experiments on Places2 and MISATO, our proposed benchmark, show that ASUKA effectively suppresses object hallucination and improves color consistency, outperforming existing diffusion- and rectified flow-based inpainting methods. The dataset, models, and code will be released on GitHub.
Abstract (translated)
URL
https://arxiv.org/abs/2601.15368
PDF
-
Local Label-Informed Feature Transfer for Generating Ground-Truth Medical Images: A Comparison of GAN- and Diffusion-Based Approaches
2026-07-21 09:13:07Rick Wilming, Irem Ozseker, Luca Matteo Cornils, Ahc\`ene Boubekki, Benedict Clark, Danny Panknin, Stefan HaufeAbstract
Validating Explainable Artificial Intelligence (XAI) methods in medical imaging requires ground-truth data with known locations of informative features. However, current approaches rely on expert annotations, which are prone to labeling errors, or on hand-crafted artificial perturbations superimposed onto healthy images to mimic lesions or malignant features, which lack clinical realism. We present Local Label-Informed Feature Transfer (LLIFT), a framework for generating semi-synthetic brain magnetic resonance images with realistic lesions placed in user-controlled regions, which does not require pixel-level lesion annotations during training. We implement LLIFT with two generative paradigms: LLIFT-GAN, a custom GAN that learns pathological features from binary class labels alone, and LLIFT-DM, a diffusion-based inpainting pipeline conditioned on bounding-box masks via ControlNet. Both approaches are evaluated on brain magnetic resonance imaging data derived from the Human Connectome Project. In evaluations, both achieve Fréchet Inception Distance scores, with respect to the real pathological distribution, that are comparable to the inter-class reference between healthy and pathological images in the given dataset. Furthermore, qualitative inspection confirms the realism of lesion structures. The resulting benchmark datasets provide spatially controlled ground truth data for evaluating XAI methods in medical imaging.
Abstract (translated)
URL
https://arxiv.org/abs/2607.18882
PDF
