Abstract
Latent inpainting in diffusion models still relies almost universally on linearly interpolating VAE latents under a downsampled mask. We propose a key principle for compositing image latents: Pixel-Equivalent Latent Compositing (PELC). An equivalent latent compositor should be the same as compositing in pixel space. This principle enables full-resolution mask control and true soft-edge alpha compositing, even though VAEs compress images 8x spatially. Modern VAEs capture global context beyond patch-aligned local structure, so linear latent blending cannot be pixel-equivalent: it produces large artifacts at mask seams and global degradation and color shifts. We introduce DecFormer, a 7.7M-parameter transformer that predicts per-channel blend weights and an off-manifold residual correction to realize mask-consistent latent fusion. DecFormer is trained so that decoding after fusion matches pixel-space alpha compositing, is plug-compatible with existing diffusion pipelines, requires no backbone finetuning and adds only 0.07% of FLUX.1-Dev's parameters and 3.5% FLOP overhead. On the FLUX.1 family, DecFormer restores global color consistency, soft-mask support, sharp boundaries, and high-fidelity masking, reducing error metrics around edges by up to 53% over standard mask interpolation. Used as an inpainting prior, a lightweight LoRA on FLUX.1-Dev with DecFormer achieves fidelity comparable to FLUX.1-Fill, a fully finetuned inpainting model. While we focus on inpainting, PELC is a general recipe for pixel-equivalent latent editing, as we demonstrate on a complex color-correction task.
Abstract (translated)
扩散模型中的潜在域图像修补技术几乎普遍依赖于在下采样掩码下的VAE潜在变量的线性插值。我们提出了一项用于合成图像潜在量的关键原则：像素等效潜在合成（Pixel-Equivalent Latent Compositing，PELC）。一个等效的潜在合成器应当与像素空间中的合成过程一致。这一原则使得全分辨率掩码控制和真正的软边缘alpha合成成为可能，即使VAE将图像的空间压缩了8倍。现代VAE捕捉到了超越局部结构对齐的全局上下文信息，因此线性潜在融合不能达到像素等效：在掩码接缝处会产生显著瑕疵，并且全局退化和颜色偏移现象严重。我们引入了一个名为DecFormer的7.7M参数Transformer模型，它能够预测每通道的混合权重以及离流形残差校正来实现一致性的潜在融合。DecFormer经过训练以确保在合并后解码的结果与像素空间中的alpha合成相匹配，并且可以无缝兼容现有的扩散流程，无需对骨干网络进行微调，仅增加了FLUX.1-Dev参数的0.07%和3.5%的FLOP开销。在FLUX.1系列中，DecFormer能够恢复全局颜色一致性、支持软掩码、保持清晰边界以及高保真度的遮罩处理，在边缘处降低了误差指标高达53%，相比标准掩码插值有显著改进。当作为修补前缀使用时，结合轻量级LoRA和FLUX.1-Dev的DecFormer模型可以实现与完全微调后的FLUX.1-Fill修补模型相当的保真度。尽管我们专注于图像修补，PELC作为一个通用的方法用于像素等效潜在编辑，我们在一个复杂的颜色校正任务中也展示了其应用效果。
URL
https://arxiv.org/abs/2512.05198
