Abstract
Diffusion models show promise for image restoration, but existing methods often struggle with inconsistent fidelity and undesirable artifacts. To address this, we introduce Kernel Density Steering (KDS), a novel inference-time framework promoting robust, high-fidelity outputs through explicit local mode-seeking. KDS employs an $N$-particle ensemble of diffusion samples, computing patch-wise kernel density estimation gradients from their collective outputs. These gradients steer patches in each particle towards shared, higher-density regions identified within the ensemble. This collective local mode-seeking mechanism, acting as "collective wisdom", steers samples away from spurious modes prone to artifacts, arising from independent sampling or model imperfections, and towards more robust, high-fidelity structures. This allows us to obtain better quality samples at the expense of higher compute by simultaneously sampling multiple particles. As a plug-and-play framework, KDS requires no retraining or external verifiers, seamlessly integrating with various diffusion samplers. Extensive numerical validations demonstrate KDS substantially improves both quantitative and qualitative performance on challenging real-world super-resolution and image inpainting tasks.
Abstract (translated)
扩散模型在图像恢复方面表现出潜力，但现有的方法往往难以保持一致的保真度，并且会产生不希望出现的伪影。为了解决这些问题，我们引入了 Kernel Density Steering (KDS)，这是一种新颖的推理框架，通过显式的局部模式寻求机制来促进稳健、高保真的输出。 KDS 使用一个由 $N$ 个粒子组成的扩散样本集合，在这些样本的集体输出中计算补丁级别的核密度估计梯度。这些梯度将每个粒子中的补丁引导到集合内识别出的共享更高密度区域，从而形成了一种“群体智慧”的机制。这种集体局部模式寻求机制可以将样本从容易产生伪影的虚假模式（由于独立采样或模型不完善）中引导出来，并朝向更稳健、高保真的结构。 通过同时采样多个粒子，我们可以以更高的计算成本换取更好的质量样本。KDS 作为一种即插即用框架，无需重新训练或外部验证器即可与各种扩散采样器无缝集成。 大量的数值验证表明，KDS 在具有挑战性的现实世界超分辨率和图像修复任务中显著提高了定性和定量性能。
URL
https://arxiv.org/abs/2507.05604
