Abstract
Diffusion models have demonstrated impressive image synthesis performance, yet many UNet-based models are trained at certain fixed resolutions. Their quality tends to degrade when generating images at out-of-training resolutions. We trace this issue to resolution-dependent parameter behaviors, where weights that function well at the default resolution can become adverse when spatial scales shift, weakening semantic alignment and causing structural instability in the UNet architecture. Based on this analysis, this paper introduces CR-Diff, a novel method that improves the cross-resolution visual consistency by pruning some parameters of the diffusion model. Specifically, CR-Diff has two stages. It first performs block-wise pruning to selectively eliminate adverse weights. Then, a pruned output amplification is conducted to further purify the pruned predictions. Empirically, extensive experiments suggest that CR-Diff can improve perceptual fidelity and semantic coherence across various diffusion backbones and unseen resolutions, while largely preserving the performance at default resolutions. Additionally, CR-Diff supports prompt-specific refinement, enabling quality enhancement on demand.
Abstract (translated)
扩散模型已展现出卓越的图像合成性能，然而许多基于UNet的模型仅在特定固定分辨率下进行训练。当生成训练分辨率之外的图像时，其质量往往会下降。我们将此问题归因于分辨率依赖的参数行为——在默认分辨率下表现良好的权重，在空间尺度变化时可能产生负面影响，从而削弱语义对齐并导致UNet架构中的结构不稳定性。基于此分析，本文提出CR-Diff这一新颖方法，通过剪枝扩散模型的部分参数来改善跨分辨率视觉一致性。具体而言，CR-Diff包含两个阶段：首先进行块级剪枝以选择性地消除不利权重；随后执行剪枝输出放大以进一步净化剪枝后的预测结果。实验表明，CR-Diff能够在多种扩散主干网络及未见分辨率上提升感知保真度与语义连贯性，同时最大程度保持默认分辨率的性能。此外，CR-Diff还支持提示词特定的细化机制，实现按需质量增强。
URL
https://arxiv.org/abs/2604.05524
