Abstract
We demonstrate that discriminative models inherently contain powerful generative capabilities, challenging the fundamental distinction between discriminative and generative architectures. Our method, Direct Ascent Synthesis (DAS), reveals these latent capabilities through multi-resolution optimization of CLIP model representations. While traditional inversion attempts produce adversarial patterns, DAS achieves high-quality image synthesis by decomposing optimization across multiple spatial scales (1x1 to 224x224), requiring no additional training. This approach not only enables diverse applications -- from text-to-image generation to style transfer -- but maintains natural image statistics ($1/f^2$ spectrum) and guides the generation away from non-robust adversarial patterns. Our results demonstrate that standard discriminative models encode substantially richer generative knowledge than previously recognized, providing new perspectives on model interpretability and the relationship between adversarial examples and natural image synthesis.
Abstract (translated)
我们展示了判别模型本质上含有强大的生成能力，这挑战了判别架构和生成架构之间的基本区别。我们的方法，直接上升合成（Direct Ascent Synthesis, DAS），通过在CLIP模型表示上的多分辨率优化揭示这些潜在的能力。虽然传统的逆向尝试会产生对抗性模式，但DAS通过将优化分解到多个空间尺度（从1x1到224x224）实现了高质量的图像合成，并且无需额外训练。这种方法不仅能够支持多种应用——包括文本到图像生成和风格迁移——还能保持自然图像统计特性（$1/f^2$频谱），并指导生成远离非鲁棒性的对抗性模式。我们的结果表明，标准判别模型编码了比之前认识到的更为丰富的生成知识，这为模型可解释性和对抗样本与自然图像合成之间的关系提供了新的视角。
URL
https://arxiv.org/abs/2502.07753
