Abstract
Inverting corrupted images into the latent space of diffusion models is challenging. Current methods, which encode an image into a single latent vector, struggle to balance structural fidelity with semantic accuracy, leading to reconstructions with semantic drift, such as blurred details or incorrect attributes. To overcome this, we introduce Prompt-Guided Dual Latent Steering (PDLS), a novel, training-free framework built upon Rectified Flow models for their stable inversion paths. PDLS decomposes the inversion process into two complementary streams: a structural path to preserve source integrity and a semantic path guided by a prompt. We formulate this dual guidance as an optimal control problem and derive a closed-form solution via a Linear Quadratic Regulator (LQR). This controller dynamically steers the generative trajectory at each step, preventing semantic drift while ensuring the preservation of fine detail without costly, per-image optimization. Extensive experiments on FFHQ-1K and ImageNet-1K under various inversion tasks, including Gaussian deblurring, motion deblurring, super-resolution and freeform inpainting, demonstrate that PDLS produces reconstructions that are both more faithful to the original image and better aligned with the semantic information than single-latent baselines.
Abstract (translated)
将受损图像转化为扩散模型的潜在空间是一项挑战。当前的方法，通过将图像编码为单一潜变量向量，难以在结构保真度和语义准确性之间取得平衡，导致重构时出现语义漂移问题，如细节模糊或属性不正确。为了克服这一难题，我们引入了基于校正流模型的新型无训练框架——提示引导双潜在控制（PDLS）。该方法利用这些模型稳定的反演路径，并将反演过程分解为两条互补路径：一条用于保持源图像完整性的结构路径和一条由提示指导的语义路径。我们将这种双重指引形式化为最优控制问题，并通过线性二次调节器（LQR）得出闭式解来实现控制器，该控制器在每一步动态调整生成轨迹，防止语义漂移的同时确保细节点的保真度，而无需进行昂贵且针对每个图像的优化。 广泛的实验在FFHQ-1K和ImageNet-1K数据集上进行了各种反演任务（包括高斯去模糊、运动去模糊、超分辨率和自由形式修复），结果表明PDLS生成的重构不仅更忠于原始图像，而且与单一潜变量基线相比，在语义信息方面也更为一致。
URL
https://arxiv.org/abs/2509.18619
