Abstract
Video face restoration faces a critical challenge in maintaining temporal consistency while recovering fine facial details from degraded inputs. This paper presents a novel approach that extends Vector-Quantized Variational Autoencoders (VQ-VAEs), pretrained on static high-quality portraits, into a video restoration framework through variational latent space modeling. Our key innovation lies in reformulating discrete codebook representations as Dirichlet-distributed continuous variables, enabling probabilistic transitions between facial features across frames. A spatio-temporal Transformer architecture jointly models inter-frame dependencies and predicts latent distributions, while a Laplacian-constrained reconstruction loss combined with perceptual (LPIPS) regularization enhances both pixel accuracy and visual quality. Comprehensive evaluations on blind face restoration, video inpainting, and facial colorization tasks demonstrate state-of-the-art performance. This work establishes an effective paradigm for adapting intensive image priors, pretrained on high-quality images, to video restoration while addressing the critical challenge of flicker artifacts. The source code has been open-sourced and is available at this https URL.
Abstract (translated)
视频面部修复面临的一个关键挑战是在恢复降质输入中的精细面部细节的同时保持时间一致性。本文提出了一种新颖的方法，该方法通过变分潜在空间建模将基于静态高质量肖像预训练的向量量化变分自动编码器（VQ-VAEs）扩展到视频修复框架中。我们的关键技术创新在于重新表述离散代码本表示为狄利克雷分布连续变量，从而使得在帧之间进行面部特征的概率转换成为可能。一种时空变换架构同时建模帧间的依赖关系，并预测潜在分布，而拉普拉斯约束重建损失与感知（LPIPS）正则化的结合增强了像素准确性和视觉质量。在盲面修复、视频插值和面部着色任务上的全面评估表明了该方法处于业界领先地位。这项工作建立了一种有效的范式，将基于高质量图像的密集图像先验知识适应到视频修复中，并解决了闪烁伪影的关键挑战。源代码已开源并可在提供的URL获取。
URL
https://arxiv.org/abs/2506.13355
