Abstract
Identity-preserving text-to-video (IPT2V) generation creates videos faithful to both a reference subject image and a text prompt. While fine-tuning large pretrained video diffusion models on ID-matched data achieves state-of-the-art results on IPT2V, data scarcity and high tuning costs hinder broader improvement. We thus introduce a Training-Free Prompt, Image, and Guidance Enhancement (TPIGE) framework that bridges the semantic gap between the video description and the reference image and design sampling guidance that enhances identity preservation and video quality, achieving performance gains at minimal this http URL, we first propose Face Aware Prompt Enhancement, using GPT-4o to enhance the text prompt with facial details derived from the reference image. We then propose Prompt Aware Reference Image Enhancement, leveraging an identity-preserving image generator to refine the reference image, rectifying conflicts with the text prompt. The above mutual refinement significantly improves input quality before video generation. Finally, we propose ID-Aware Spatiotemporal Guidance Enhancement, utilizing unified gradients to optimize identity preservation and video quality jointly during this http URL method outperforms prior work and is validated by automatic and human evaluations on a 1000 video test set, winning first place in the ACM Multimedia 2025 Identity-Preserving Video Generation Challenge, demonstrating state-of-the-art performance and strong generality. The code is available at this https URL.
Abstract (translated)
身份保持型文本到视频（IPT2V）生成旨在创建既忠实于参考主体图像又符合文本提示的视频。尽管通过在匹配ID的数据上微调大规模预训练的视频扩散模型可以实现当前最先进的IPT2V性能，但数据稀缺和高昂的调整成本限制了进一步改进的可能性。因此，我们引入了一个无需训练的提示、图像和引导增强（TPIGE）框架，该框架弥合了视频描述与参考图像之间的语义差距，并设计了一种采样指导方法以提升身份保持性和视频质量，在最少的计算资源需求下实现了性能提升。 首先，我们提出了面部感知型提示增强，利用GPT-4o从参考图像中提取面部细节来优化文本提示。接着，我们提出了提示感知型参考图像增强，通过使用一种保真度高的图像生成器精炼参考图像，从而解决与文本提示之间的冲突。上述互惠改进在视频生成前显著提升了输入质量。最后，我们提出了身份感知时空引导增强，利用统一的梯度优化身份保持和视频质量。 这种方法不仅超越了以往的工作，并通过自动评估和人类评价在一个包含1000个视频的数据集上得到了验证，在ACM多媒体2025年身份保持型视频生成挑战赛中获得第一名，展示了其领先性能和广泛适用性。相关代码可在该链接获取。
URL
https://arxiv.org/abs/2509.01362
