Abstract
Video head swapping aims to replace the entire head of a video subject, including facial identity, head shape, and hairstyle, with that of a reference image, while preserving the target body, background, and motion dynamics. Due to the lack of ground-truth paired swapping data, prior methods typically train on cross-frame pairs of the same person within a video and rely on mask-based inpainting to mitigate identity leakage. Beyond potential boundary artifacts, this paradigm struggles to recover essential cues occluded by the mask, such as facial pose, expressions, and motion dynamics. To address these issues, we prompt a video editing model to synthesize new heads for existing videos as fake swapping inputs, while maintaining frame-synchronized facial poses and expressions. This yields HeadSwapBench, the first cross-identity paired dataset for video head swapping, which supports both training (\TrainNum{} videos) and benchmarking (\TestNum{} videos) with genuine outputs. Leveraging this paired supervision, we propose DirectSwap, a mask-free, direct video head-swapping framework that extends an image U-Net into a video diffusion model with a motion module and conditioning inputs. Furthermore, we introduce the Motion- and Expression-Aware Reconstruction (MEAR) loss, which reweights the diffusion loss per pixel using frame-difference magnitudes and facial-landmark proximity, thereby enhancing cross-frame coherence in motion and expressions. Extensive experiments demonstrate that DirectSwap achieves state-of-the-art visual quality, identity fidelity, and motion and expression consistency across diverse in-the-wild video scenes. We will release the source code and the HeadSwapBench dataset to facilitate future research.
Abstract (translated)
视频头部交换的目标是用参考图像中的整个头部（包括面部识别、头形和发型）替换视频主体的头部，同时保留目标身体、背景及运动动态。由于缺乏配对的真实交换数据，先前的方法通常在来自同一视频内不同帧的跨帧配对上进行训练，并依赖基于掩码的修复技术来减少身份信息泄漏。除了可能存在的边界伪影之外，这种范式难以恢复被掩码遮挡的重要线索，例如面部姿态、表情和运动动态。 为了解决这些问题，我们提示一个视频编辑模型生成用于现有视频的人造头部交换输入，同时保持逐帧同步的面部姿势和表情。这产生了HeadSwapBench，这是首个用于视频头部交换的真实配对数据集，支持训练（\TrainNum{}个视频）和基准测试（\TestNum{}个视频），并提供真实输出。利用这种配对监督，我们提出了DirectSwap，这是一种无掩码的直接视频头部交换框架，它将图像U-Net扩展为带有运动模块和条件输入的视频扩散模型。此外，我们引入了基于运动和表情感知重建（MEAR）损失，该损失通过帧差值幅度和面部特征点接近度来重新加权扩散损失每像素权重，从而增强跨帧在动作与表达上的一致性。 大量的实验表明，DirectSwap实现了视觉质量、身份忠实度以及在各种野外视频场景中的运动和表情一致性方面的最新技术水平。我们将发布源代码和HeadSwapBench数据集以促进未来的研究工作。
URL
https://arxiv.org/abs/2512.09417
