Abstract
AI-generated video has revolutionized short video production, filmmaking, and personalized media, making video local editing an essential tool. However, this progress also blurs the line between reality and fiction, posing challenges in multimedia forensics. To solve this urgent issue, V2A-Mark is proposed to address the limitations of current video tampering forensics, such as poor generalizability, singular function, and single modality focus. Combining the fragility of video-into-video steganography with deep robust watermarking, our method can embed invisible visual-audio localization watermarks and copyright watermarks into the original video frames and audio, enabling precise manipulation localization and copyright protection. We also design a temporal alignment and fusion module and degradation prompt learning to enhance the localization accuracy and decoding robustness. Meanwhile, we introduce a sample-level audio localization method and a cross-modal copyright extraction mechanism to couple the information of audio and video frames. The effectiveness of V2A-Mark has been verified on a visual-audio tampering dataset, emphasizing its superiority in localization precision and copyright accuracy, crucial for the sustainable development of video editing in the AIGC video era.
Abstract (translated)
AI生成的视频已经推动了短视频制作、电影制作和个性化媒体的发展，使视频本地编辑成为必不可少的工具。然而，这一进步也模糊了现实与虚构之间的界线，对多媒体forensics造成了挑战。为解决这一紧迫问题，V2A-Mark提出了通过解决当前视频篡改forensics的局限性来 addressing the limitations of current video tampering forensics，例如缺乏一般性、单一功能和单模态关注。将视频转录为视频的隐式视觉-音频本地化水印和版权水印相结合，我们的方法可以将隐形的视觉-音频本地化水印和版权水印嵌入原始视频帧和音频中，实现精确的本地处理和版权保护。我们还设计了一个时间对齐和融合模块以及退化提示学习来提高定位准确性和解码稳健性。同时，我们引入了音频级联定位方法和跨模态版权提取机制，将音频和视频帧的信息耦合在一起。V2A-Mark的有效性已在视觉-音频篡改数据集上得到验证，强调了其在本地定位精度和版权准确性方面的优越性，这对AIGC视频时代的视频编辑的可持续发展至关重要。
URL
https://arxiv.org/abs/2404.16824
