Abstract
Large language models (LLMs), trained on large-scale text, have recently attracted significant attention for their strong performance across many tasks. Motivated by this, we investigate whether a text-trained LLM can help localize fake words in partially fake speech, where only specific words within a speech are edited. We build a speech LLM to perform fake word localization via next token prediction. Experiments and analyses on AV-Deepfake1M and PartialEdit indicates that the model frequently leverages editing-style pattern learned from the training data, particularly word-level polarity substitutions for those two databases we discussed, as cues for localizing fake words. Although such particular patterns provide useful information in an in-domain scenario, how to avoid over-reliance on such particular pattern and improve generalization to unseen editing styles remains an open question.
Abstract (translated)
大规模语言模型（LLM），经过大量文本训练，因其在多项任务中的出色表现而最近备受关注。受此启发，我们研究了一个基于文本训练的LLM是否能够在部分伪造演讲中帮助定位假词，其中只有特定词汇被修改。我们构建了一个语音LLM，并通过下一个标记预测来执行伪造单词定位。在AV-Deepfake1M和PartialEdit数据库上的实验与分析表明，该模型经常利用从训练数据中学到的编辑风格模式，特别是针对这两个数据库中提到的单字级极性替换作为定位假词的线索。尽管这种特定模式在同类场景下提供了有用的信息，但如何避免过度依赖这些特定模式，并提高对未见编辑风格的泛化能力仍然是一个开放的问题。
URL
https://arxiv.org/abs/2603.11205
