Abstract
Large Language Models (LLMs) excel at rewriting tasks such as text style transfer and grammatical error correction. While there is considerable overlap between the inputs and outputs in these tasks, the decoding cost still increases with output length, regardless of the amount of overlap. By leveraging the overlap between the input and the output, Kaneko and Okazaki (2023) proposed model-agnostic edit span representations to compress the rewrites to save computation. They reported an output length reduction rate of nearly 80% with minimal accuracy impact in four rewriting tasks. In this paper, we propose alternative edit phrase representations inspired by phrase-based statistical machine translation. We systematically compare our phrasal representations with their span representations. We apply the LLM rewriting model to the task of Automatic Speech Recognition (ASR) post editing and show that our target-phrase-only edit representation has the best efficiency-accuracy trade-off. On the LibriSpeech test set, our method closes 50-60% of the WER gap between the edit span model and the full rewrite model while losing only 10-20% of the length reduction rate of the edit span model.
Abstract (translated)
大型语言模型（LLMs）在诸如文本风格转换和语法错误纠正等重写任务中表现出色。尽管这些任务中的输入和输出之间存在一定的重叠，但解码成本仍然会随着输出长度的增加而上升，无论重叠程度如何。通过利用输入与输出之间的重叠，Kaneko 和 Okazaki（2023）提出了模型无关的编辑跨度表示方法来压缩重写内容以节省计算资源。他们在四个重写任务中报告了近 80% 的输出长度缩减率，并且对准确性的影响很小。 在本文中，我们提出了一种受基于短语的统计机器翻译启发的替代编辑短语表示方法。我们系统地比较了我们的短语表示与他们的跨度表示。我们将LLM重写模型应用于自动语音识别（ASR）后处理任务，并表明仅目标短语的编辑表示具有最佳的效率-准确性权衡。 在LibriSpeech测试集上，我们的方法缩小了编辑跨度模型和完整重写模型之间50-60%的词错误率（WER）差距，同时仅损失了编辑跨度模型长度缩减率的10-20%。
URL
https://arxiv.org/abs/2501.13831
