Abstract
During the finetuning stage of text generation tasks, standard cross-entropy loss treats all tokens equally. This can lead models to overemphasize high-frequency, low-information tokens, neglecting lower-frequency tokens crucial for specificity and informativeness in generated content. This paper introduces a novel loss function, Power-Law Decay Loss (PDL), specifically designed to optimize the finetuning process for text generation. The core motivation for PDL stems from observations in information theory and linguistics: the informativeness of a token is often inversely proportional to its frequency of occurrence. PDL re-weights the contribution of each token in the standard cross-entropy loss based on its frequency in the training corpus, following a power-law decay. Specifically, the weights for high-frequency tokens are reduced, while low-frequency, information-dense tokens are assigned higher weights. This mechanism guides the model during finetuning to focus more on learning and generating tokens that convey specific and unique information, thereby enhancing the quality, diversity, and informativeness of the generated text. We theoretically elaborate on the motivation and construction of PDL and discuss its potential applications and advantages across various text generation finetuning tasks, such as abstractive summarization, dialogue systems, and style transfer.
Abstract (translated)
在文本生成任务的微调阶段，标准交叉熵损失函数会平等对待所有标记(token)。这可能导致模型过度强调高频率但信息量低的标记，而忽略了那些对生成内容的具体性和信息性至关重要的低频标记。本文介绍了一种新颖的损失函数——幂律衰减损失(PDL)，它专门用于优化文本生成任务中的微调过程。 PDL的核心动机源于信息论和语言学观察：一个标记的信息量通常与其出现频率成反比。因此，PDL根据训练语料库中标记的频率重新加权标准交叉熵损失中每个标记的贡献值，并遵循幂律衰减原则。具体来说，高频率标记的权重被降低，而低频且信息密集型标记则赋予更高的权重。 这种机制在微调过程中引导模型更加关注学习和生成传达特定及独特信息的标记，从而提高生成文本的质量、多样性和信息量。本文从理论上详细阐述了PDL的动机及其构造，并讨论了其在摘要概括、对话系统以及风格转换等各类文本生成任务中的潜在应用和优势。
URL
https://arxiv.org/abs/2505.16900
