Abstract
Modern neural language models achieve high accuracy in text generation, yet precise control over generation length remains underdeveloped. In this paper, we first investigate a recent length control method based on Reverse Positional Embeddings (RPE) and show its limits when control is requested beyond the training distribution. In particular, using a discrete countdown signal tied to the absolute remaining token count leads to instability. To provide robust length control, we introduce Progress Ratio Embeddings (PRE), as continuous embeddings tied to a trigonometric impatience signal. PRE integrates seamlessly into standard Transformer architectures, providing stable length fidelity without degrading text accuracy under standard evaluation metrics. We further show that PRE generalizes well to unseen target lengths. Experiments on two widely used news-summarization benchmarks validate these findings.
Abstract (translated)
现代神经语言模型在文本生成中实现了高精度，但在控制生成长度方面仍然存在不足。本文首先研究了一种基于逆向位置嵌入（RPE）的最新长度控制方法，并展示了当请求超出训练分布范围时该方法的局限性。特别是，使用与绝对剩余令牌数量绑定的离散倒计时信号会导致不稳定。为了提供稳健的长度控制，我们引入了进度比嵌入（PRE），即连续地绑定到三角函数不耐烦信号的嵌入。PRE可以无缝集成到标准Transformer架构中，在不降低标准评估指标下的文本准确性的情况下，提供了稳定的长度保真度。此外，我们还展示了PRE能够很好地泛化到未见过的目标长度上。在两个广泛使用的新闻摘要基准测试上的实验验证了这些发现。
URL
https://arxiv.org/abs/2512.06938
