Abstract
In recent years, there has been a resurgence of interest in non-autoregressive text generation in the context of general language modeling. Unlike the well-established autoregressive language modeling paradigm, which has a plethora of standard training and inference libraries, implementations of non-autoregressive language modeling have largely been bespoke making it difficult to perform systematic comparisons of different methods. Moreover, each non-autoregressive language model typically requires it own data collation, loss, and prediction logic, making it challenging to reuse common components. In this work, we present the XLM python package, which is designed to make implementing small non-autoregressive language models faster with a secondary goal of providing a suite of small pre-trained models (through a companion xlm-models package) that can be used by the research community. The code is available at this https URL.
Abstract (translated)
近年来，非自回归文本生成在通用语言模型领域的兴趣重新兴起。与已经成熟的自回归语言建模范式不同，后者有大量的标准训练和推理库，而非自回归语言模型的实现大多数是定制化的，这使得对不同方法进行系统的比较变得困难。此外，每一个非自回归语言模型通常都需要自己的数据收集、损失函数和预测逻辑，从而难以重用通用组件。在本工作中，我们介绍了XLM Python包，它旨在加快小型非自回归语言模型的实现速度，并且其次要目标是提供一系列可以通过配套的xlm-models包使用的预训练小模型，以供研究社区使用。代码可以在[这个链接](https://this https URL)找到。
URL
https://arxiv.org/abs/2512.17065
