Abstract
This paper introduces Qomhrá, a bilingual Irish-English large language model (LLM), developed under low-resource constraints presenting a complete pipeline spanning bilingual continued pre-training, instruction tuning, and alignment from human preferences. Newly accessible Irish corpora and English text are mixed and curated to improve Irish performance while preserving English ability. 6 closed-weight LLMs are judged for their Irish text generation by a native speaker, a learner and other LLMs. Google's Gemini-2.5-Pro is ranked the highest and is subsequently used to synthesise instruction tuning and human preference datasets. Two datasets are contributed leveraging Gemini-2.5-Pro: a 30K Irish-English parallel instruction tuning dataset and a 1K human preference dataset, generating accepted and rejected responses that show near perfect alignment with a native Irish speaker. Qomhrá is comprehensively evaluated across benchmarks testing translation, gender understanding, topic identification and world knowledge with gains of up to 29% in Irish and 44% in English. Qomhrá also undergoes instruction tuning and demonstrates clear progress in instruction following, crucial for chatbot functionality.
Abstract (translated)
这篇论文介绍了Qomhrá，这是一个在资源有限的情况下开发的爱尔兰语-英语双语大型语言模型（LLM）。该模型涵盖了从双语持续预训练、指令微调到人类偏好的对齐在内的完整流程。通过混合和整理新的可获取的爱尔兰语语料库和英语文本，以提高爱尔兰语的表现同时保持其在英语方面的能力。六种封闭权重的LLM由一位母语使用者、学习者以及其他LLM对其生成的爱尔兰语文本进行了评估。Google的Gemini-2.5-Pro排名最高，并被用于合成指令微调和人类偏好数据集。该研究贡献了两个基于Gemini-2.5-Pro的数据集：一个30K条目的爱尔兰语-英语并行指令微调数据集，以及一个人类偏好数量为1K的偏好数据集，这两个数据集中包含了母语使用者接受或拒绝的回答，这些回答几乎完美地与一位爱尔兰母语者的要求一致。Qomhrá在跨基准测试中表现出色，在翻译、性别理解、话题识别和世界知识方面分别提高了高达29%（爱尔兰语）和44%（英语）。此外，通过指令微调，Qomhrá还展示了明显遵循指令的进步，这对于聊天机器人的功能至关重要。
URL
https://arxiv.org/abs/2510.17652
