Abstract
Large Language Models (LLMs) exhibit strong multilingual capabilities, yet remain fundamentally constrained by the severe imbalance in global language resources. While over 7,000 languages are spoken worldwide, only a small subset (fewer than 100) has sufficient digital presence to meaningfully influence modern LLM training. This disparity leads to systematic underperformance, cultural misalignment, and limited accessibility for speakers of low-resource and extreme-low-resource languages. To address this gap, we introduce Bring Your Own Language (BYOL), a unified framework for scalable, language-aware LLM development tailored to each language's digital footprint. BYOL begins with a language resource classification that maps languages into four tiers (Extreme-Low, Low, Mid, High) using curated web-scale corpora, and uses this classification to select the appropriate integration pathway. For low-resource languages, we propose a full-stack data refinement and expansion pipeline that combines corpus cleaning, synthetic text generation, continual pretraining, and supervised finetuning. Applied to Chichewa and Maori, this pipeline yields language-specific LLMs that achieve approximately 12 percent average improvement over strong multilingual baselines across 12 benchmarks, while preserving English and multilingual capabilities via weight-space model merging. For extreme-low-resource languages, we introduce a translation-mediated inclusion pathway, and show on Inuktitut that a tailored machine translation system improves over a commercial baseline by 4 BLEU, enabling high-accuracy LLM access when direct language modeling is infeasible. Finally, we release human-translated versions of the Global MMLU-Lite benchmark in Chichewa, Maori, and Inuktitut, and make our codebase and models publicly available at this https URL .
Abstract (translated)
大型语言模型（LLMs）展现出强大的多语种能力，但仍受全球语言资源严重不平衡的限制。全世界有超过7000种语言被使用，然而只有其中一小部分（不到100种）具有足够的数字存在感，足以对现代LLM训练产生实质性影响。这种不均衡导致了系统性表现不佳、文化错位以及低资源和极低资源语言使用者的访问受限问题。 为解决这一差距，我们引入了一个名为“自带语言”(BYOL) 的统一框架，旨在根据每种语言的数字足迹开发出可扩展且具有语言意识的大型语言模型。BYOL从一个使用精心策划的大规模网络语料库来划分语言资源等级（极低、低、中、高）的语言资源分类开始，并利用这一分类选择合适的集成路径。 对于低资源语言，我们提出了一种全栈数据精炼和扩展管道，该管道结合了文集清理、合成文本生成、持续预训练以及监督微调。应用于奇切瓦语（Chichewa）和毛利语（Maori），此流程产生了特定于每种语言的LLMs，在12个基准测试中平均超越强大的多语言基线约12%，同时通过权重空间模型合并保持了英语及多语言能力。 对于极低资源的语言，我们引入了一条基于翻译的包容路径，并展示了在因纽特语（Inuktitut）上的定制机器翻译系统优于商业基线4个BLEU分值的结果，当直接建模不可行时，此方法能实现高精度LLM的访问权。 最后，我们在奇切瓦、毛利和因纽特语言中发布了人类翻译版本的全球多语言基准测试（Global MMLU-Lite），并且我们正在开源我们的代码库及模型，详情请参阅这个链接：[提供该研究的GitHub或官网链接]。
URL
https://arxiv.org/abs/2601.10804
