Abstract
We introduce Swan, a family of embedding models centred around the Arabic language, addressing both small-scale and large-scale use cases. Swan includes two variants: Swan-Small, based on ARBERTv2, and Swan-Large, built on ArMistral, a pretrained Arabic large language model. To evaluate these models, we propose ArabicMTEB, a comprehensive benchmark suite that assesses cross-lingual, multi-dialectal, multi-domain, and multi-cultural Arabic text embedding performance, covering eight diverse tasks and spanning 94 datasets. Swan-Large achieves state-of-the-art results, outperforming Multilingual-E5-large in most Arabic tasks, while the Swan-Small consistently surpasses Multilingual-E5 base. Our extensive evaluations demonstrate that Swan models are both dialectally and culturally aware, excelling across various Arabic domains while offering significant monetary efficiency. This work significantly advances the field of Arabic language modelling and provides valuable resources for future research and applications in Arabic natural language processing. Our models and benchmark will be made publicly accessible for research.
Abstract (translated)
我们介绍Swan，这是一个以阿拉伯语为中心的嵌入模型系列，旨在解决小规模和大规模使用场景。Swan包括两个变体：基于ARBERTv2的Swan-Small，以及建立在预训练阿拉伯语大语言模型ArMistral之上的Swan-Large。为了评估这些模型，我们提出了ArabicMTEB，这是一个全面的基准测试套件，用于评估跨语言、多方言、多领域和多文化的阿拉伯文本嵌入性能，涵盖八种不同的任务，并跨越94个数据集。Swan-Large实现了最先进的结果，在大多数阿拉伯语任务中超越了Multilingual-E5-large，而Swan-Small则在几乎所有情况下都优于Multilingual-E5 base。我们的广泛评估表明，Swan模型既具备方言意识也具有文化意识，能够在各种阿拉伯语领域内表现出色，并且提供显著的成本效益。这项工作大大推进了阿拉伯语言建模领域的进展，并为未来阿拉伯自然语言处理的研究和应用提供了宝贵的资源。我们将公开发布我们的模型和基准测试供研究使用。
URL
https://arxiv.org/abs/2411.01192
