Abstract
Multilingual language models have significantly advanced due to rapid progress in natural language processing. Models like BLOOM 1.7B, trained on diverse multilingual datasets, aim to bridge linguistic gaps. However, their effectiveness in capturing linguistic knowledge, particularly for low-resource languages, remains an open question. This study critically examines MLMs capabilities in multilingual understanding, semantic representation, and cross-lingual knowledge transfer. While these models perform well for high-resource languages, they struggle with less-represented ones. Additionally, traditional evaluation methods often overlook their internal syntactic and semantic encoding. This research addresses key limitations through three objectives. First, it assesses semantic similarity by analyzing multilingual word embeddings for consistency using cosine similarity. Second, it examines BLOOM-1.7B and Qwen2 through Named Entity Recognition and sentence similarity tasks to understand their linguistic structures. Third, it explores cross-lingual knowledge transfer by evaluating generalization from high-resource to low-resource languages in sentiment analysis and text classification. By leveraging linguistic probing, performance metrics, and visualizations, this study provides insights into the strengths and limitations of MLMs. The findings aim to enhance multilingual NLP models, ensuring better support for both high- and low-resource languages, thereby promoting inclusivity in language technologies.
Abstract (translated)
多语言语言模型由于自然语言处理的迅速进步而得到了显著的发展。像BLOOM 1.7B这样的模型，在多样化的多语种数据集上进行训练，旨在弥合语言鸿沟。然而，它们在捕捉低资源语言的语言知识方面的有效性仍然是一个开放性问题。这项研究从批判的角度审视了多语言语言模型（MLMs）在跨语言理解、语义表示和跨语言知识迁移方面的能力。虽然这些模型对于高资源语言表现良好，但在处理代表性不足的语言时却面临挑战。此外，传统的评估方法通常忽视了它们内部的句法和语义编码。这项研究通过三个目标来解决关键限制：首先，它通过使用余弦相似度分析多语言词嵌入的一致性来评估语义相似性；其次，它通过命名实体识别和句子相似性任务来考察BLOOM-1.7B和Qwen2的语言结构理解能力；第三，它通过从高资源语言到低资源语言的情感分析和文本分类中的泛化性能评估跨语言知识迁移。该研究利用了语言探测、性能指标和可视化技术，从而为多语言自然语言处理（NLP）模型的优势和局限性提供了见解。这些发现旨在增强多语言NLP模型，确保对高资源和低资源语言都提供更好的支持，进而促进语言技术的包容性。
URL
https://arxiv.org/abs/2502.04269
