Abstract
This paper introduces UQA, a novel dataset for question answering and text comprehension in Urdu, a low-resource language with over 70 million native speakers. UQA is generated by translating the Stanford Question Answering Dataset (SQuAD2.0), a large-scale English QA dataset, using a technique called EATS (Enclose to Anchor, Translate, Seek), which preserves the answer spans in the translated context paragraphs. The paper describes the process of selecting and evaluating the best translation model among two candidates: Google Translator and Seamless M4T. The paper also benchmarks several state-of-the-art multilingual QA models on UQA, including mBERT, XLM-RoBERTa, and mT5, and reports promising results. For XLM-RoBERTa-XL, we have an F1 score of 85.99 and 74.56 EM. UQA is a valuable resource for developing and testing multilingual NLP systems for Urdu and for enhancing the cross-lingual transferability of existing models. Further, the paper demonstrates the effectiveness of EATS for creating high-quality datasets for other languages and domains. The UQA dataset and the code are publicly available at this http URL.
Abstract (translated)
本文介绍了一个名为UQA的新型数据集，用于 Urdu 语料库中问题回答和文本理解。UQA 由翻译斯坦福问题回答数据集 (SQuAD2.0) 生成，这是一个大型英语问题回答数据集，使用一种称为 EATS (将括号内保留翻译文本上下文中的答案范围) 的技术生成。本文描述了从两个候选者（Google 翻译器和 Seamless M4T）中选择和评估最佳翻译模型的过程。此外，本文还在 UQA 上基准了多种最先进的跨语言 QA 模型，包括 mBERT、XLM-RoBERTa 和 mT5，并报告了有希望的结果。对于 XLM-RoBERTa-XL，我们的 F1 分数为 85.99 和 74.56。UQA 是一个有价值的资源，可用于开发和测试 Urdu 和其他多语言 NLP 系统，以及增强现有模型的跨语言可转移性。此外，本文还证明了 EATS 在创建高质量数据集在其他语言和领域中的有效性。UQA 数据集和代码可在此处下载：<http://www.aclweb.org/anthology/N18-1196>
URL
https://arxiv.org/abs/2405.01458
