Abstract
The growing prominence of large language models (LLMs) necessitates the exploration of their capabilities beyond English. This research investigates the Telugu language proficiency of ChatGPT and Gemini, two leading LLMs. Through a designed set of 20 questions encompassing greetings, grammar, vocabulary, common phrases, task completion, and situational reasoning, the study delves into their strengths and weaknesses in handling Telugu. The analysis aims to identify the LLM that demonstrates a deeper understanding of Telugu grammatical structures, possesses a broader vocabulary, and exhibits superior performance in tasks like writing and reasoning. By comparing their ability to comprehend and use everyday Telugu expressions, the research sheds light on their suitability for real-world language interaction. Furthermore, the evaluation of adaptability and reasoning capabilities provides insights into how each LLM leverages Telugu to respond to dynamic situations. This comparative analysis contributes to the ongoing discussion on multilingual capabilities in AI and paves the way for future research in developing LLMs that can seamlessly integrate with Telugu-speaking communities.
Abstract (translated)
大语言模型（LLMs）越来越突出，需要探索其超越英语的能力。这项研究调查了ChatGPT和Gemini这两个领先LLM在泰米尔语上的能力。通过设计一组20个问题涵盖问候、语法、词汇、常用短语、任务完成和情境推理，研究深入探讨了它们在处理泰米尔语方面的优势和不足。分析旨在确定展示对泰米尔语语法结构有更深刻理解、具有更丰富词汇、并且在写作和推理任务上表现优异的LLM。通过比较它们理解和使用日常泰米尔语表达的能力，这项研究揭示了它们在现实语言交互中的适用性。此外，对可扩展性和推理能力评估提供了关于每个LLM如何利用泰米尔语应对动态情况的见解。这种比较分析为讨论人工智能的多语言能力以及为开发可以无缝融入泰米尔语社区LLM的研究奠定了基础。
URL
https://arxiv.org/abs/2404.19369
