Abstract
Educational technology innovations that have been developed based on large language models (LLMs) have shown the potential to automate the laborious process of generating and analysing textual content. While various innovations have been developed to automate a range of educational tasks (e.g., question generation, feedback provision, and essay grading), there are concerns regarding the practicality and ethicality of these innovations. Such concerns may hinder future research and the adoption of LLMs-based innovations in authentic educational contexts. To address this, we conducted a systematic literature review of 118 peer-reviewed papers published since 2017 to pinpoint the current state of research on using LLMs to automate and support educational tasks. The practical and ethical challenges of LLMs-based innovations were also identified by assessing their technological readiness, model performance, replicability, system transparency, privacy, equality, and beneficence. The findings were summarised into three recommendations for future studies, including updating existing innovations with state-of-the-art models (e.g., GPT-3), embracing the initiative of open-sourcing models/systems, and adopting a human-centred approach throughout the developmental process. These recommendations could support future research to develop practical and ethical innovations for supporting diverse educational tasks and benefiting students, teachers, and institutions.
Abstract (translated)
基于大型语言模型(LLMs)的开发出来的教育技术创新，展现出了自动化生成和分析文本内容的潜力。虽然各种创新旨在自动化多种教育任务(例如问题生成、反馈提供和作文评估)，但对这些创新的实用性和道德性存在担忧。这些担忧可能会阻碍未来研究和在真实教育环境中采用LLMs-based创新。为了解决这个问题，我们进行了一项系统性的文献综述，自2017年以来发表了118篇论文，以确定使用LLMs来自动化和支持教育任务的最新研究状态。通过评估LLMs的创新技术的可行性、模型性能、可重复性、系统透明度、隐私、平等和福利，我们也确定了LLMs-based创新的实际和道德挑战。总结这些发现，我们提出了三个未来的研究建议，包括更新现有的创新使用最先进的模型(例如GPT-3)、拥抱开源模型/系统的倡议，并在整个发展过程中采用人中心的方法。这些建议可支持未来研究开发支持多种教育任务的实际和道德创新，以造福学生、教师和机构。
URL
https://arxiv.org/abs/2303.13379
