Abstract
As the online learning landscape evolves, the need for personalization is increasingly evident. Although educational resources are burgeoning, educators face challenges selecting materials that both align with intended learning outcomes and address diverse learner needs. Large Language Models (LLMs) are attracting growing interest for their potential to create learning resources that better support personalization, but verifying coverage of intended outcomes still requires human alignment review, which is costly and limits scalability. We propose a framework that supports the cost-effective automation of evaluating alignment between educational resources and intended learning outcomes. Using human-generated materials, we benchmarked LLM-based text-embedding models and found that the most accurate model (Voyage) achieved 79% accuracy in detecting alignment. We then applied the optimal model to LLM-generated resources and, via expert evaluation, confirmed that it reliably assessed correspondence to intended outcomes (83% accuracy). Finally, in a three-group experiment with 360 learners, higher alignment scores were positively related to greater learning performance, chi-squared(2, N = 360) = 15.39, p < 0.001. These findings show that embedding-based alignment scores can facilitate scalable personalization by confirming alignment with learning outcomes, which allows teachers to focus on tailoring content to diverse learner needs.
Abstract (translated)
随着在线学习领域的不断发展，个性化教学的需求变得越来越明显。尽管教育资源日益丰富，但教师在选择既能与预期的学习成果相符合又能满足多样化学生需求的材料时面临着挑战。大型语言模型（LLMs）因其有潜力生成更支持个性化的学习资源而引起了越来越多的兴趣，但是验证这些资源是否覆盖了预期的教学目标仍然需要人工审核，这既昂贵又限制了可扩展性。我们提出了一种框架，以实现对教育材料与预期学习成果之间一致性的评估的低成本自动化。 使用由人类创建的材料作为基准，我们测试了基于大型语言模型生成的文字嵌入（text-embedding）模型，并发现最准确的模型（Voyage）在检测一致性方面达到了79%的准确性。接着，我们将最佳模型应用于LLM生成的学习资源中，并通过专家评估确认该模型能够可靠地评估与预期成果的一致性，其准确率为83%。最后，在一个包括360名学习者的三组实验中，我们发现更高的一致性得分与更好的学习表现呈正相关关系，卡方检验结果为chi-squared(2, N = 360) = 15.39，p < 0.001。 这些研究结果表明，基于嵌入的一致性评分可以通过确认教育材料与学习成果之间的一致性来促进可扩展的个性化教学。这使教师能够专注于根据多样化学生需求调整内容，从而进一步提升教学质量。
URL
https://arxiv.org/abs/2512.13658
