Abstract
We explore the use of small language models (SLMs) for automatic question generation as a complement to the prevalent use of their large counterparts in learning analytics research. We present a novel question generation pipeline that leverages both the text generation and the probabilistic reasoning abilities of SLMs to generate high-quality questions. Adopting a "generate-then-validate" strategy, our pipeline first performs expansive generation to create an abundance of candidate questions and refine them through selective validation based on novel probabilistic reasoning. We conducted two evaluation studies, one with seven human experts and the other with a large language model (LLM), to assess the quality of the generated questions. Most judges (humans or LLMs) agreed that the generated questions had clear answers and generally aligned well with the intended learning objectives. Our findings suggest that an SLM can effectively generate high-quality questions when guided by a well-designed pipeline that leverages its strengths.
Abstract (translated)
我们探讨了使用小型语言模型（SLM）进行自动问题生成的用途，作为在学习分析研究中广泛使用的大型模型的一种补充。我们提出了一种新颖的问题生成管道，该管道利用了SLM的文字生成能力和概率推理能力来产生高质量的问题。采用“先生成再验证”的策略，我们的管道首先通过广泛的生成过程创建了大量的候选问题，并通过基于新型概率推理的精选验证对其进行细化和优化。 为了评估所生成问题的质量，我们进行了两项评价研究：一项是与七位人类专家合作进行的研究；另一项则是借助大型语言模型（LLM）开展的研究。大多数评判者（无论是人类还是LLM）都认为生成的问题有明确的答案，并且通常符合预期的学习目标。我们的发现表明，在一个设计得当的管道指导下，SLM能够有效地生成高质量的问题，该管道充分地利用了其优势。
URL
https://arxiv.org/abs/2512.10110
