Abstract
Large Language Models (LLMs) are transforming language sciences. However, their widespread deployment currently suffers from methodological fragmentation and a lack of systematic soundness. This study proposes two comprehensive methodological frameworks designed to guide the strategic and responsible application of LLMs in language sciences. The first method-selection framework defines and systematizes three distinct, complementary approaches, each linked to a specific research goal: (1) prompt-based interaction with general-use models for exploratory analysis and hypothesis generation; (2) fine-tuning of open-source models for confirmatory, theory-driven investigation and high-quality data generation; and (3) extraction of contextualized embeddings for further quantitative analysis and probing of model internal mechanisms. We detail the technical implementation and inherent trade-offs of each method, supported by empirical case studies. Based on the method-selection framework, the second systematic framework proposed provides constructed configurations that guide the practical implementation of multi-stage research pipelines based on these approaches. We then conducted a series of empirical experiments to validate our proposed framework, employing retrospective analysis, prospective application, and an expert evaluation survey. By enforcing the strategic alignment of research questions with the appropriate LLM methodology, the frameworks enable a critical paradigm shift in language science research. We believe that this system is fundamental for ensuring reproducibility, facilitating the critical evaluation of LLM mechanisms, and providing the structure necessary to move traditional linguistics from ad-hoc utility to verifiable, robust science.
Abstract (translated)
大型语言模型（LLMs）正在重塑语言科学。然而，它们的广泛应用目前面临着方法论上的碎片化和系统性不足的问题。本研究提出了两个全面的方法框架，旨在指导大型语言模型在语言科学研究中的战略性和负责任的应用。第一个方法选择框架定义并系统化了三种不同的互补方法，每种方法都与特定的研究目标相关联：(1) 通过通用模型的提示交互进行探索性分析和假设生成；(2) 对开源模型进行微调以开展理论驱动的确证性研究以及高质量数据生成；(3) 提取上下文化的嵌入以便进一步进行定量分析和探究模型内部机制。我们详细说明了每种方法的技术实现及其内在权衡，并通过实证案例研究来支持这些观点。 基于该方法选择框架，第二个系统化框架提出了构建配置，为根据这三种方法实施多阶段的研究流程提供指导。随后，我们进行了系列实证实验以验证提出的框架的有效性，包括回顾分析、前瞻性应用和专家评估调查。通过确保研究问题与适当的大型语言模型方法的战略一致性，这些框架能够引发语言科学研究中的关键范式转变。我们认为，这一系统对于确保可重复性、促进对LLM机制的批判性评价以及提供将传统语言学从临时实用性提升到经验证实的强大科学所需的基本结构至关重要。
URL
https://arxiv.org/abs/2512.09552
