Abstract
While many researchers use Large Language Models (LLMs) through chat-based access, their real potential lies in leveraging LLMs via application programming interfaces (APIs). This paper conceptualizes LLMs as universal text processing machines and presents a comprehensive workflow for employing LLMs in three qualitative and quantitative content analysis tasks: (1) annotation (an umbrella term for qualitative coding, labeling and text classification), (2) summarization, and (3) information extraction. The workflow is explicitly human-centered. Researchers design, supervise, and validate each stage of the LLM process to ensure rigor and transparency. Our approach synthesizes insights from extensive methodological literature across multiple disciplines: political science, sociology, computer science, psychology, and management. We outline validation procedures and best practices to address key limitations of LLMs, such as their black-box nature, prompt sensitivity, and tendency to hallucinate. To facilitate practical implementation, we provide supplementary materials, including a prompt library and Python code in Jupyter Notebook format, accompanied by detailed usage instructions.
Abstract (translated)
尽管许多研究者通过聊天界面使用大语言模型（LLMs），但其真正潜力在于通过应用程序编程接口（API）调用LLMs。本文提出将LLMs概念化为通用文本处理机器，并针对三类定性与定量内容分析任务——（1）标注（涵盖定性编码、标签与文本分类的统称）、（2）摘要生成、（3）信息抽取——提出了系统化的工作流程。该流程明确以人为中心：研究者主导设计、监督并验证LLM处理的每个阶段，以确保研究的严谨性与透明度。我们的方法综合了政治学、社会学、计算机科学、心理学及管理学等多学科的方法论文献，提出了针对LLMs关键局限性的验证方案与最佳实践，包括其黑箱特性、提示词敏感性及幻觉倾向。为便于实践应用，我们提供了补充材料，包含提示词库及Jupyter Notebook格式的Python代码，并附有详细使用说明。
URL
https://arxiv.org/abs/2603.19271
