Abstract
ChatGPT and other state-of-the-art large language models (LLMs) are rapidly transforming multiple fields, offering powerful tools for a wide range of applications. These models, commonly trained on vast datasets, exhibit human-like text generation capabilities, making them useful for research tasks such as ideation, literature review, coding, drafting, and outreach. We conducted a study involving 13 astronomers at different career stages and research fields to explore LLM applications across diverse tasks over several months and to evaluate their performance in research-related activities. This work was accompanied by an anonymous survey assessing participants' experiences and attitudes towards LLMs. We provide a detailed analysis of the tasks attempted and the survey answers, along with specific output examples. Our findings highlight both the potential and limitations of LLMs in supporting research while also addressing general and research-specific ethical considerations. We conclude with a series of recommendations, emphasizing the need for researchers to complement LLMs with critical thinking and domain expertise, ensuring these tools serve as aids rather than substitutes for rigorous scientific inquiry.
Abstract (translated)
ChatGPT和其他大型语言模型(LLMs)正在迅速改变多个领域,为各种应用提供了强大的工具。这些模型通常在庞大的数据集上进行训练,具有类似于人类的文本生成能力,因此它们在研究任务(如创意激发、文献综述、编码、起草和外联)中具有很大的价值。 我们进行了一项研究,涉及13位不同职业阶段和研究领域的天文学家,在数月的时间里探讨了LLM在各种任务上的应用,以评估它们在研究活动中的表现。这项工作还有匿名调查,评估参与者的LLM经验和态度。我们提供了任务尝试分析和调查答案的具体输出示例。 我们的研究结果突出了LLM在支持研究方面的潜力和限制,并探讨了通用和研究的道德考虑。我们得出结论,建议研究人员在LLM中增加批判性思维和专业知识,确保这些工具成为严谨科学研究的辅助,而不是替代品。
URL
https://arxiv.org/abs/2409.20252