Abstract
There has been enormous interest in generative AI since ChatGPT was launched in 2022. However, there are concerns about the accuracy and consistency of the outputs of generative AI. We have carried out an exploratory study on the application of this new technology in research data processing. We identified tasks for which rule-based or traditional machine learning approaches were difficult to apply, and then performed these tasks using generative AI. We demonstrate the feasibility of using the generative AI model Claude 3 Opus in three research projects involving complex data processing tasks: 1) Information extraction: We extract plant species names from historical seedlists (catalogues of seeds) published by botanical gardens. 2) Natural language understanding: We extract certain data points (name of drug, name of health indication, relative effectiveness, cost-effectiveness, etc.) from documents published by Health Technology Assessment organisations in the EU. 3) Text classification: We assign industry codes to projects on the crowdfunding website Kickstarter. We share the lessons we learnt from these use cases: How to determine if generative AI is an appropriate tool for a given data processing task, and if so, how to maximise the accuracy and consistency of the results obtained.
Abstract (translated)
自2022年ChatGPT推出以来，生成式人工智能引起了极大的兴趣。然而，人们对生成式AI输出的准确性和一致性表示担忧。我们进行了一项探索性研究，探讨这项新技术在研究数据处理中的应用情况。我们确定了规则基础或传统机器学习方法难以应对的任务，并使用生成式AI执行这些任务。我们在三个涉及复杂数据处理的研究项目中展示了Claude 3 Opus这一生成式AI模型的应用可行性： 1. **信息提取**：从植物园出版的历史种子目录（种子名录）中抽取植物物种名称。 2. **自然语言理解**：从欧盟健康技术评估组织发布的文档中抽取特定的数据点，例如药物名称、健康指征名称、相对效果和成本效益等。 3. **文本分类**：为众筹网站Kickstarter上的项目分配行业代码。 我们分享了在这些应用场景中学到的经验教训：如何判断生成式AI是否适合某一具体数据处理任务，并且如果合适的话，如何最大程度地提高所得结果的准确性和一致性。
URL
https://arxiv.org/abs/2504.15829
