Abstract
Social Media and Internet have the potential to be exploited as a source of opinion to enrich Decision Making solutions. Crowd Decision Making (CDM) is a methodology able to infer opinions and decisions from plain texts, such as reviews published in social media platforms, by means of Sentiment Analysis. Currently, the emergence and potential of Large Language Models (LLMs) lead us to explore new scenarios of automatically understand written texts, also known as natural language processing. This paper analyzes the use of ChatGPT based on prompt design strategies to assist in CDM processes to extract opinions and make decisions. We integrate ChatGPT in CDM processes as a flexible tool that infer the opinions expressed in texts, providing numerical or linguistic evaluations where the decision making models are based on the prompt design strategies. We include a multi-criteria decision making scenario with a category ontology for criteria. We also consider ChatGPT as an end-to-end CDM model able to provide a general opinion and score on the alternatives. We conduct empirical experiments on real data extracted from TripAdvisor, the TripR-2020Large dataset. The analysis of results show a promising branch for developing quality decision making models using ChatGPT. Finally, we discuss the challenges of consistency, sensitivity and explainability associated to the use of LLMs in CDM processes, raising open questions for future studies.
Abstract (translated)
社交媒体和互联网具有成为意见来源的潜力，以丰富决策解决方案。众包决策（CDM）是一种能够通过情感分析从社交媒体平台上的普通文本中推断观点和决策的 methodology。目前，大型语言模型的出现和潜在可能性引发了我们对自动理解书面文本（自然语言处理）的新场景的探索。本文分析基于提示设计策略使用 ChatGPT 协助 CDM 过程以提取观点和做出决策。我们将 ChatGPT 集成到 CDM 过程中，作为一种灵活的工具，推断文本中表达的观点，为决策模型提供数值或语言评价。我们还包括一个多标准决策场景和一个类别本体，用于定义标准。我们还考虑 ChatGPT 是一个端到端 CDM 模型，能够提供对替代方案的一般意见和评分。我们在 TripAdvisor 和 TripR-2020Large 数据集上进行实证研究。分析结果表明，使用 ChatGPT 可以帮助开发质量决策模型。最后，我们讨论了使用 LLMs 在 CDM 过程中存在的挑战、敏感性和可解释性，提出了未来研究的开放问题。
URL
https://arxiv.org/abs/2403.15587
