Abstract
Customer-provided reviews have become an important source of information for business owners and other customers alike. However, effectively analyzing millions of unstructured reviews remains challenging. While large language models (LLMs) show promise for natural language understanding, their application to large-scale review analysis has been limited by computational costs and scalability concerns. This study proposes a hybrid approach that uses LLMs for aspect identification while employing classic machine-learning methods for sentiment classification at scale. Using ChatGPT to analyze sampled restaurant reviews, we identified key aspects of dining experiences and developed sentiment classifiers using human-labeled reviews, which we subsequently applied to 4.7 million reviews collected over 17 years from a major online platform. Regression analysis reveals that our machine-labeled aspects significantly explain variance in overall restaurant ratings across different aspects of dining experiences, cuisines, and geographical regions. Our findings demonstrate that combining LLMs with traditional machine learning approaches can effectively automate aspect-based sentiment analysis of large-scale customer feedback, suggesting a practical framework for both researchers and practitioners in the hospitality industry and potentially, other service sectors.
Abstract (translated)
客户提供的评论已经成为商家和其他顾客获取信息的重要来源。然而，有效地分析数以百万计的非结构化评论仍然是一项挑战。虽然大型语言模型（LLM）在自然语言理解方面显示出潜力，但其在大规模评价分析中的应用因计算成本和可扩展性问题而受到限制。本研究提出了一种混合方法，利用LLM进行方面识别，并采用传统的机器学习方法进行大规模情感分类。通过使用ChatGPT对采样的餐厅评论进行分析，我们确定了用餐体验的关键方面，并基于人工标注的评论开发了情感分类器，随后将其应用于17年间从主要在线平台收集到的470万条评论中。回归分析表明，我们的机器标注的方面在不同用餐体验、菜肴类型和地区方面显著解释了餐厅总体评分的变化。研究结果证明，结合LLM和传统机器学习方法可以有效地自动化大规模客户反馈的情感方面分析，这为酒店业研究人员和从业人员以及可能的服务行业提供了实用框架。
URL
https://arxiv.org/abs/2602.21082
