Abstract
This work evaluates FinGPT, a financial domain-specific language model, across six key natural language processing (NLP) tasks: Sentiment Analysis, Text Classification, Named Entity Recognition, Financial Question Answering, Text Summarization, and Stock Movement Prediction. The evaluation uses finance-specific datasets to assess FinGPT's capabilities and limitations in real-world financial applications. The results show that FinGPT performs strongly in classification tasks such as sentiment analysis and headline categorization, often achieving results comparable to GPT-4. However, its performance is significantly lower in tasks that involve reasoning and generation, such as financial question answering and summarization. Comparisons with GPT-4 and human benchmarks highlight notable performance gaps, particularly in numerical accuracy and complex reasoning. Overall, the findings indicate that while FinGPT is effective for certain structured financial tasks, it is not yet a comprehensive solution. This research provides a useful benchmark for future research and underscores the need for architectural improvements and domain-specific optimization in financial language models.
Abstract (translated)
这项工作评估了FinGPT，这是一种专门针对金融领域的语言模型，并对其在六个关键自然语言处理（NLP）任务中的表现进行了评价：情感分析、文本分类、命名实体识别、财务问答、文本摘要和股票价格走势预测。该评估使用了特定于金融的数据集来考察FinGPT在现实世界金融应用中的能力和局限性。 结果显示，FinGPT在诸如情感分析和新闻标题分类等分类任务中表现出色，并经常达到与GPT-4相媲美的结果水平。然而，在涉及推理和生成的任务（如财务问答和摘要生成）方面，它的表现显著较低。与GPT-4及人类基准的比较表明了明显的性能差距，特别是在数值准确性和复杂推理能力上。 总体而言，研究发现表明尽管FinGPT在某些结构化的金融任务中是有效的，但它还不是一个全面的解决方案。这项研究为未来的研究提供了有用的基准，并强调了对金融语言模型架构改进和领域特定优化的需求。
URL
https://arxiv.org/abs/2507.08015
