Abstract
Predicting stock prices presents challenges in financial forecasting. While traditional approaches such as ARIMA and RNNs are prevalent, recent developments in Large Language Models (LLMs) offer alternative methodologies. This paper introduces an approach that integrates LLMs with daily financial news for stock price prediction. To address the challenge of processing news data and identifying relevant content, we utilize stock name embeddings within attention mechanisms. Specifically, we encode news articles using a pre-trained LLM and implement three attention-based pooling techniques -- self-attentive, cross-attentive, and position-aware self-attentive pooling -- to filter news based on stock relevance. The filtered news embeddings, combined with historical stock prices, serve as inputs to the prediction model. Unlike prior studies that focus on individual stocks, our method trains a single generalized model applicable across multiple stocks. Experimental results demonstrate a 7.11% reduction in Mean Absolute Error (MAE) compared to the baseline, indicating the utility of stock name embeddings for news filtering and price forecasting within a generalized framework.
Abstract (translated)
股票价格预测在金融预测领域面临挑战。尽管ARIMA和RNN等传统方法应用广泛，但大语言模型（LLMs）的最新进展提供了替代方案。本文提出一种将LLM与每日财经新闻相结合用于股价预测的方法。为应对新闻数据处理及识别相关内容的挑战，我们在注意力机制中引入股票名称嵌入。具体而言，我们使用预训练LLM编码新闻文章，并实施三种基于注意力的池化技术——自注意力池化、交叉注意力池化及位置感知自注意力池化——依据股票相关性过滤新闻。经筛选的新闻嵌入与历史股价共同作为预测模型输入。与以往聚焦个股的研究不同，我们的方法训练出适用于多支股票的单一个体化通用模型。实验结果显示，相比基线方法，平均绝对误差（MAE）降低7.11%，表明在通用框架中，股票名称嵌入对新闻筛选与价格预测具有实用价值。
URL
https://arxiv.org/abs/2603.19286
