Abstract
Traditional recommender systems such as matrix factorization methods rely on learning a shared dense embedding space to represent both items and user preferences. Sequence models such as RNN, GRUs, and, recently, Transformers have also excelled in the task of sequential recommendation. This task requires understanding the sequential structure present in users' historical interactions to predict the next item they may like. Building upon the success of Large Language Models (LLMs) in a variety of tasks, researchers have recently explored using LLMs that are pretrained on vast corpora of text for sequential recommendation. To use LLMs in sequential recommendations, both the history of user interactions and the model's prediction of the next item are expressed in text form. We propose CALRec, a two-stage LLM finetuning framework that finetunes a pretrained LLM in a two-tower fashion using a mixture of two contrastive losses and a language modeling loss: the LLM is first finetuned on a data mixture from multiple domains followed by another round of target domain finetuning. Our model significantly outperforms many state-of-the-art baselines (+37% in Recall@1 and +24% in NDCG@10) and systematic ablation studies reveal that (i) both stages of finetuning are crucial, and, when combined, we achieve improved performance, and (ii) contrastive alignment is effective among the target domains explored in our experiments.
Abstract (translated)
传统推荐系统，如矩阵分解方法，依赖于学习共同的密集嵌入空间来表示物品和用户偏好。序列模型，如RNN、GRU和最近的自然语言处理模型Transformer，也在序列推荐任务中表现出色。这项任务需要理解用户历史交互中的序列结构，以预测他们可能喜欢下一个项目。在大型语言模型（LLMs）在各种任务上的成功的基础上，研究人员最近探索了使用LLMs进行序列推荐。要使用LLMs进行序列推荐，用户历史交互的背景和模型的预测下一个项目的文本形式都需要表达。我们提出了CALRec，一种两阶段LLM微调框架，它通过混合两种对比性损失和一个语言建模损失，在两层结构中对预训练的LLM进行微调：首先对来自多个领域的数据混合进行微调，然后进行目标域微调。我们的模型在许多最先进的基线（+37%的召回度@1和+24%的NDCG@10）上都显著超过了很多状态-of-the-art，而且系统消融研究揭示了（i）微调阶段是至关重要的，而且，当结合时，我们实现更好的性能，（ii）在探索的各个目标域中，对比性对目标域之间的差异有效的特点。
URL
https://arxiv.org/abs/2405.02429
