Abstract
Recommender selects and presents top-K items to the user at each online request, and a recommendation session consists of several sequential requests. Formulating a recommendation session as a Markov decision process and solving it by reinforcement learning (RL) framework has attracted increasing attention from both academic and industry communities. In this paper, we propose a RL-based industrial short-video recommender ranking framework, which models and maximizes user watch-time in an environment of user multi-aspect preferences by a collaborative multi-agent formulization. Moreover, our proposed framework adopts a model-based learning approach to alleviate the sample selection bias which is a crucial but intractable problem in industrial recommender system. Extensive offline evaluations and live experiments confirm the effectiveness of our proposed method over alternatives. Our proposed approach has been deployed in our real large-scale short-video sharing platform, successfully serving over hundreds of millions users.
Abstract (translated)
推荐系统选择并提供给用户每个在线请求的前K个物品，而推荐会话包括多个连续的请求。将推荐会话表示为一个马尔可夫决策过程并通过强化学习（RL）框架求解，已经吸引了学术界和工业界的越来越多的关注。在本文中，我们提出了一个基于强化学习的工业短视频推荐排名框架，该框架通过多智能体协同公式建模并最大化用户的观看时间来模拟和优化用户的多方面偏好。此外，我们所提出的框架采用基于模型的学习方法来减轻工业推荐系统中样本选择偏差这个关键但难以解决的问题。 广泛的离线评估和现场实验证实了我们所提出方法的有效性超过了其他替代方案。我们的方法已经成功部署在我们的大型视频分享平台上，为数超过数百万用户提供了服务。
URL
https://arxiv.org/abs/2405.01847
