Abstract
Click-through rate (CTR) prediction plays an important role in personalized recommendations. Recently, sample-level retrieval-based models (e.g., RIM) have achieved remarkable performance by retrieving and aggregating relevant samples. However, their inefficiency at the inference stage makes them impractical for industrial applications. To overcome this issue, this paper proposes a universal plug-and-play Retrieval-Oriented Knowledge (ROK) framework. Specifically, a knowledge base, consisting of a retrieval-oriented embedding layer and a knowledge encoder, is designed to preserve and imitate the retrieved & aggregated representations in a decomposition-reconstruction paradigm. Knowledge distillation and contrastive learning methods are utilized to optimize the knowledge base, and the learned retrieval-enhanced representations can be integrated with arbitrary CTR models in both instance-wise and feature-wise manners. Extensive experiments on three large-scale datasets show that ROK achieves competitive performance with the retrieval-based CTR models while reserving superior inference efficiency and model compatibility.
Abstract (translated)
点击率（CTR）预测在个性化推荐中扮演着重要角色。最近，基于样本级检索的模型（例如，RIM）通过检索和聚合相关样本取得了显著的性能。然而，他们在推理阶段的高效率使得它们对于工业应用不实用。为了克服这个问题，本文提出了一种通用的挂载即插即用检索导向知识（ROK）框架。具体来说，一个知识库，包括一个检索嵌入层和一个知识编码器，被设计成保留并模仿在分解重建范式中检索到的&聚合表示。使用知识蒸馏和对比学习方法来优化知识库，学到的检索增强表示可以以实例和特征的方式与任意CTR模型集成。在三个大型数据集上的实验表明，ROK在检索基于CTR的模型方面具有竞争力的性能，同时保留卓越的推理效率和模型兼容性。
URL
https://arxiv.org/abs/2404.18304
