Abstract
Input features play a crucial role in the predictive performance of DNN-based industrial recommender systems with thousands of categorical and continuous fields from users, items, contexts, and their interactions. Noisy features and inappropriate embedding dimension assignments can impair the performance of recommender systems and introduce unnecessary complexity in model training and online serving. Optimizing the input configuration of DNN models, including feature selection and embedding dimension assignment, has become one of the essential topics in feature engineering. Typically, feature selection and embedding dimension search are optimized sequentially, i.e., feature selection is performed first, followed by embedding dimension search to determine the optimal dimension size for each selected feature. In contrast, this paper studies the joint optimization of feature selection and embedding dimension search. To this end, we propose a differentiable neural \textbf{i}nput \textbf{razor}, namely \textbf{i-Razor}. Specifically, inspired by recent advances in neural architecture search, we introduce an end-to-end differentiable model to learn the relative importance between different embedding regions of each feature. Furthermore, a flexible pruning algorithm is proposed to simultaneously achieve feature filtering and dimension size derivation. Extensive experiments on two large-scale public datasets in the Click-Through-Rate (CTR) prediction task demonstrate the efficacy and superiority of i-Razor in balancing model complexity and performance.
Abstract (translated)
URL
https://arxiv.org/abs/2204.00281
