Abstract
In this article, we present our approach to single-modality vision representation learning. Understanding vision representations of product content is vital for recommendations, search, and advertising applications in e-commerce. We detail and contrast techniques used to fine tune large-scale vision representation learning models in an efficient manner under low-resource settings, including several pretrained backbone architectures, both in the convolutional neural network as well as the vision transformer family. We highlight the challenges for e-commerce applications at-scale and highlight the efforts to more efficiently train, evaluate, and serve visual representations. We present ablation studies for several downstream tasks, including our visually similar ad recommendations. We evaluate the offline performance of the derived visual representations in downstream tasks. To this end, we present a novel text-to-image generative offline evaluation method for visually similar recommendation systems. Finally, we include online results from deployed machine learning systems in production at Etsy.
Abstract (translated)
在本文中，我们介绍了我们Single-modality Vision Representation Learning的方法。理解产品内容的 Vision Representation 对于电子商务中的推荐、搜索和广告应用至关重要。我们详细对比了在资源非常有限的情况下，如何高效地优化大规模 Vision Representation 学习模型的方法，其中包括在卷积神经网络和视觉转换器家族中多个预训练主干架构的方法。我们强调了 Scale 上电子商务应用面临的挑战，并重点介绍了更有效地训练、评估和提供服务视觉Representation 的努力。我们介绍了多个后续任务的实验结果，包括我们类似的广告推荐。我们评估了后续任务中衍生的视觉Representation 的离线表现。为此，我们提出了一种 novel 的文本到图像生成离线评估方法，适用于类似的推荐系统。最后，我们包括在Etsy 生产中的部署机器学习系统的离线结果。
URL
https://arxiv.org/abs/2305.13399
