Abstract
This paper describes the winning solution of all 5 tasks for the Amazon KDD Cup 2024 Multi Task Online Shopping Challenge for LLMs. The challenge was to build a useful assistant, answering questions in the domain of online shopping. The competition contained 57 diverse tasks, covering 5 different task types (e.g. multiple choice) and across 4 different tracks (e.g. multi-lingual). Our solution is a single model per track. We fine-tune Qwen2-72B-Instruct on our own training dataset. As the competition released only 96 example questions, we developed our own training dataset by processing multiple public datasets or using Large Language Models for data augmentation and synthetic data generation. We apply wise-ft to account for distribution shifts and ensemble multiple LoRA adapters in one model. We employed Logits Processors to constrain the model output on relevant tokens for the tasks. AWQ 4-bit Quantization and vLLM are used during inference to predict the test dataset in the time constraints of 20 to 140 minutes depending on the track. Our solution achieved the first place in each individual track and is the first place overall of Amazons KDD Cup 2024.
Abstract (translated)
本文描述了亚马逊KDD Cup 2024多任务在线购物挑战中所有5个任务的获胜解决方案。挑战是在在线购物领域的回答问题，比赛包含了57个不同的任务，涵盖了5种不同的任务类型（例如多选题）和4个不同的赛道（例如多语言）。我们的解决方案是一个模型 per track。我们对自己的训练数据进行了微调。由于比赛只发布了96个示例问题，因此我们通过处理多个公共数据集或使用大型语言模型进行数据增强和合成数据生成来创建自己的训练数据。我们应用了 wise-ft 来处理分布漂移和模型输出对相关词的限制。在推理过程中，我们使用了 AWQ 4 位量化器和 vLLM 来预测基于任务的测试数据在20到140分钟内的时间约束。我们的解决方案在每条赛道上均获得了第一，在整体亚马逊KDD Cup 2024中也获得了第一。
URL
https://arxiv.org/abs/2408.04658
