Abstract
Using natural language, Conversational Bot offers unprecedented ways to many challenges in areas such as information searching, item recommendation, and question answering. Existing bots are usually developed through retrieval-based or generative-based approaches, yet both of them have their own advantages and disadvantages. To assemble this two approaches, we propose a hybrid retrieval-generation network (HeroNet) with the three-fold ideas: 1). To produce high-quality sentence representations, HeroNet performs multi-task learning on two subtasks: Similar Queries Discovery and Query-Response Matching. Specifically, the retrieval performance is improved while the model size is reduced by training two lightweight, task-specific adapter modules that share only one underlying T5-Encoder model. 2). By introducing adversarial training, HeroNet is able to solve both retrieval\&generation tasks simultaneously while maximizing performance of each other. 3). The retrieval results are used as prior knowledge to improve the generation performance while the generative result are scored by the discriminator and their scores are integrated into the generator's cross-entropy loss function. The experimental results on a open dataset demonstrate the effectiveness of the HeroNet and our code is available at this https URL
Abstract (translated)
使用自然语言,对话机器人提供了前所未有的方式来解决信息搜索、物品推荐和问答等领域中的许多挑战。现有的机器人通常通过检索或生成方法来开发,但这两种方法都有其优点和缺点。为了组合这两种方法,我们提出了一种混合检索生成网络( HeroNet),采用了三个方面的想法: 1). 为了提高句子表示质量, HeroNet 在两个子任务上执行多任务学习:相似查询发现和查询响应匹配。具体来说,检索性能得到了提高,而模型大小通过训练两个轻量级、任务特定的适应器模块而缩小,这些模块仅共享一个 T5 编码器模型。2). 通过引入对抗训练, HeroNet可以同时解决检索和生成任务,并最大化彼此的性能。3). 检索结果用作先前知识,以提高生成性能,而生成结果由判别器评分,其评分被集成到生成器的污染熵损失函数中。在一个开放数据集上的实验结果证明了 HeroNet 的有效性,我们的代码可在该 https URL 上可用。
URL
https://arxiv.org/abs/2301.12400
