Abstract
Obstacles hindering the development of capsule networks for challenging NLP applications include poor scalability to large output spaces and less reliable routing processes. In this paper, we introduce: 1) an agreement score to evaluate the performance of routing processes at instance level; 2) an adaptive optimizer to enhance the reliability of routing; 3) capsule compression and partial routing to improve the scalability of capsule networks. We validate our approach on two NLP tasks, namely: multi-label text classification and question answering. Experimental results show that our approach considerably improves over strong competitors on both tasks. In addition, we gain the best results in low-resource settings with few training instances.
Abstract (translated)
阻碍胶囊网络发展以挑战NLP应用的障碍包括对大输出空间的不可扩展性和不太可靠的路由过程。在本文中,我们介绍了:1)在实例级评估路由过程性能的协议评分;2)增强路由可靠性的自适应优化器;3)胶囊压缩和部分路由以提高胶囊网络的可扩展性。我们在两个NLP任务上验证了我们的方法,即:多标签文本分类和问题解答。实验结果表明,在这两项任务上,我们的方法比强大的竞争对手有了很大的改进。此外,我们在低资源环境中获得最佳结果,培训实例很少。
URL
https://arxiv.org/abs/1906.02829