Abstract
Meta-learning is widely used for few-shot slot tagging in the task of few-shot learning. The performance of existing methods is, however, seriously affected by catastrophic forgetting. This phenomenon is common in deep learning as the training and testing modules fail to take into account historical information, i.e. previously trained episodes in the metric-based meta-learning. To overcome this predicament, we propose the Memory-based Contrastive Meta-learning (MCML) method. Specifically, we propose a learn-from-memory mechanism that use explicit memory to keep track of the label representations of previously trained episodes and propose a contrastive learning method to compare the current label embedded in the few shot episode with the historic ones stored in the memory, and an adaption-from memory mechanism to determine the output label based on the contrast between the input labels embedded in the test episode and the label clusters in the memory. Experimental results show that MCML is scalable and outperforms metric-based meta-learning and optimization-based meta-learning on all 1shot, 5-shot, 10-shot, and 20-shot scenarios of the SNIPS dataset.
Abstract (translated)
URL
https://arxiv.org/abs/2108.11635
