Abstract
While Curriculum learning (CL) has recently gained traction in Natural language Processing Tasks, it still isn't being analyzed adequately. Previous works only show their effectiveness but fail short to fully explain and interpret the internal workings. In this paper, we analyze curriculum learning in sentiment analysis along multiple axes. Some of these axes have been proposed by earlier works that need deeper study. Such analysis requires understanding where curriculum learning works and where it doesn't. Our axes of analysis include Task difficulty on CL, comparing CL pacing techniques, and qualitative analysis by visualizing the movement of attention scores in the model as curriculum phases progress. We find that curriculum learning works best for difficult tasks and may even lead to a decrement in performance for tasks that have higher performance without curriculum learning. Within curriculum pacing, we see that One-Pass curriculum strategies suffer from catastrophic forgetting and attention movement visualization shows that curriculum learning breaks down the main task into easier sub-tasks which the model solves easily.
Abstract (translated)
URL
https://arxiv.org/abs/2102.09990
