Abstract
Meta learning has been widely used to exploit rich-resource source tasks to improve the performance of low-resource target tasks. Unfortunately, most existing meta learning approaches treat different source tasks equally, ignoring the relatedness of source tasks to the target task in knowledge transfer. To mitigate this issue, we propose a reinforcement-based multi-source meta-transfer learning framework (Meta-RTL) for low-resource commonsense reasoning. In this framework, we present a reinforcement-based approach to dynamically estimating source task weights that measure the contribution of the corresponding tasks to the target task in the meta-transfer learning. The differences between the general loss of the meta model and task-specific losses of source-specific temporal meta models on sampled target data are fed into the policy network of the reinforcement learning module as rewards. The policy network is built upon LSTMs that capture long-term dependencies on source task weight estimation across meta learning iterations. We evaluate the proposed Meta-RTL using both BERT and ALBERT as the backbone of the meta model on three commonsense reasoning benchmark datasets. Experimental results demonstrate that Meta-RTL substantially outperforms strong baselines and previous task selection strategies and achieves larger improvements on extremely low-resource settings.
Abstract (translated)
元学习已被广泛应用于从丰富资源源任务中利用到低资源目标任务的性能提升。然而，大多数现有的元学习方法将不同的源任务平等对待，忽略了源任务与目标任务之间的相关性。为了减轻这个问题，我们提出了一个基于强化学习的多源元转移学习框架（Meta-RTL）用于低资源常识推理。在这个框架中，我们提出了一个基于强化的方法来动态估计源任务权重，该权重衡量了相应任务对目标任务的贡献。元模型的一般损失和特定源元模型的任务特定损失在随机目标数据上的差异被输入到元学习模型的策略网络作为奖励。策略网络基于LSTM，捕捉了元学习迭代过程中源任务权重估计的长远依赖关系。我们使用BERT和ALBERT作为元模型的元核，在三个常识推理基准数据集上评估所提出的Meta-RTL。实验结果表明，Meta-RTL在很大程度上超过了强大的基线和以前的任务选择策略，并在极度低资源设置上实现了更大的改进。
URL
https://arxiv.org/abs/2409.19075
