Abstract
Machine learning models have made incredible progress, but they still struggle when applied to examples from unseen domains. This study focuses on a specific problem of domain generalization, where a model is trained on one source domain and tested on multiple target domains that are unseen during training. We propose IMO: Invariant features Masks for Out-of-Distribution text classification, to achieve OOD generalization by learning invariant features. During training, IMO would learn sparse mask layers to remove irrelevant features for prediction, where the remaining features keep invariant. Additionally, IMO has an attention module at the token level to focus on tokens that are useful for prediction. Our comprehensive experiments show that IMO substantially outperforms strong baselines in terms of various evaluation metrics and settings.
Abstract (translated)
机器学习模型已经取得了巨大的进展，但在应用到未见过的领域时，它们仍然存在困难。本研究关注于领域泛化问题，即在训练过程中，模型学习一个未见过的领域，而在测试过程中，对多个未见过的领域进行测试。我们提出IMO：Invariant features Masks for Out-of-Distribution text classification，通过学习不变的特征来实现OOD泛化。在训练过程中，IMO会学习稀疏的掩码层，用于消除预测过程中的无关特征，而剩余的特征保持不变。此外，IMO在词级别有一个注意力模块，专注于对预测有用的词进行关注。我们的全面实验结果表明，IMO在各种评估指标和设置方面都显著优于强大的基线。
URL
https://arxiv.org/abs/2404.13504
