Abstract
The robustness of Transformer-based Natural Language Inference encoders is frequently compromised as they tend to rely more on dataset biases than on the intended task-relevant features. Recent studies have attempted to mitigate this by reducing the weight of biased samples during the training process. However, these debiasing methods primarily focus on identifying which samples are biased without explicitly determining the biased components within each case. This limitation restricts those methods' capability in out-of-distribution inference. To address this issue, we aim to train models to adopt the logic humans use in explaining causality. We propose a simple, comprehensive, and interpretable method: Explanation based Bias Decoupling Regularization (EBD-Reg). EBD-Reg employs human explanations as criteria, guiding the encoder to establish a tripartite parallel supervision of Distinguishing, Decoupling and Aligning. This method enables encoders to identify and focus on keywords that represent the task-relevant features during inference, while discarding the residual elements acting as biases. Empirical evidence underscores that EBD-Reg effectively guides various Transformer-based encoders to decouple biases through a human-centric lens, significantly surpassing other methods in terms of out-of-distribution inference capabilities.
Abstract (translated)
基于Transformer的自然语言推理模型的稳健性经常受到数据集偏差的影响，而不是依赖于预期的任务相关特征。为了解决这个问题，最近的研究尝试通过在训练过程中减少偏差样本的权重来减轻这种偏差。然而，这些去偏方法主要关注于确定哪些样本存在偏差，而没有明确确定每个案例中的偏差组件。这个限制限制了这些方法在离散分布推理方面的能力。为了应对这个问题，我们旨在训练模型以模仿人类解释因果关系的逻辑。我们提出了一种简单、全面、可解释的方法：解释偏差去耦正则化（EBD-Reg）。EBD-Reg利用人类解释作为标准，指导编码器建立区分、去耦和align的三角关系。这种方法使编码器能够在推理过程中识别和关注代表任务相关特征的关键字，同时丢弃作为偏见的残余元素。实证证据表明，EBD-Reg有效地通过人本主义视角引导各种Transformer编码器通过去偏来解耦偏差，在离散分布推理能力上显著超越其他方法。
URL
https://arxiv.org/abs/2404.13390
