Abstract
Understanding the importance of the inputs on the output is useful across many tasks. This work provides an information-theoretic framework to analyse the influence of inputs for text classification tasks. Natural language processing (NLP) tasks take either a single element input or multiple element inputs to predict an output variable, where an element is a block of text. Each text element has two components: an associated semantic meaning and a linguistic realization. Multiple-choice reading comprehension (MCRC) and sentiment classification (SC) are selected to showcase the framework. For MCRC, it is found that the context influence on the output compared to the question influence reduces on more challenging datasets. In particular, more challenging contexts allow a greater variation in complexity of questions. Hence, test creators need to carefully consider the choice of the context when designing multiple-choice questions for assessment. For SC, it is found the semantic meaning of the input text dominates (above 80\% for all datasets considered) compared to its linguistic realisation when determining the sentiment. The framework is made available at: this https URL
Abstract (translated)
理解输出输入的重要性在许多任务中都有用。这项工作提供了一个信息论框架，用于分析文本分类任务中输入的影响。自然语言处理（NLP）任务可以接受单个元素输入或多元素输入，以预测输出变量，其中元素是一个文本块。每个文本元素都有两个组成部分：相关的语义意义和语言实现。选择多项选择阅读理解（MCRC）和情感分类（SC）来展示框架。对于MCRC，研究发现，与问题影响相比，语境对输出影响在更具挑战性的数据集上减少。特别是，更具挑战性的上下文允许更广泛的问题复杂性的变化。因此，测试创建者需要在设计多项选择题时仔细考虑语境的选择。对于SC，研究发现输入文本的语义意义占主导地位（在所有考虑的数据集上均超过80%）。当确定情感时，其语言实现的影响力相对较小。框架可在以下链接中访问：https://this URL
URL
https://arxiv.org/abs/2402.00978
