Abstract
The volume and diversity of digital information have led to a growing reliance on Machine Learning techniques, such as Natural Language Processing, for interpreting and accessing appropriate data. While vector and graph embeddings represent data for similarity tasks, current state-of-the-art pipelines lack guaranteed explainability, failing to determine similarity for given full texts accurately. These considerations can also be applied to classifiers exploiting generative language models with logical prompts, which fail to correctly distinguish between logical implication, indifference, and inconsistency, despite being explicitly trained to recognise the first two classes. We present a novel pipeline designed for hybrid explainability to address this. Our methodology combines graphs and logic to produce First-Order Logic representations, creating machine- and human-readable representations through Montague Grammar. Preliminary results indicate the effectiveness of this approach in accurately capturing full text similarity. To the best of our knowledge, this is the first approach to differentiate between implication, inconsistency, and indifference for text classification tasks. To address the limitations of existing approaches, we use three self-contained datasets annotated for the former classification task to determine the suitability of these approaches in capturing sentence structure equivalence, logical connectives, and spatiotemporal reasoning. We also use these data to compare the proposed method with language models pre-trained for detecting sentence entailment. The results show that the proposed method outperforms state-of-the-art models, indicating that natural language understanding cannot be easily generalised by training over extensive document corpora. This work offers a step toward more transparent and reliable Information Retrieval from extensive textual data.
Abstract (translated)
数字信息的量和多样性导致了对机器学习技术（如自然语言处理）在解释和访问适当数据方面的日益依赖。虽然向量和图嵌入可以表示用于相似性任务的数据，但当前最先进的管道缺乏可保证的透明度，在确定给定全文的相似性方面不够准确。这些考虑同样适用于利用逻辑提示的生成式语言模型分类器，尽管它们被明确训练来识别前两类，但在区分逻辑蕴含、无关性和不一致性时仍会失败。 我们提出了一种新型管道，旨在通过混合可解释性解决上述问题。我们的方法结合了图和逻辑来产生一阶逻辑表示，并通过Montague语法生成机器和人类都易于理解的表示形式。初步结果显示，这种方法在准确捕捉全文相似性方面是有效的。据我们所知，这是第一个能够在文本分类任务中区分蕴含、不一致性和无关性的方法。 为了解决现有方法的局限性，我们使用了三个自包含的数据集，这些数据集被注释用于上述分类任务，以确定这些方法在捕捉句子结构等价性、逻辑连接词和时空推理方面的能力。同时，我们也利用这些数据来将提出的模型与预训练的语言模型进行对比，后者旨在检测句子蕴含。 实验结果表明，我们提出的方法优于最先进的模型，这表明自然语言理解不能通过大规模文档语料库的培训轻易实现泛化。这项工作朝着从大量文本数据中获取更透明和可靠的信息检索迈出了重要一步。
URL
https://arxiv.org/abs/2507.05017
