Abstract
Machine Reading Comprehension (MRC) poses a significant challenge in the field of Natural Language Processing (NLP). While mainstream MRC methods predominantly leverage extractive strategies using encoder-only models such as BERT, generative approaches face the issue of out-of-control generation -- a critical problem where answers generated are often incorrect, irrelevant, or unfaithful to the source text. To address these limitations in generative models for MRC, we introduce the Question-Attended Span Extraction (QASE) module. Integrated during the fine-tuning phase of pre-trained generative language models (PLMs), QASE significantly enhances their performance, allowing them to surpass the extractive capabilities of advanced Large Language Models (LLMs) such as GPT-4. Notably, these gains in performance do not come with an increase in computational demands. The efficacy of the QASE module has been rigorously tested across various datasets, consistently achieving or even surpassing state-of-the-art (SOTA) results.
Abstract (translated)
机器阅读理解（MRC）在自然语言处理（NLP）领域是一个具有挑战性的问题。虽然主流的MRC方法主要依赖于仅使用编码器模型的提取策略，如BERT，但生成方法面临着生成过控制的问题——这是一个关键问题，因为生成的答案往往是不准确的、无关的或与原文不符。为了解决这些限制，我们引入了问题关注区间提取（QASE）模块。在预训练生成语言模型（PLMs）的微调阶段集成QASE模块，显著增强了它们的性能，使它们能够超越高级大型语言模型（LLMs） such as GPT-4的提取能力。值得注意的是，这些性能提升并没有增加计算负担。QASE模块的有效性已在各种数据集上进行了严格的测试，并始终达到或甚至超过了最先进的水平（SOTA）。
URL
https://arxiv.org/abs/2404.17991
