Abstract
Compositional generalization is the ability of a model to generalize to complex, previously unseen types of combinations of entities from just having seen the primitives. This type of generalization is particularly relevant to the semantic parsing community for applications such as task-oriented dialogue, text-to-SQL parsing, and information retrieval, as they can harbor infinite complexity. Despite the success of large language models (LLMs) in a wide range of NLP tasks, unlocking perfect compositional generalization still remains one of the few last unsolved frontiers. The past few years has seen a surge of interest in works that explore the limitations of, methods to improve, and evaluation metrics for compositional generalization capabilities of LLMs for semantic parsing tasks. In this work, we present a literature survey geared at synthesizing recent advances in analysis, methods, and evaluation schemes to offer a starting point for both practitioners and researchers in this area.
Abstract (translated)
组合泛化是指模型能够从仅看到基本元素来泛化到复杂、之前未见过的实体组合。对于自然语言处理社区中诸如面向任务对话、文本到关系数据库解析和信息检索等应用，这种泛化尤为重要，因为它们可能包含无限复杂性。尽管在自然语言处理领域，大型语言模型（LLMs）在各种NLP任务上取得了成功，但实现完美组合泛化仍然是不可能的几个最后的未解决的前沿。在过去的几年里，对研究探索LLM在语义解析任务上组合泛化能力的局限性、方法和评估指标的兴趣浓厚。在这篇论文中，我们进行了一篇文献调查，旨在为该领域的实践者和研究人员提供一些关于LLM组合泛化分析、方法和评估方案的起点。
URL
https://arxiv.org/abs/2404.13074
