Abstract
Recent research shows that more data and larger models can provide more accurate solutions to natural language problems requiring reasoning. However, models can easily fail to provide solutions in unobserved complex input compositions due to not achieving the level of abstraction required for generalizability. To alleviate this issue, we propose training the language models with neuro-symbolic techniques that can exploit the logical rules of reasoning as constraints and provide additional supervision sources to the model. Training models to adhere to the regulations of reasoning pushes them to make more effective abstractions needed for generalizability and transfer learning. We focus on a challenging problem of spatial reasoning over text. Our results on various benchmarks using multiple language models confirm our hypothesis of effective domain transfer based on neuro-symbolic training.
Abstract (translated)
最近的研究表明，更多的数据和更大的模型能够提供更准确的自然语言问题解决方案，这些解决方案需要推理。然而，由于模型没有达到所需的抽象程度，它们很容易在未观察到的复杂输入组合中无法提供解决方案。为了解决这个问题，我们提出了使用神经符号技术训练语言模型的方法，这些技术可以利用推理的逻辑规则作为约束，并为模型提供额外的监督来源。通过训练模型遵循推理规则，它们被迫做出更多的泛化性抽象，这对于迁移学习和可解释性非常重要。我们关注于文本中的空间推理问题。使用多种语言模型在各种基准上的结果证实了基于神经符号训练的有效领域转移符合我们的假设。
URL
https://arxiv.org/abs/2406.13828
