Abstract
Chain-of-thought (CoT) prompting can guide language models to engage in complex multi-step reasoning. The quality of provided demonstrations significantly impacts the success of downstream inference tasks. While existing automated methods prioritize accuracy and semantics in these demonstrations, we show that the underlying reasoning patterns play a more crucial role in such tasks. In this paper, we propose Pattern-Aware CoT, a prompting method that considers the diversity of demonstration patterns. By incorporating patterns such as step length and reasoning process within intermediate steps, PA-CoT effectively mitigates the issue of bias induced by demonstrations and enables better generalization to diverse scenarios. We conduct experiments on nine reasoning benchmark tasks using two open-source LLMs. The results show that our method substantially enhances reasoning performance and exhibits robustness to errors. The code will be made publicly available.
Abstract (translated)
提示（CoT）引导语言模型参与复杂的多步骤推理。提供的演示质量对下游推理任务的 success具有重要影响。虽然现有的自动方法在这些演示中优先考虑准确性和语义，但我们表明，在这种任务中，潜在的推理模式起着更关键的作用。在本文中，我们提出了感知模式的CoT，一种关注演示模式多样性的提示方法。通过将模式如步长和推理过程等纳入中间步骤，PA-CoT有效地减轻了演示引起的有偏见问题，并使对不同情景的泛化更好。我们使用两个开源的LLM在九个推理基准任务上进行了实验。结果表明，我们的方法显著增强了推理性能并表现出了对错误的鲁棒性。代码将公开可用。
URL
https://arxiv.org/abs/2404.14812
