Paper Reading AI Learner

Mitigating Reversal Curse via Semantic-aware Permutation Training

2024-03-01 18:55:20
Qingyan Guo, Rui Wang, Junliang Guo, Xu Tan, Jiang Bian, Yujiu Yang

Abstract

While large language models (LLMs) have achieved impressive performance across diverse tasks, recent studies showcase that causal LLMs suffer from the "reversal curse". It is a typical example that the model knows "A's father is B", but is unable to reason "B's child is A". This limitation poses a challenge to the advancement of artificial general intelligence (AGI), as it suggests a gap in the models' ability to comprehend and apply bidirectional reasoning. In this paper, we first conduct substantial evaluation and identify that the root cause of the reversal curse lies in the different word order between the training and inference stage, namely, the poor ability of causal language models to predict antecedent words within the training data. Accordingly, permutation on the training data is considered as a potential solution, since this can make the model predict antecedent words or tokens. However, previous permutation methods may disrupt complete phrases or entities, thereby posing challenges for the model to comprehend and learn from training data. To address this issue, we propose Semantic-aware Permutation Training (SPT), which addresses this issue by segmenting the training sentences into semantic units (i.e., entities or phrases) with an assistant language model and permuting these units before feeding into the model. Extensive experiments demonstrate that SPT effectively mitigates the reversal curse since the performance on reversed questions approximates that on the forward ones, and significantly advances the performance of existing works.

Abstract (translated)

虽然大型语言模型(LLMs)在各种任务上取得了令人印象深刻的性能,但最近的研究表明,因果LLM存在“反演诅咒”。这是一种典型的例子,即模型知道“A的父亲是B”,但却无法进行双向推理,即“B的孩子是A”。这种局限对人工智能通用智能(AGI)的发展构成了挑战,因为它表明了模型理解和应用双向推理的能力存在差距。在本文中,我们首先进行了大量的评估,并发现反演诅咒的根源在于训练和推理阶段之间的单词顺序不同,即因果语言模型在训练数据中预测先决词的能力较差。因此,对训练数据的排列是一种潜在的解决方案,因为这样可以使得模型预测先决词或标记。然而,以前的方法可能会扰乱完整的短语或实体,从而挑战模型理解和从训练数据中学习。为了应对这个问题,我们提出了语义感知排列训练(SPT),通过将训练句子划分为助手语言模型的语义单位(即实体或短语)来进行排列,然后在输入模型之前对这些单位进行随机排列。大量实验证明,SPT有效地减轻了反演诅咒,因为反向问题上的性能与正向问题上的性能相近,并且显著推动了现有工作的性能。

URL

https://arxiv.org/abs/2403.00758

PDF

https://arxiv.org/pdf/2403.00758.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot