Abstract
Large language models (LLMs) have significantly advanced in various fields and intelligent agent applications. However, current LLMs that learn from human or external model supervision are costly and may face performance ceilings as task complexity and diversity increase. To address this issue, self-evolution approaches that enable LLM to autonomously acquire, refine, and learn from experiences generated by the model itself are rapidly growing. This new training paradigm inspired by the human experiential learning process offers the potential to scale LLMs towards superintelligence. In this work, we present a comprehensive survey of self-evolution approaches in LLMs. We first propose a conceptual framework for self-evolution and outline the evolving process as iterative cycles composed of four phases: experience acquisition, experience refinement, updating, and evaluation. Second, we categorize the evolution objectives of LLMs and LLM-based agents; then, we summarize the literature and provide taxonomy and insights for each module. Lastly, we pinpoint existing challenges and propose future directions to improve self-evolution frameworks, equipping researchers with critical insights to fast-track the development of self-evolving LLMs.
Abstract (translated)
大语言模型（LLMs）在各种领域和智能机器人应用方面取得了显著的进步。然而，当前从人类或外部模型监督中学习的LLM成本较高，且随着任务复杂性和多样性的增加，可能面临性能上限。为解决这个问题，自进化方法使LLM能够自主获取、精炼并从模型自身生成的经验中学习，正在快速发展。这一新训练范式在很大程度上受到了人类经验学习的启发，为LLM达到超级智能提供了潜力。在这项工作中，我们全面调查了LLM的自进化方法。我们首先提出了一个自进化的概念框架，并概述了自进化的演变过程由四个阶段组成：经验获取、经验精炼、更新和评估。接下来，我们分类了LLM和基于LLM的智能代理的演化目标；然后，我们总结了文献，并为每个模块提供了分类和见解。最后，我们指出了现有挑战，并为改善自进化框架提出了未来方向，以便研究人员能够关键性地快速推进LLM的自进化发展。
URL
https://arxiv.org/abs/2404.14387
