Abstract
The emergence of agentic reinforcement learning (Agentic RL) marks a paradigm shift from conventional reinforcement learning applied to large language models (LLM RL), reframing LLMs from passive sequence generators into autonomous, decision-making agents embedded in complex, dynamic worlds. This survey formalizes this conceptual shift by contrasting the degenerate single-step Markov Decision Processes (MDPs) of LLM-RL with the temporally extended, partially observable Markov decision processes (POMDPs) that define Agentic RL. Building on this foundation, we propose a comprehensive twofold taxonomy: one organized around core agentic capabilities, including planning, tool use, memory, reasoning, self-improvement, and perception, and the other around their applications across diverse task domains. Central to our thesis is that reinforcement learning serves as the critical mechanism for transforming these capabilities from static, heuristic modules into adaptive, robust agentic behavior. To support and accelerate future research, we consolidate the landscape of open-source environments, benchmarks, and frameworks into a practical compendium. By synthesizing over five hundred recent works, this survey charts the contours of this rapidly evolving field and highlights the opportunities and challenges that will shape the development of scalable, general-purpose AI agents.
Abstract (translated)
代理强化学习（Agentic RL）的出现标志着从传统的大规模语言模型（LLM RL）中使用的常规强化学习向全新范式的转变，将LLM从被动的序列生成器转变为嵌入复杂动态环境中的自主决策智能体。本次综述通过对比LLM-RL中退化的单步马尔可夫决策过程（MDPs）与定义Agentic RL的扩展时间、部分可观测马尔可夫决策过程（POMDPs），正式确立了这一概念性转变。在此基础上，我们提出了一种全面的双层分类法：一个围绕核心代理能力组织，包括规划、工具使用、记忆、推理、自我改进和感知；另一个则围绕这些能力在各种任务领域中的应用进行组织。我们的主要论点是强化学习作为将这些静态、启发式模块转换为适应性强且稳健的代理行为的关键机制。 为了支持并加速未来的研究，我们将开源环境、基准测试和框架整合成一份实用的概览。通过综合分析超过五百篇近期的相关文献，本次综述勾勒出了这一迅速发展的领域的发展轮廓，并突显了将塑造可扩展通用人工智能代理开发的机会与挑战。
URL
https://arxiv.org/abs/2509.02547
