Abstract
The fixed-size context of Transformer makes GPT models incapable of generating arbitrarily long text. In this paper, we introduce RecurrentGPT, a language-based simulacrum of the recurrence mechanism in RNNs. RecurrentGPT is built upon a large language model (LLM) such as ChatGPT and uses natural language to simulate the Long Short-Term Memory mechanism in an LSTM. At each timestep, RecurrentGPT generates a paragraph of text and updates its language-based long-short term memory stored on the hard drive and the prompt, respectively. This recurrence mechanism enables RecurrentGPT to generate texts of arbitrary length without forgetting. Since human users can easily observe and edit the natural language memories, RecurrentGPT is interpretable and enables interactive generation of long text. RecurrentGPT is an initial step towards next-generation computer-assisted writing systems beyond local editing suggestions. In addition to producing AI-generated content (AIGC), we also demonstrate the possibility of using RecurrentGPT as an interactive fiction that directly interacts with consumers. We call this usage of generative models by ``AI As Contents'' (AIAC), which we believe is the next form of conventional AIGC. We further demonstrate the possibility of using RecurrentGPT to create personalized interactive fiction that directly interacts with readers instead of interacting with writers. More broadly, RecurrentGPT demonstrates the utility of borrowing ideas from popular model designs in cognitive science and deep learning for prompting LLMs. Our code is available at this https URL and an online demo is available at this https URL.
Abstract (translated)
Transformer的固定上下文使得GPT模型无法生成任意长文本。在本文中，我们介绍了RecurrentGPT，一个基于语言模型的RNN的再循环机制的模拟对象。RecurrentGPT基于像ChatGPT这样的大型语言模型构建，并使用自然语言模拟来模拟LSTM中的长短期记忆机制。在每个时间步骤中，RecurrentGPT生成一篇段落，并更新其语言模型上的长期-短期记忆存储在硬盘和提示中。这种再循环机制使得RecurrentGPT能够生成任意长度的文本而不会忘记。由于人类用户可以轻松观察和编辑自然语言 memories,RecurrentGPT可以被解释，并实现交互式的生成长文本。RecurrentGPT是超越本地编辑建议的下一代计算机辅助写作系统的初步步骤。除了生成AI生成的内容(AIGC)，我们还展示了使用RecurrentGPT作为直接与消费者互动的交互 fiction的可能性。我们称之为“AI作为内容”(AIAC)，我们认为它是传统AIGC的下一个形式。我们还展示了使用RecurrentGPT创建个性化交互 fiction的可能性，该交互 fiction直接与读者互动而不是与作者互动。更广泛地说，RecurrentGPT展示了从认知科学和深度学习中受欢迎的模型设计中提取灵感的实用性。我们的代码可在该httpsURL上获取，一个在线演示可在该httpsURL上获取。
URL
https://arxiv.org/abs/2305.13304
