Abstract
As prompts play an increasingly critical role in large language models (LLMs), optimizing textual prompts has become a crucial challenge. The Textual Gradient Descent (TGD) framework has emerged as a promising data-driven approach that iteratively refines textual prompts using LLM - suggested updates (or textual gradients) over minibatches of training samples. In this paper, we empirically demonstrate that scaling the number of training examples initially improves but later degrades TGD's performance across multiple downstream NLP tasks. However, while data scaling improves results for most tasks, it also significantly increases the computational cost when leveraging LLMs. To address this, we draw inspiration from numerical gradient descent and propose Textual Stochastic Gradient Descent with Momentum (TSGD-M) - a method that facilitates scalable in-context learning by reweighting prompt sampling based on past batch distributions. Across nine NLP tasks spanning three domains - including BIG-Bench Hard (BBH), natural language understanding tasks, and reasoning tasks - TSGD-M significantly outperforms TGD baselines that do not incorporate reweighted sampling, while also reducing variance in most tasks.
Abstract (translated)
随着提示在大型语言模型（LLM）中扮演越来越重要的角色，优化文本提示已成为一个关键挑战。文本梯度下降（TGD）框架作为一种有前景的数据驱动方法应运而生，该方法通过使用LLM提供的更新建议（或称为文本梯度），在小批量训练样本上迭代地精炼文本提示。在这篇论文中，我们实证证明了增加初始训练示例的数量可以改善但最终会降低TGD框架在多个下游NLP任务上的性能表现。然而，在大多数情况下，数据量的扩展虽然提高了结果质量，但也显著增加了利用LLM进行计算的成本。为了解决这个问题，我们借鉴数值梯度下降的方法，并提出了带有动量的文本随机梯度下降（TSGD-M）——这种方法通过根据过去的批次分布重新加权提示采样来促进可扩展的上下文学习。在涵盖三大领域的九个NLP任务上（包括BIG-Bench Hard (BBH)、自然语言理解任务和推理任务），TSGD-M显著优于不使用重加权采样的TGD基线方法，并且在大多数任务中也降低了方差。 此段落概述了改进文本提示优化技术的研究成果，提出了新的TSGD-M方法来提高大型语言模型的性能，同时降低计算成本。
URL
https://arxiv.org/abs/2506.00400
