Abstract
Prompt engineering is crucial for leveraging large language models (LLMs), but existing methods often rely on a single optimization trajectory, limiting adaptability and efficiency while suffering from narrow perspectives, gradient conflicts, and high computational cost. We propose MAPGD (Multi-Agent Prompt Gradient Descent), a framework integrating multi-agent collaboration with gradient-based optimization. MAPGD features specialized agents for task clarity, example selection, format design, and stylistic refinement; semantic gradient coordination to resolve conflicts; bandit-based candidate selection for efficient exploration-exploitation; and theoretical convergence guarantees. Experiments on classification, generation, and reasoning tasks show MAPGD outperforms single-agent and random baselines in accuracy and efficiency. Ablations confirm the benefits of gradient fusion, agent specialization, and conflict resolution, providing a unified, gradient-inspired multi-agent approach to robust and interpretable prompt optimization.
Abstract (translated)
提示工程对于充分利用大型语言模型（LLM）至关重要，但现有方法往往依赖单一的优化路径，这限制了其适应性和效率，并且容易出现视野狭窄、梯度冲突和计算成本过高的问题。我们提出了MAPGD（多代理提示梯度下降），这是一种结合了多代理协作与基于梯度优化的框架。MAPGD的特点包括：专门负责任务清晰性、示例选择、格式设计和风格精炼的代理；用于解决冲突的语义梯度协调；基于强盗算法的选择候选方案以实现高效的探索-利用平衡；以及理论上的收敛保证。在分类、生成和推理任务上的实验表明，MAPGD比单一代理和随机基线方法在准确性和效率方面表现更优。消融分析证实了梯度融合、代理专门化及冲突解决带来的益处，为稳健且可解释的提示优化提供了一种统一的、以梯度为基础的多代理方法。
URL
https://arxiv.org/abs/2509.11361
