Abstract
Quality-Diversity (QD) approaches are a promising direction to develop open-ended processes as they can discover archives of high-quality solutions across diverse niches. While already successful in many applications, QD approaches usually rely on combining only one or two solutions to generate new candidate solutions. As observed in open-ended processes such as technological evolution, wisely combining large diversity of these solutions could lead to more innovative solutions and potentially boost the productivity of QD search. In this work, we propose to exploit the pattern-matching capabilities of generative models to enable such efficient solution combinations. We introduce In-context QD, a framework of techniques that aim to elicit the in-context capabilities of pre-trained Large Language Models (LLMs) to generate interesting solutions using the QD archive as context. Applied to a series of common QD domains, In-context QD displays promising results compared to both QD baselines and similar strategies developed for single-objective optimization. Additionally, this result holds across multiple values of parameter sizes and archive population sizes, as well as across domains with distinct characteristics from BBO functions to policy search. Finally, we perform an extensive ablation that highlights the key prompt design considerations that encourage the generation of promising solutions for QD.
Abstract (translated)
质量多样性（QD）方法是开发开放性过程的有前途的方向，因为它们可以揭示不同领域的优质解决方案。虽然已经在许多应用中取得成功，但QD方法通常仅结合一个或两个解决方案来生成新的候选解决方案。如开放性过程（如技术进步）中所观察到的，明智地结合这些解决方案的大多样性可能会导致更具创新性的解决方案，并可能提高QD搜索的生产力。在这项工作中，我们提出了利用生成模型的模式匹配能力来实现这种有效的解决方案组合。我们引入了In-Context QD，一个旨在通过将预训练的大语言模型（LLMs）在QD档案中生成有趣解决方案的技术框架。将In-Context QD应用于一系列常见的QD领域，与QD基线和为单目标优化 similar strategies 相比，显示出积极的结果。此外，这一结果在参数大小和档案人口大小不同的领域以及具有不同特性的领域中均成立。最后，我们进行了一项广泛的消融，重点突出了鼓励QD生成有前途的解决方案的关键提示设计考虑。
URL
https://arxiv.org/abs/2404.15794
