Abstract
In planning, stitching is an ability of algorithms to piece together sub-trajectories of data they are trained on to generate new and diverse behaviours. While stitching is historically a strength of offline reinforcement learning, recent generative behavioural cloning (BC) methods have also shown proficiency at stitching. However, the main factors behind this are poorly understood, hindering the development of new algorithms that can reliably stitch. Focusing on diffusion planners trained via BC, we find two properties are needed to compose: \emph{positional equivariance} and \emph{local receptiveness}. We use these two properties to explain architecture, data, and inference choices in existing generative BC methods based on diffusion planning, including replanning frequency, data augmentation, and data scaling. Experimental comparisions show that (1) while locality is more important than positional equivariance in creating a diffusion planner capable of composition, both are crucial (2) enabling these properties through relatively simple architecture choices can be competitive with more computationally expensive methods such as replanning or scaling data, and (3) simple inpainting-based guidance can guide architecturally compositional models to enable generalization in goal-conditioned settings.
Abstract (translated)
在规划过程中，缝合（stitching）是算法将它们训练时所使用的数据子轨迹拼接起来以生成新的、多样的行为的能力。虽然历史上离线强化学习在缝合方面表现出色，但最近的生成式行为克隆（Behavior Cloning, BC）方法也展示了其在这方面的能力。然而，导致这种能力的主要因素尚不明确，这阻碍了开发能够可靠地进行缝合的新算法的发展。 通过研究基于行为克隆训练的扩散规划器，我们发现两种属性是必要的：\emph{位置等变性}（positional equivariance）和\emph{局部响应性}（local receptiveness）。这两种性质被用来解释现有的基于扩散规划的生成式BC方法中的架构、数据集及推理选择，包括重新计划频率、数据增强以及数据缩放。实验比较表明： 1. 尽管在创建能够进行组合的扩散规划器时，局部响应性比位置等变性更为重要，但两者都至关重要。 2. 通过简单的架构选择来实现这些性质可以与更耗费计算资源的方法（如重新计划或扩大数据集）相竞争。 3. 简单的修补指导法可以帮助具有可组合架构的模型在目标条件设定下进行泛化。 总的来说，这项研究揭示了基于扩散规划器的生成式行为克隆方法中的关键性质，并为如何通过简单的架构设计来实现这些特性以增强算法的性能提供了见解。
URL
https://arxiv.org/abs/2505.18083
