Abstract
Human creativity follows a perceptual process, moving from abstract ideas to finer details during creation. While 3D generative models have advanced dramatically, models specifically designed to assist human imagination in 3D creation -- particularly for detailing abstractions from coarse to fine -- have not been explored. We propose a framework that enables intuitive and interactive 3D shape generation by iteratively splitting bounding boxes to refine the set of bounding boxes. The main technical components of our framework are two generative models: the box-splitting generative model and the box-to-shape generative model. The first model, named BoxSplitGen, generates a collection of 3D part bounding boxes with varying granularity by iteratively splitting coarse bounding boxes. It utilizes part bounding boxes created through agglomerative merging and learns the reverse of the merging process -- the splitting sequences. The model consists of two main components: the first learns the categorical distribution of the box to be split, and the second learns the distribution of the two new boxes, given the set of boxes and the indication of which box to split. The second model, the box-to-shape generative model, is trained by leveraging the 3D shape priors learned by an existing 3D diffusion model while adapting the model to incorporate bounding box conditioning. In our experiments, we demonstrate that the box-splitting generative model outperforms token prediction models and the inpainting approach with an unconditional diffusion model. Also, we show that our box-to-shape model, based on a state-of-the-art 3D diffusion model, provides superior results compared to a previous model.
Abstract (translated)
人类的创造力遵循一种感知过程，从抽象概念逐步细化到具体细节。虽然三维生成模型取得了显著进步，但专门用于协助人类在三维创作中将粗糙的概念转化为精细细节的设计尚未得到充分探索。我们提出了一种框架，通过迭代地划分边界框来生成直观且交互式的三维形状。该框架的主要技术组件是两个生成式模型：一个是边界框分割生成模型（BoxSplitGen），另一个是从边界框到形状的生成模型。 第一个模型，名为BoxSplitGen，通过迭代地分割粗糙的边界框产生一系列不同粒度的3D部件边界框集合。它利用聚合合并产生的部分边界框，并学习与其相对的过程——即分割序列。该模型包含两个主要组成部分：第一部分负责从一组给定的边界框中预测要被分裂的具体边界框，而第二部分则基于指示哪些边界框需要被分割的信息来生成新的子边界框。 第二个模型是从现有的三维扩散模型所学得的形状先验出发训练而成的，同时该模型也适应性地融入了边界框条件。通过这种方式，它可以更有效地将边界框转换为详细的三维形状。 在我们的实验中，我们展示了边界框分割生成模型优于基于令牌预测和无条件扩散模型的方法。此外，我们也证明了基于先进的3D扩散模型构建的边界框到形状模型相比之前的方法提供了更好的结果。
URL
https://arxiv.org/abs/2602.20666
