Abstract
Image editing approaches have become more powerful and flexible with the advent of powerful text-conditioned generative models. However, placing objects in an environment with a precise location and orientation still remains a challenge, as this typically requires carefully crafted inpainting masks or prompts. In this work, we show that a carefully designed visual map, combined with coarse object masks, is sufficient for high quality object placement. We design a conditioning signal that resolves ambiguities, while being flexible enough to allow for changing of shapes or object orientations. By building on an inpainting model, we leave the background intact by design, in contrast to methods that model objects and background jointly. We demonstrate the effectiveness of our method in the automotive setting, where we compare different conditioning signals in novel object placement tasks. These tasks are designed to measure edit quality not only in terms of appearance, but also in terms of pose and location accuracy, including cases that require non-trivial shape changes. Lastly, we show that fine location control can be combined with appearance control to place existing objects in precise locations in a scene.
Abstract (translated)
随着强大的文本条件生成模型的出现，图像编辑方法已经变得更加强大和灵活。然而，在精确的位置和方向上将物体放置在环境中仍然是一项挑战，这通常需要精心设计的修补掩模或提示词。在这项工作中，我们展示了通过结合精细设计的视觉地图与粗略的对象遮罩，就足以实现高质量的物体放置。我们设计了一种条件信号，这种信号能够解决歧义问题，并且足够灵活以允许形状变化或对象方向的变化。通过建立在修补模型的基础上，我们的方法有意地保留背景不变，这与其他同时建模物体和背景的方法形成了对比。我们在汽车设置中演示了我们方法的有效性，在该设置中，我们将不同条件信号应用于新颖的物体放置任务中进行比较。这些任务旨在从外观、姿态以及位置精度等方面评估编辑质量，包括需要进行非平凡形状变化的情况。最后，我们展示了精细的位置控制可以与外观控制相结合，以将现有的对象精确地放置在一个场景中的特定位置。
URL
https://arxiv.org/abs/2506.21446
