Abstract
Generative image models are increasingly being used for training data augmentation in vision tasks. In the context of automotive object detection, methods usually focus on producing augmented frames that look as realistic as possible, for example by replacing real objects with generated ones. Others try to maximize the diversity of augmented frames, for example by pasting lots of generated objects onto existing backgrounds. Both perspectives pay little attention to the locations of objects in the scene. Frame layouts are either reused with little or no modification, or they are random and disregard realism entirely. In this work, we argue that optimal data augmentation should also include realistic augmentation of layouts. We introduce a scene-aware probabilistic location model that predicts where new objects can realistically be placed in an existing scene. By then inpainting objects in these locations with a generative model, we obtain much stronger augmentation performance than existing approaches. We set a new state of the art for generative data augmentation on two automotive object detection tasks, achieving up to $2.8\times$ higher gains than the best competing approach ($+1.4$ vs. $+0.5$ mAP boost). We also demonstrate significant improvements for instance segmentation.
Abstract (translated)
生成图像模型在视觉任务的数据增强中应用日益增多。特别是在汽车目标检测领域，通常的方法主要集中在产生尽可能逼真的数据增强帧上，比如用生成的物体替换真实的物体。另一些方法则试图最大化增强帧的多样性，例如在现有背景上粘贴大量生成的物体。然而，这两种方法都很少关注场景中物体的位置。现有的布局要么是简单重复使用而几乎不作修改，要么就是随机放置且完全不顾及现实性。 本文提出了一种观点：最优的数据增强应包括对布局的真实感增强。我们引入了一个场景感知的概率位置模型，该模型可以预测在现有场景中新的物体能被合理放置的位置。通过在这个位置上用生成模型进行图像修复（inpainting），我们得到了比现有方法更强的数据增强效果。 我们在两个汽车目标检测任务上的生成数据增强方面设立了新的技术标准，并取得了高达2.8倍于最佳竞争方法的增益（+1.4 vs. +0.5 mAP提升）。此外，我们也展示了实例分割方面的显著改进。
URL
https://arxiv.org/abs/2504.17076
