Abstract
Pedestrian detection models in autonomous driving systems often lack robustness due to insufficient representation of dangerous pedestrian scenarios in training datasets. To address this limitation, we present a novel framework for controllable pedestrian video editing in multi-view driving scenarios by integrating video inpainting and human motion control techniques. Our approach begins by identifying pedestrian regions of interest across multiple camera views, expanding detection bounding boxes with a fixed ratio, and resizing and stitching these regions into a unified canvas while preserving cross-view spatial relationships. A binary mask is then applied to designate the editable area, within which pedestrian editing is guided by pose sequence control conditions. This enables flexible editing functionalities, including pedestrian insertion, replacement, and removal. Extensive experiments demonstrate that our framework achieves high-quality pedestrian editing with strong visual realism, spatiotemporal coherence, and cross-view consistency. These results establish the proposed method as a robust and versatile solution for multi-view pedestrian video generation, with broad potential for applications in data augmentation and scenario simulation in autonomous driving.
Abstract (translated)
在自主驾驶系统中，行人检测模型由于训练数据集中危险行人性状表示不足而往往缺乏鲁棒性。为了解决这一局限性，我们提出了一种结合视频修复和人体运动控制技术的新型多视角驾驶场景可控行人视频编辑框架。我们的方法首先通过多个摄像头视图来识别行人的感兴趣区域，并以固定的比率扩展检测边界框。然后将这些区域调整大小并拼接到一个统一的画布上，同时保留跨视图的空间关系。接着应用二值掩码来指定可编辑的区域，在此区域内行人编辑则由姿态序列控制条件指导。这使我们能够实现包括行人的插入、替换和移除在内的灵活编辑功能。广泛的实验表明，我们的框架在生成具有高质量行人编辑效果的同时，还具备强烈的视觉真实感、时空一致性和跨视图一致性。这些结果证明了所提出的方法作为多视角行人视频生成的稳健且多功能解决方案的有效性，并展示了它在数据增强和场景模拟方面的广泛应用潜力，在自主驾驶领域尤为突出。
URL
https://arxiv.org/abs/2508.00299
