Abstract
Semantic segmentation networks require large amounts of pixel-level annotated data, which are costly to obtain for real-world images. Computer graphics engines can generate synthetic images alongside their ground-truth annotations. However, models trained on such images can perform poorly on real images due to the domain gap between real and synthetic images. Style transfer methods can reduce this difference by applying a realistic style to synthetic images. Choosing effective data transformations and their sequence is difficult due to the large combinatorial search space of style transfer operators. Using multi-objective genetic algorithms, we optimize pipelines to balance structural coherence and style similarity to target domains. We study the use of paired-image metrics on individual image samples during evolution to enable rapid pipeline evaluation, as opposed to standard distributional metrics that require the generation of many images. After optimization, we evaluate the resulting Pareto front using distributional metrics and segmentation performance. We apply this approach to standard datasets in synthetic-to-real domain adaptation: from the video game GTA5 to real image datasets Cityscapes and ACDC, focusing on adverse conditions. Results demonstrate that evolutionary algorithms can propose diverse augmentation pipelines adapted to different objectives. The contribution of this work is the formulation of style transfer as a sequencing problem suitable for evolutionary optimization and the study of efficient metrics that enable feasible search in this space. The source code is available at: this https URL.
Abstract (translated)
语义分割网络需要大量的像素级标注数据，这些数据在获取真实世界的图像时成本高昂。计算机图形引擎可以生成合成图像及其地面实况注释。然而，在这类图像上进行训练的模型可能在真实图像上的表现不佳，因为真实图像和合成图像之间存在领域差距。通过将现实风格应用于合成图像，风格迁移方法可以减少这种差异。但由于风格转换操作的组合搜索空间很大，选择有效的数据变换及其顺序是很困难的。 我们使用多目标遗传算法来优化管道，以便在结构一致性和与目标领域的样式相似性之间取得平衡。我们在进化过程中研究了对单个图像样本使用配对图像度量的方法，以实现快速的流水线评估，而不是需要生成许多图像的标准分布度量。优化后，我们使用分布度量和分割性能来评估所得的帕累托前沿。 我们将这种方法应用于从视频游戏GTA5到真实图像数据集Cityscapes和ACDC（假设是某个特定数据集）的合成到现实领域适应的标准数据集中，重点在于不利条件。结果表明，进化算法可以提出适合不同目标的各种增强管道。 这项工作的贡献在于将风格迁移表述为一个适用于进化优化的序列问题，并研究了在这一空间中进行可行搜索的有效度量。源代码可在以下网址获得：this https URL.
URL
https://arxiv.org/abs/2602.03625
