Abstract
Generative Models are a valuable tool for the controlled creation of high-quality image data. Controlled diffusion models like the ControlNet have allowed the creation of labeled distributions. Such synthetic datasets can augment the original training distribution when discriminative models, like semantic segmentation, are trained. However, this augmentation effect is limited since ControlNets tend to reproduce the original training distribution. This work introduces a method to utilize data from unlabeled domains to train ControlNets by introducing the concept of uncertainty into the control mechanism. The uncertainty indicates that a given image was not part of the training distribution of a downstream task, e.g., segmentation. Thus, two types of control are engaged in the final network: an uncertainty control from an unlabeled dataset and a semantic control from the labeled dataset. The resulting ControlNet allows us to create annotated data with high uncertainty from the target domain, i.e., synthetic data from the unlabeled distribution with labels. In our scenario, we consider retinal OCTs, where typically high-quality Spectralis images are available with given ground truth segmentations, enabling the training of segmentation networks. The recent development in Home-OCT devices, however, yields retinal OCTs with lower quality and a large domain shift, such that out-of-the-pocket segmentation networks cannot be applied for this type of data. Synthesizing annotated images from the Home-OCT domain using the proposed approach closes this gap and leads to significantly improved segmentation results without adding any further supervision. The advantage of uncertainty-guidance becomes obvious when compared to style transfer: it enables arbitrary domain shifts without any strict learning of an image style. This is also demonstrated in a traffic scene experiment.
Abstract (translated)
生成模型是用于控制高质量图像数据创建的宝贵工具。受控扩散模型，如ControlNet，允许创建标记分布。这样的合成数据集可以在训练辨别性模型（例如语义分割）时扩充原始训练分布。然而，这种增强效果有限，因为ControlNets倾向于复制原训练数据分布。这项工作引入了一种方法，利用未标注领域的数据来训练ControlNets，并通过将不确定性概念引入控制机制中实现这一目标。不确定性表示给定的图像并非下游任务（如分割）原始训练数据集的一部分。因此，在最终网络中有两种类型的控制：一种是从未标记数据集中来的不确定性控制和一种是从标记数据集来的语义控制。由此产生的ControlNet使我们能够从目标领域创建具有高不确定性的注释数据，即带有标签的来自未标注分布的合成图像。 在我们的场景中考虑视网膜OCT（光学相干断层扫描），通常有高质量的Spectralis图像及其给定的真实分割结果，这可以用来训练分割网络。然而，最近家用OCT设备的发展带来了较低质量和域偏移较大的视网膜OCT数据，使得现成的分割网络无法应用于此类数据。利用所提出的方法从Home-OCT领域生成注释图像填补了这一空白，并在不添加额外监督的情况下显著提高了分割结果。 与风格转移相比，不确定性指导的优势变得明显：它可以实现任意域偏移而不严格学习图像样式。这一点也在交通场景实验中得到了验证。
URL
https://arxiv.org/abs/2510.11346
