Abstract
Semantic segmentation requires a holistic understanding of the physical world, as it assigns semantic labels to spatially continuous and structurally coherent objects rather than to isolated pixels. However, existing data-free knowledge distillation (DFKD) methods-primarily designed for classification-often disregard this continuity, resulting in significant performance degradation when applied directly to segmentation tasks. In this paper, we introduce DFSS, a novel data-free distillation framework tailored for semantic segmentation. Unlike prior approaches that treat pixels independently, DFSS respects the structural and contextual continuity of real-world scenes. Our key insight is to leverage Batch Normalization (BN) statistics from a teacher model to guide Approximate Distribution Sampling (ADS), enabling the selection of data that better reflects the original training distribution-without relying on potentially misleading teacher predictions. Additionally, we propose Weighted Distribution Progressive Distillation (WDPD), which dynamically prioritizes reliable samples that are more closely aligned with the original data distribution early in training and gradually incorporates more challenging cases, mirroring the natural progression of learning in human perception. Extensive experiments on standard benchmarks demonstrate that DFSS consistently outperforms existing data-free distillation methods for semantic segmentation, achieving state-of-the-art results with significantly reduced reliance on auxiliary data.
Abstract (translated)
语义分割需要对物理世界有一个整体的理解，因为它为连续且结构连贯的对象分配语义标签，而不仅仅是孤立的像素。然而，现有的无数据知识蒸馏（DFKD）方法主要针对分类设计，常常忽视这种连续性，在直接应用于分割任务时会导致性能显著下降。在本文中，我们介绍了 DFSS，一种专为语义分割定制的新颖无数据蒸馏框架。与之前将像素独立处理的方法不同，DFSS 尊重现实场景中的结构和上下文连贯性。我们的核心见解是利用教师模型的批量归一化（BN）统计信息来指导近似分布采样（ADS），从而选择更好地反映原始训练分布的数据——而不依赖可能具有误导性的教师预测。 此外，我们提出了加权分布渐进式蒸馏（WDPD），该方法在训练初期优先考虑与原始数据分布更加一致的可靠样本，并逐渐纳入更具挑战性的情况，这反映了人类感知学习过程中的自然进展。在标准基准测试上的广泛实验表明，DFSS 一贯优于现有的无数据知识蒸馏方法，在语义分割中实现了最先进的结果，并且显著减少了对辅助数据的依赖。
URL
https://arxiv.org/abs/2512.13175
