Abstract
Dataset condensation (DC) enables the creation of compact, privacy-preserving synthetic datasets that can match the utility of real patient records, supporting democratised access to highly regulated clinical data for developing downstream clinical models. State-of-the-art DC methods supervise synthetic data by aligning the training dynamics of models trained on real and those trained on synthetic data, typically using full stochastic gradient descent (SGD) trajectories as alignment targets; however, these trajectories are often noisy, high-curvature, and storage-intensive, leading to unstable gradients, slow convergence, and substantial memory overhead. We address these limitations by replacing full SGD trajectories with smooth, low-loss parametric surrogates, specifically quadratic Bézier curves that connect the initial and final model states from real training trajectories. These mode-connected paths provide noise-free, low-curvature supervision signals that stabilise gradients, accelerate convergence, and eliminate the need for dense trajectory storage. We theoretically justify Bézier-mode connections as effective surrogates for SGD paths and empirically show that the proposed method outperforms state-of-the-art condensation approaches across five clinical datasets, yielding condensed datasets that enable clinically effective model development.
Abstract (translated)
数据集浓缩（DC）能够创建紧凑且保护隐私的合成数据集，这些数据集可以与真实患者记录具有相同的效用，支持对高度监管的临床数据进行民主化访问，以开发下游临床模型。最先进的DC方法通过将基于真实数据和合成数据训练的模型的训练动态相匹配来监督生成的合成数据，通常使用完整的随机梯度下降（SGD）轨迹作为对齐目标；然而，这些轨迹往往噪声大、曲率高且占用大量存储空间，导致不稳定的梯度、缓慢收敛以及显著的记忆开销。我们通过用平滑且低损失的参数替代品替换完整的SGD轨迹来解决这些问题，具体来说是二次贝塞尔曲线，它连接基于真实训练轨迹初始和最终模型状态。这些模型相连路径提供无噪声、曲率低的监督信号，可以稳定梯度、加速收敛并消除对密集轨迹存储的需求。 我们从理论上证明了贝塞尔-模式连接作为SGD路径的有效替代品，并通过实验证明所提出的方法在五个临床数据集上超越了最先进的浓缩方法，在生成压缩的数据集方面表现出色，这些数据集能够支持有效的临床模型开发。
URL
https://arxiv.org/abs/2510.05805
