Abstract
Text data has become extremely valuable on large language models (LLMs) and even lead to general artificial intelligence (AGI). A lot of high-quality text in the real world is private and cannot be freely used due to privacy concerns. Therefore, differentially private (DP) synthetic text generation has been proposed, aiming to produce high-utility synthetic data while protecting sensitive information. However, existing DP synthetic text generation imposes uniform guarantees that often overprotect non-sensitive content, resulting in substantial utility loss and computational overhead. Therefore, we propose Secret-Protected Evolution (SecPE), a novel framework that extends private evolution with secret-aware protection. Theoretically, we show that SecPE satisfies $(\mathrm{p}, \mathrm{r})$-secret protection, constituting a relaxation of Gaussian DP that enables tighter utility-privacy trade-offs, while also substantially reducing computational complexity relative to baseline methods. Empirically, across the OpenReview, PubMed, and Yelp benchmarks, SecPE consistently achieves lower Fréchet Inception Distance (FID) and higher downstream task accuracy than GDP-based Aug-PE baselines, while requiring less noise to attain the same level of protection. Our results highlight that secret-aware guarantees can unlock more practical and effective privacy-preserving synthetic text generation.
Abstract (translated)
文本数据在大型语言模型（LLMs）中变得极其有价值，甚至可以引领通用人工智能（AGI）的发展。然而，现实世界中的大量高质量文本由于隐私问题而无法自由使用且是私有的。因此，提出了差异隐私（DP）合成文本生成方法，旨在在保护敏感信息的同时产生高价值的合成数据。但是，现有的DP合成文本生成方法通常对非敏感内容提供过度保护，导致显著的效用损失和计算开销增加。为此，我们提出了一种新的框架——秘密保护演化（Secret-Protected Evolution, SecPE），该框架通过引入对秘密信息的认识来扩展私有演化的保护措施。 从理论上讲，我们证明SecPE满足$(\mathrm{p}, \mathrm{r})$-秘密保护标准，这是一个高斯DP的放松版本，能够实现更加紧密的效用与隐私权衡，并且相对于基线方法大幅减少了计算复杂性。在实证研究中，通过OpenReview、PubMed和Yelp等基准数据集上的实验表明，SecPE在弗雷歇 inception 距离（FID）方面始终低于基于GDP的Aug-PE基线模型，并实现了更高的下游任务准确性，同时达到了相同级别的保护所需的噪声更少。我们的研究结果强调了秘密意识保证可以在隐私保护合成文本生成中解锁更多实用且有效的解决方案。
URL
https://arxiv.org/abs/2510.10990
