Abstract
We explore how private synthetic text can be generated by suitably prompting a large language model (LLM). This addresses a challenge for organizations like hospitals, which hold sensitive text data like patient medical records, and wish to share it in order to train machine learning models for medical tasks, while preserving patient privacy. Methods that rely on training or finetuning a model may be out of reach, either due to API limits of third-party LLMs, or due to ethical and legal prohibitions on sharing the private data with the LLM itself. We propose Differentially Private Keyphrase Prompt Seeding (DP-KPS), a method that generates a private synthetic text corpus from a sensitive input corpus, by accessing an LLM only through privatized prompts. It is based on seeding the prompts with private samples from a distribution over phrase embeddings, thus capturing the input corpus while achieving requisite output diversity and maintaining differential privacy. We evaluate DP-KPS on downstream ML text classification tasks, and show that the corpora it generates preserve much of the predictive power of the original ones. Our findings offer hope that institutions can reap ML insights by privately sharing data with simple prompts and little compute.
Abstract (translated)
我们研究了通过适当提示大型语言模型（LLM）生成私人合成文本的方法。这解决了像医院这样的组织面临的挑战，这些组织持有敏感的文本数据，如患者医疗记录，并希望在保护患者隐私的同时分享这些数据以训练用于医学任务的机器学习模型。依赖于对模型进行训练或微调的方法可能难以实现，要么是因为第三方LLM的API限制，要么是由于将私有数据与LLM本身共享时存在的伦理和法律禁令。我们提出了一种名为“差异隐私关键词提示播种”（DP-KPS）的方法，该方法通过仅使用私有化提示访问LLM来从敏感输入语料库生成私人合成文本语料库。它基于使用来自短语嵌入分布的私有样本对提示进行播种，从而捕获输入语料库的同时实现所需的输出多样性并保持差分隐私。我们在下游ML文本分类任务上评估了DP-KPS，并展示了其所生成的语料库保留了原始数据大部分的预测能力。我们的发现表明，机构可以通过简单的提示和少量计算私下分享数据来获取机器学习见解，这为未来的应用提供了希望。
URL
https://arxiv.org/abs/2502.13193
