Abstract
Large language models (LLMs) are increasingly adapted to proprietary and domain-specific corpora that contain sensitive information, creating a tension between formal privacy guarantees and efficient deployment through model compression. Differential privacy (DP), typically enforced via DP-SGD, provides record-level protection but often incurs substantial utility loss in autoregressive generation, where optimization noise can amplify exposure bias and compounding errors along long rollouts. Existing approaches to private distillation either apply DP-SGD to both teacher and student, worsening computation and the privacy--utility tradeoff, or rely on DP synthetic text generation from a DP-trained teacher, avoiding DP on the student at the cost of DP-optimizing a large teacher and introducing an offline generation pipeline. We propose \textbf{Differentially Private On-Policy Distillation (DP-OPD)}, a synthesis-free framework that enforces privacy solely through DP-SGD on the student while leveraging a frozen teacher to provide dense token-level targets on \emph{student-generated} trajectories. DP-OPD instantiates this idea via \emph{private generalized knowledge distillation} on continuation tokens. Under a strict privacy budget ($\varepsilon=2.0$), DP-OPD improves perplexity over DP fine-tuning and off-policy DP distillation, and outperforms synthesis-based DP distillation (Yelp: 44.15$\rightarrow$41.68; BigPatent: 32.43$\rightarrow$30.63), while substantially simplifying the training pipeline. In particular, \textbf{DP-OPD collapses private compression into a single DP student-training loop} by eliminating DP teacher training and offline synthetic text generation. Code will be released upon publication at this https URL.
Abstract (translated)
大型语言模型（LLMs）正日益适配于包含敏感信息的专有和领域特定语料库，这造成了形式化隐私保证与通过模型压缩实现高效部署之间的紧张关系。通常通过DP-SGD实施的差分隐私（DP）提供记录级保护，但在自回归生成中常导致显著的效用损失，因为优化噪声可能加剧曝光偏差并沿长序列传播累积错误。现有的私有蒸馏方法要么对教师和学生均应用DP-SGD，从而恶化计算负担与隐私-效用权衡；要么依赖从DP训练教师生成DP合成文本，虽避免对学生进行DP处理，却需以DP优化大型教师为代价并引入离线生成流程。我们提出\textbf{差分隐私同策略蒸馏（DP-OPD）}，一个无合成框架，该框架仅通过学生的DP-SGD实施隐私保护，同时利用冻结的教师为学生生成轨迹上的密集词级目标提供指导。DP-OPD通过在续写词上实施\textit{私有广义知识蒸馏}来实现这一构想。在严格隐私预算（$\varepsilon=2.0$）下，DP-OPD相较于DP微调和离策略DP蒸馏提升了困惑度表现，并超越了基于合成的DP蒸馏（Yelp：44.15$\rightarrow$41.68；BigPatent：32.43$\rightarrow$30.63），同时大幅简化了训练流程。特别地，\textbf{DP-OPD通过消除DP教师训练和离线合成文本生成，将私有压缩整合至单一DP学生训练循环中}。代码将在发表后通过此https URL发布。
URL
https://arxiv.org/abs/2604.04461
