Abstract
Large Language Models (LLMs) have achieved remarkable performance and received significant research interest. The enormous computational demands, however, hinder the local deployment on devices with limited resources. The current prevalent LLM inference paradigms require users to send queries to the service providers for processing, which raises critical privacy concerns. Existing approaches propose to allow the users to obfuscate the token embeddings before transmission and utilize local models for denoising. Nonetheless, transmitting the token embeddings and deploying local models may result in excessive communication and computation overhead, preventing practical implementation. In this work, we propose \textbf{DEL}, a framework for \textbf{D}ifferentially private and communication \textbf{E}fficient \textbf{L}LM split inference. More specifically, an embedding projection module and a differentially private stochastic quantization mechanism are proposed to reduce the communication overhead in a privacy-preserving manner. To eliminate the need for local models, we adapt soft prompt at the server side to compensate for the utility degradation caused by privacy. To the best of our knowledge, this is the first work that utilizes soft prompt to improve the trade-off between privacy and utility in LLM inference, and extensive experiments on text generation and natural language understanding benchmarks demonstrate the effectiveness of the proposed method.
Abstract (translated)
大型语言模型（LLMs）已经取得了显著的性能，并且受到了广泛的研究兴趣。然而，巨大的计算需求阻碍了这些模型在资源有限的设备上的本地部署。目前流行的LLM推理范式要求用户将查询发送给服务提供商进行处理，这引发了重要的隐私问题。现有方法提出允许用户在传输之前对令牌嵌入进行混淆处理，并利用本地模型进行去噪。然而，传输令牌嵌入和部署本地模型可能会导致过多的通信和计算开销，从而阻碍实际应用。 为此，我们提出了**DEL框架**（Differentially private and communication Efficient LLM split inference），旨在以保护隐私的方式减少通信开销并实现有效的LLM分割推理。具体来说，我们设计了一个嵌入投影模块，并提出了一种差分私有随机量化机制来降低通信开销。为了消除对本地模型的需求，我们在服务器端采用软提示（soft prompt）来补偿由于隐私保护导致的效用下降。 据我们所知，这是首次利用软提示来改进LLM推理中隐私与性能之间的权衡的工作，并且在文本生成和自然语言理解基准上的广泛实验表明了该方法的有效性。
URL
https://arxiv.org/abs/2602.11513
