Abstract
LLM-powered chatbots are becoming widely adopted in applications such as healthcare, personal assistants, industry hiring decisions, etc. In many of these cases, chatbots are fed sensitive, personal information in their prompts, as samples for in-context learning, retrieved records from a database, or as part of the conversation. The information provided in the prompt could directly appear in the output, which might have privacy ramifications if there is sensitive information there. As such, in this paper, we aim to understand the input copying and regurgitation capabilities of these models during inference and how they can be directly instructed to limit this copying by complying with regulations such as HIPAA and GDPR, based on their internal knowledge of them. More specifically, we find that when ChatGPT is prompted to summarize cover letters of a 100 candidates, it would retain personally identifiable information (PII) verbatim in 57.4% of cases, and we find this retention to be non-uniform between different subgroups of people, based on attributes such as gender identity. We then probe ChatGPT's perception of privacy-related policies and privatization mechanisms by directly instructing it to provide compliant outputs and observe a significant omission of PII from output.
Abstract (translated)
利用机器学习(LM)技术构建的聊天机器人在医疗、个人助理、行业雇用决策等应用中越来越普遍采用。在这些应用中，聊天机器人在提示中接收敏感个人信息，作为上下文学习样本、从数据库中检索记录或者作为对话的一部分。提示中提供的信息可以直接出现在输出中，如果存在敏感信息，这可能会带来隐私问题。因此，在本文中，我们旨在理解这些模型在推理时的输入复制和再分发能力，以及它们如何通过遵守HIPAA和GDPR等法规来限制这种复制，基于它们内部的相关知识。具体来说，我们发现，当ChatGPT被提示概括100个候选人的求职信时，它在57.4%的案例中保留相同的个人身份信息(PII)，并且我们发现这种保留在不同人群体中并不均匀，基于性别身份等属性。随后，我们直接指示ChatGPT提供符合法规的输出，并观察输出中显著缺少PII的情况。
URL
https://arxiv.org/abs/2305.15008
