Abstract
This work adapts and studies the gradient-based Membership Inference Test (gMINT) to the classification of text based on LLMs. MINT is a general approach intended to determine if given data was used for training machine learning models, and this work focuses on its application to the domain of Natural Language Processing. Using gradient-based analysis, the MINT model identifies whether particular data samples were included during the language model training phase, addressing growing concerns about data privacy in machine learning. The method was evaluated in seven Transformer-based models and six datasets comprising over 2.5 million sentences, focusing on text classification tasks. Experimental results demonstrate MINTs robustness, achieving AUC scores between 85% and 99%, depending on data size and model architecture. These findings highlight MINTs potential as a scalable and reliable tool for auditing machine learning models, ensuring transparency, safeguarding sensitive data, and fostering ethical compliance in the deployment of AI/NLP technologies.
Abstract (translated)
这项工作将基于梯度的成员推断测试（gMINT）应用于基于大型语言模型（LLMs）的文本分类，并对其进行了研究。MINT是一种通用方法，旨在确定给定的数据是否用于训练机器学习模型，而本工作则重点关注其在自然语言处理领域的应用。通过基于梯度的分析，MINT模型能够识别特定数据样本是否包含于语言模型的训练阶段中，从而应对日益增长的关于机器学习中的数据隐私问题的关注。 该方法在七个基于Transformer的模型和六个包含超过250万句子的数据集上进行了评估，重点在于文本分类任务。实验结果表明，MINT具有较强的鲁棒性，在不同数据规模和模型架构下达到了85%到99%之间的AUC评分。这些发现凸显了MINT作为一种可扩展且可靠的工具在机器学习模型审计中的潜力，确保透明度、保护敏感信息，并促进AI/NLP技术部署过程中的伦理合规性。
URL
https://arxiv.org/abs/2503.07384
