Abstract
Detecting LLM training data is generally framed as a membership inference attack (MIA) problem. However, conventional MIAs operate passively on fixed model weights, using log-likelihoods or text generations. In this work, we introduce \textbf{Active Data Reconstruction Attack} (ADRA), a family of MIA that actively induces a model to reconstruct a given text through training. We hypothesize that training data are \textit{more reconstructible} than non-members, and the difference in their reconstructibility can be exploited for membership inference. Motivated by findings that reinforcement learning (RL) sharpens behaviors already encoded in weights, we leverage on-policy RL to actively elicit data reconstruction by finetuning a policy initialized from the target model. To effectively use RL for MIA, we design reconstruction metrics and contrastive rewards. The resulting algorithms, \textsc{ADRA} and its adaptive variant \textsc{ADRA+}, improve both reconstruction and detection given a pool of candidate data. Experiments show that our methods consistently outperform existing MIAs in detecting pre-training, post-training, and distillation data, with an average improvement of 10.7\% over the previous runner-up. In particular, \MethodPlus~improves over Min-K\%++ by 18.8\% on BookMIA for pre-training detection and by 7.6\% on AIME for post-training detection.
Abstract (translated)
检测大型语言模型（LLM）的训练数据通常被视为成员推断攻击（MIA）问题。然而，传统的MIA是被动操作固定模型权重的方法，使用对数似然或文本生成。在这项工作中，我们引入了**主动数据重建攻击**（ADRA），这是一个通过训练诱导模型重构给定文本的一类MIA方法。我们的假设是，训练数据比非成员数据更具可重构性，并且它们之间可重构性的差异可以用于成员推断。鉴于强化学习（RL）能够增强权重中已编码的行为这一发现，我们利用在目标模型上初始化策略的在策略RL来主动引发数据重建。为了有效使用RL进行MIA，我们设计了重构指标和对比奖励。由此产生的算法，ADRA及其自适应变体ADRA+，在给定候选数据池的情况下，在重构和检测方面都得到了改进。实验表明，我们的方法在检测预训练、后训练以及蒸馏数据时，相对于现有的MIA方法平均提高了10.7%，超越了之前的亚军。特别地，\MethodPlus 在BookMIA上的预训练检测中比Min-K\%++提升了18.8%，而在AIME上的后训练检测中提升了7.6%。
URL
https://arxiv.org/abs/2602.19020
