Abstract
The need for interpretability in deep learning has driven interest in counterfactual explanations, which identify minimal changes to an instance that change a model's prediction. Current counterfactual (CF) generation methods require task-specific fine-tuning and produce low-quality text. Large Language Models (LLMs), though effective for high-quality text generation, struggle with label-flipping counterfactuals (i.e., counterfactuals that change the prediction) without fine-tuning. We introduce two simple classifier-guided approaches to support counterfactual generation by LLMs, eliminating the need for fine-tuning while preserving the strengths of LLMs. Despite their simplicity, our methods outperform state-of-the-art counterfactual generation methods and are effective across different LLMs, highlighting the benefits of guiding counterfactual generation by LLMs with classifier information. We further show that data augmentation by our generated CFs can improve a classifier's robustness. Our analysis reveals a critical issue in counterfactual generation by LLMs: LLMs rely on parametric knowledge rather than faithfully following the classifier.
Abstract (translated)
深度学习中对可解释性的需求推动了反事实解释的兴趣，这种解释能识别出使模型预测发生变化的实例最小修改。目前的反事实（CF）生成方法需要针对特定任务进行微调，并且会产生质量较低的文本。虽然大型语言模型（LLMs）在高质量文本生成方面非常有效，但它们在没有微调的情况下难以产生标签翻转型反事实（即改变预测结果的反事实）。我们介绍了两种简单的基于分类器引导的方法来支持由LLM生成反事实，这些方法消除了对微调的需求，同时保持了LLMs的优点。尽管这两种方法很简单，但它们超越了最先进的反事实生成方法，并且在不同的LLMs上都非常有效，这突显了通过分类器信息指导LLMs进行反事实生成的好处。我们进一步展示了由我们的生成CF进行的数据增强可以提高分类器的鲁棒性。我们的分析揭示了一个关键问题：LLMs依赖于参数知识而不是忠实遵循分类器。
URL
https://arxiv.org/abs/2503.04463
