Adversarial
-
Securing Deep Generative Models with Universal Adversarial Signature
2023-05-25 17:59:01Yu Zeng, Mo Zhou, Yuan Xue, Vishal M. PatelAbstract
Recent advances in deep generative models have led to the development of methods capable of synthesizing high-quality, realistic images. These models pose threats to society due to their potential misuse. Prior research attempted to mitigate these threats by detecting generated images, but the varying traces left by different generative models make it challenging to create a universal detector capable of generalizing to new, unseen generative models. In this paper, we propose to inject a universal adversarial signature into an arbitrary pre-trained generative model, in order to make its generated contents more detectable and traceable. First, the imperceptible optimal signature for each image can be found by a signature injector through adversarial training. Subsequently, the signature can be incorporated into an arbitrary generator by fine-tuning it with the images processed by the signature injector. In this way, the detector corresponding to the signature can be reused for any fine-tuned generator for tracking the generator identity. The proposed method is validated on the FFHQ and ImageNet datasets with various state-of-the-art generative models, consistently showing a promising detection rate. Code will be made publicly available at \url{this https URL}.
Abstract (translated)
深度学习模型的最新发展导致能够合成高质量、现实感强的图像的方法的开发。这些模型对社会构成了威胁，因为它们的潜在滥用可能性。先前的研究试图通过检测生成图像来减轻这些威胁，但不同生成模型留下的差异痕迹使得创建一个能够普遍适用于新、未见面的生成模型的通用检测器变得困难。在本文中，我们提议将一种通用的对抗性签名注入任意训练好的生成模型中，以使其生成的内容更容易检测和追踪。首先，通过对抗训练，每个图像的可见最优签名可以通过签名注入器找到。随后，签名可以与由签名注入器处理的图像进行微调，并将其注入任意生成器中。这样，与签名对应的检测器就可以用于任何微调生成器的跟踪生成器身份。该提议方法在FFHQ和ImageNet等各种先进生成模型的多种数据集上进行了验证， consistently 显示有 promising 的检测率。代码将在\url{this https URL}上公开发布。
URL
https://arxiv.org/abs/2305.16310
PDF
-
UDPM: Upsampling Diffusion Probabilistic Models
2023-05-25 17:25:14Shady Abu-Hussein, Raja GiryesAbstract
In recent years, Denoising Diffusion Probabilistic Models (DDPM) have caught significant attention. By composing a Markovian process that starts in the data domain and then gradually adds noise until reaching pure white noise, they achieve superior performance in learning data distributions. Yet, these models require a large number of diffusion steps to produce aesthetically pleasing samples, which is inefficient. In addition, unlike common generative adversarial networks, the latent space of diffusion models is not interpretable. In this work, we propose to generalize the denoising diffusion process into an Upsampling Diffusion Probabilistic Model (UDPM), in which we reduce the latent variable dimension in addition to the traditional noise level addition. As a result, we are able to sample images of size $256\times 256$ with only 7 diffusion steps, which is less than two orders of magnitude compared to standard DDPMs. We formally develop the Markovian diffusion processes of the UDPM, and demonstrate its generation capabilities on the popular FFHQ, LSUN horses, ImageNet, and AFHQv2 datasets. Another favorable property of UDPM is that it is very easy to interpolate its latent space, which is not the case with standard diffusion models. Our code is available online \url{this https URL}
Abstract (translated)
近年来，去噪扩散概率模型(DDPM)吸引了大量关注。通过构建始于数据域的马尔可夫过程，然后逐渐添加噪声，直到达到纯白色噪声的水平，这些模型在学习数据分布方面表现出更好的性能。然而，这些模型需要许多扩散步骤来产生审美上满意的样本，效率较低。此外，与常见的生成对抗网络不同，扩散模型的隐状态空间无法解释。在本文中，我们提议将去噪扩散过程泛化为增采样扩散概率模型(UDPM)，其中我们除了传统的噪声水平增加外，还减少了隐变量维度。因此，我们只需要7个扩散步骤就能样本大小为256×256的图像，比标准DDPM的规模小得多。我们正式开发了UDPM的马尔可夫扩散过程，并在流行的FFHQ、LCNS horses、ImageNet和AFHQv2数据集上展示了其生成能力。UDPM的另一个有利特性是，它很容易进行隐状态空间的插值，而标准扩散模型则无法做到。我们的代码现在在线 \url{this https URL}。
URL
https://arxiv.org/abs/2305.16269
PDF
-
Incomplete Multimodal Learning for Complex Brain Disorders Prediction
2023-05-25 16:29:16Reza Shirkavand, Liang Zhan, Heng Huang, Li Shen, Paul M. ThompsonAbstract
Recent advancements in the acquisition of various brain data sources have created new opportunities for integrating multimodal brain data to assist in early detection of complex brain disorders. However, current data integration approaches typically need a complete set of biomedical data modalities, which may not always be feasible, as some modalities are only available in large-scale research cohorts and are prohibitive to collect in routine clinical practice. Especially in studies of brain diseases, research cohorts may include both neuroimaging data and genetic data, but for practical clinical diagnosis, we often need to make disease predictions only based on neuroimages. As a result, it is desired to design machine learning models which can use all available data (different data could provide complementary information) during training but conduct inference using only the most common data modality. We propose a new incomplete multimodal data integration approach that employs transformers and generative adversarial networks to effectively exploit auxiliary modalities available during training in order to improve the performance of a unimodal model at inference. We apply our new method to predict cognitive degeneration and disease outcomes using the multimodal imaging genetic data from Alzheimer's Disease Neuroimaging Initiative (ADNI) cohort. Experimental results demonstrate that our approach outperforms the related machine learning and deep learning methods by a significant margin.
Abstract (translated)
近年来，获取各种脑数据源的进步，为整合多模态脑数据，帮助早期识别复杂的脑障碍创造了新的机会。然而，当前的数据整合方法通常需要完整的生物医学数据模态，这可能不一定可行，因为一些模态只有在大规模的研究群体才能提供，并且在常规临床实践中收集是禁止的。特别是对于脑疾病的研究，研究群体可能包括神经影像学数据和基因数据，但在实际临床诊断中，我们通常需要仅基于神经影像进行疾病预测。因此，我们希望设计一种机器学习模型，可以在训练期间使用所有可用的数据(不同的数据可以提供补充信息)，但仅使用最常见的数据模态进行推理。我们提出了一种新的不完整的多模态数据整合方法，利用Transformer和生成对抗网络有效地利用训练期间提供的辅助模态，以提高单一模态模型的推理性能。我们应用我们的新方法来预测阿尔茨海默病神经影像学倡议(ADNI)研究群体中的多模态影像基因数据中的脑功能退化和疾病结果。实验结果显示，我们的方法相比相关的机器学习和深度学习方法表现出显著的优势。
URL
https://arxiv.org/abs/2305.16222
PDF
-
On the Robustness of Segment Anything
2023-05-25 16:28:30Yihao Huang, Yue Cao, Tianlin Li, Felix Juefei-Xu, Di Lin, Ivor W.Tsang, Yang Liu, Qing GuoAbstract
Segment anything model (SAM) has presented impressive objectness identification capability with the idea of prompt learning and a new collected large-scale dataset. Given a prompt (e.g., points, bounding boxes, or masks) and an input image, SAM is able to generate valid segment masks for all objects indicated by the prompts, presenting high generalization across diverse scenarios and being a general method for zero-shot transfer to downstream vision tasks. Nevertheless, it remains unclear whether SAM may introduce errors in certain threatening scenarios. Clarifying this is of significant importance for applications that require robustness, such as autonomous vehicles. In this paper, we aim to study the testing-time robustness of SAM under adversarial scenarios and common corruptions. To this end, we first build a testing-time robustness evaluation benchmark for SAM by integrating existing public datasets. Second, we extend representative adversarial attacks against SAM and study the influence of different prompts on robustness. Third, we study the robustness of SAM under diverse corruption types by evaluating SAM on corrupted datasets with different prompts. With experiments conducted on SA-1B and KITTI datasets, we find that SAM exhibits remarkable robustness against various corruptions, except for blur-related corruption. Furthermore, SAM remains susceptible to adversarial attacks, particularly when subjected to PGD and BIM attacks. We think such a comprehensive study could highlight the importance of the robustness issues of SAM and trigger a series of new tasks for SAM as well as downstream vision tasks.
Abstract (translated)
Segment anything模型(Sam)以prompt learning和收集大型数据集的新想法，展示了令人印象深刻的对象识别能力。给定prompt(例如点、边界框或掩膜)和输入图像，Sam能够生成所有由prompt指示的对象的有效分块Mask，在各种情况下表现出高泛化能力，是直接转移到后续视觉任务通用的方法。然而，仍然不清楚Sam在某些威胁情况下可能会引入错误。澄清这一点对于需要鲁棒性的应用程序，例如自动驾驶车辆等具有重要意义。在本文中，我们旨在研究Sam在对抗场景和常见腐败情况下的测试时鲁棒性。为此，我们首先建立了Sam的测试时鲁棒性评估基准，通过整合现有公共数据集。其次，我们扩展了代表性的对抗攻击对Sam进行研究，并探讨不同prompt对鲁棒性的影响了。通过在SAB和KITTI数据集上进行实验，我们发现Sam表现出对多种腐败的 remarkable 鲁棒性，除了与模糊相关的腐败。此外，Sam仍然容易受到对抗攻击，特别是在受到PGD和BIM攻击的情况下。我们认为这种全面研究可以强调Sam的鲁棒性问题的重要性，并触发一系列新的任务，为Sam以及后续视觉任务。
URL
https://arxiv.org/abs/2305.16220
PDF
-
Unifying GANs and Score-Based Diffusion as Generative Particle Models
2023-05-25 15:20:10Jean-Yves Franceschi, Mike Gartrell, Ludovic Dos Santos, Thibaut Issenhuth, Emmanuel de Bézenac, Mickaël Chen, Alain RakotomamonjyAbstract
Particle-based deep generative models, such as gradient flows and score-based diffusion models, have recently gained traction thanks to their striking performance. Their principle of displacing particle distributions by differential equations is conventionally seen as opposed to the previously widespread generative adversarial networks (GANs), which involve training a pushforward generator network. In this paper, we challenge this interpretation and propose a novel framework that unifies particle and adversarial generative models by framing generator training as a generalization of particle models. This suggests that a generator is an optional addition to any such generative model. Consequently, integrating a generator into a score-based diffusion model and training a GAN without a generator naturally emerge from our framework. We empirically test the viability of these original models as proofs of concepts of potential applications of our framework.
Abstract (translated)
粒子based deep生成模型，如梯度流和评分based扩散模型，最近由于其惊人的表现而获得了进展。它们通过微分方程替代粒子分布的替代性原则，通常被视为与之前广泛采用的生成对抗网络(GANs)不同的生成模型。在这篇文章中，我们挑战了这一解释，并提出了一个独特的框架，将生成器训练视为粒子模型的扩展。这意味着生成器是任何这种生成模型的可选添加。因此，将生成器融入评分based扩散模型中，并自然从我们的框架中产生GAN进行训练。我们 empirical 测试了这些原始模型的有效性，作为我们框架的潜在应用概念的证明。
URL
https://arxiv.org/abs/2305.16150
PDF
-
PDE+: Enhancing Generalization via PDE with Adaptive Distributional Diffusion
2023-05-25 08:23:26Yige Yuan, Bingbing Xu, Bo Lin, Liang Hou, Fei Sun, Huawei Shen, Xueqi ChengAbstract
The generalization of neural networks is a central challenge in machine learning, especially concerning the performance under distributions that differ from training ones. Current methods, mainly based on the data-driven paradigm such as data augmentation, adversarial training, and noise injection, may encounter limited generalization due to model non-smoothness. In this paper, we propose to investigate generalization from a Partial Differential Equation (PDE) perspective, aiming to enhance it directly through the underlying function of neural networks, rather than focusing on adjusting input data. Specifically, we first establish the connection between neural network generalization and the smoothness of the solution to a specific PDE, namely ``transport equation''. Building upon this, we propose a general framework that introduces adaptive distributional diffusion into transport equation to enhance the smoothness of its solution, thereby improving generalization. In the context of neural networks, we put this theoretical framework into practice as PDE+ (\textbf{PDE} with \textbf{A}daptive \textbf{D}istributional \textbf{D}iffusion) which diffuses each sample into a distribution covering semantically similar inputs. This enables better coverage of potentially unobserved distributions in training, thus improving generalization beyond merely data-driven methods. The effectiveness of PDE+ is validated in extensive settings, including clean samples and various corruptions, demonstrating its superior performance compared to SOTA methods.
Abstract (translated)
神经网络的泛化是机器学习中的核心挑战，特别是关于训练数据和分布之间的性能。目前的方法，主要是基于数据驱动范式，例如数据增强、对抗训练和噪声注入，可能会因为模型的不平滑而遇到有限的泛化能力。在本文中，我们提议从偏微分方程(PDE)的角度研究泛化问题，旨在通过神经网络的基函数增强其 underlying 函数的平滑性，而不是仅仅关注调整输入数据。具体而言，我们首先建立了神经网络泛化与特定PDE解决方案平滑性的联系，即“传输方程”。基于这一点，我们提出了一个通用框架，将自适应分布扩散引入传输方程，以增强其解决方案的平滑性，从而改善泛化能力。在神经网络的背景下，我们将这个理论框架应用于实践，将其称为PDE+,(PDE with Adaptive Distributional Diffusion)，将每个样本扩散到覆盖语义上相似的输入的分布中。这使能够在训练过程中更好地覆盖可能存在未观测到的分布，从而超越了仅仅基于数据驱动方法的泛化能力。PDE+的效果在广泛的设置中得到了验证，包括干净样本和各种欺诈，证明了它与SOTA方法相比的优越性能。
URL
https://arxiv.org/abs/2305.15835
PDF
-
Healing Unsafe Dialogue Responses with Weak Supervision Signals
2023-05-25 06:15:53Zi Liang, Pinghui Wang, Ruofei Zhang, Shuo Zhang, Xiaofan Ye Yi Huang, Junlan FengAbstract
Recent years have seen increasing concerns about the unsafe response generation of large-scale dialogue systems, where agents will learn offensive or biased behaviors from the real-world corpus. Some methods are proposed to address the above issue by detecting and replacing unsafe training examples in a pipeline style. Though effective, they suffer from a high annotation cost and adapt poorly to unseen scenarios as well as adversarial attacks. Besides, the neglect of providing safe responses (e.g. simply replacing with templates) will cause the information-missing problem of dialogues. To address these issues, we propose an unsupervised pseudo-label sampling method, TEMP, that can automatically assign potential safe responses. Specifically, our TEMP method groups responses into several clusters and samples multiple labels with an adaptively sharpened sampling strategy, inspired by the observation that unsafe samples in the clusters are usually few and distribute in the tail. Extensive experiments in chitchat and task-oriented dialogues show that our TEMP outperforms state-of-the-art models with weak supervision signals and obtains comparable results under unsupervised learning settings.
Abstract (translated)
近年来，人们对大规模对话系统的不安全响应生成日益关注，这些系统将从现实世界的数据集学习具有攻击性或偏见的行为。有一些方法建议通过在管道中检测并替换不安全的训练示例来解决上述问题。虽然有效，但它们面临着高标注成本，并且对于未观察到的场景和对抗攻击的适应性较差。此外，忽略了提供安全响应(例如简单地替换为模板)将会导致对话信息的丢失问题。为了解决这些问题，我们提出了一种 unsupervised 的伪标签采样方法 TEMP，该方法可以自动分配可能的安全响应。具体而言，我们的 TEMP 方法将响应分为多个簇，并使用自适应的增强采样策略样本多个标签，灵感来自于观察簇中的不安全样本通常很少，分布在尾部。在闲聊和任务导向的对话实验中，广泛研究表明，我们的 TEMP 在弱监督信号下的表现力比先进的模型更强，并能够在无监督学习设置下获得类似的结果。
URL
https://arxiv.org/abs/2305.15757
PDF
-
PEARL: Preprocessing Enhanced Adversarial Robust Learning of Image Deraining for Semantic Segmentation
2023-05-25 04:44:17Xianghao Jiao, Yaohua Liu, Jiaxin Gao, Xinyuan Chu, Risheng Liu, Xin FanAbstract
In light of the significant progress made in the development and application of semantic segmentation tasks, there has been increasing attention towards improving the robustness of segmentation models against natural degradation factors (e.g., rain streaks) or artificially attack factors (e.g., adversarial attack). Whereas, most existing methods are designed to address a single degradation factor and are tailored to specific application scenarios. In this work, we present the first attempt to improve the robustness of semantic segmentation tasks by simultaneously handling different types of degradation factors. Specifically, we introduce the Preprocessing Enhanced Adversarial Robust Learning (PEARL) framework based on the analysis of our proposed Naive Adversarial Training (NAT) framework. Our approach effectively handles both rain streaks and adversarial perturbation by transferring the robustness of the segmentation model to the image derain model. Furthermore, as opposed to the commonly used Negative Adversarial Attack (NAA), we design the Auxiliary Mirror Attack (AMA) to introduce positive information prior to the training of the PEARL framework, which improves defense capability and segmentation performance. Our extensive experiments and ablation studies based on different derain methods and segmentation models have demonstrated the significant performance improvement of PEARL with AMA in defense against various adversarial attacks and rain streaks while maintaining high generalization performance across different datasets.
Abstract (translated)
由于在语义分割任务的发展和应用方面取得了显著进展，人们对分割模型的鲁棒性提出了越来越多的关注。相对于大多数现有方法，它们设计用于解决一个特定的退化因子，并针对特定的应用场景进行定制，上述工作提出了一种改进语义分割任务鲁棒性的新方法，即处理不同类型的退化因子。具体而言，我们提出了预处理增强对抗性鲁棒学习框架(PEARL)框架，该框架基于我们提出的简单对抗训练框架(NAT)的分析。我们的方法有效地处理了雨 streaks 和对抗性扰动，通过将分割模型的鲁棒性转移到图像生成模型中，实现了对图像生成模型的增强。此外，与常见的消极对抗攻击(NAA)相比，我们设计了一种辅助 Mirror 攻击(AMA)，在 pearL 框架的训练之前引入了积极信息，从而提高了防御能力和分割性能。基于不同生成模型和分割模型的不同生成方法以及评估方法进行了广泛的实验和凝练研究，证明了 AMA 在抵御各种对抗攻击和雨 streaks 方面显著性能改进，同时在不同数据集上保持了高泛化性能。
URL
https://arxiv.org/abs/2305.15709
PDF
-
Rethink Diversity in Deep Learning Testing
2023-05-25 04:13:51Zi Wang, Jihye Choi, Somesh JhaAbstract
Deep neural networks (DNNs) have demonstrated extraordinary capabilities and are an integral part of modern software systems. However, they also suffer from various vulnerabilities such as adversarial attacks and unfairness. Testing deep learning (DL) systems is therefore an important task, to detect and mitigate those vulnerabilities. Motivated by the success of traditional software testing, which often employs diversity heuristics, various diversity measures on DNNs have been proposed to help efficiently expose the buggy behavior of DNNs. In this work, we argue that many DNN testing tasks should be treated as directed testing problems rather than general-purpose testing tasks, because these tasks are specific and well-defined. Hence, the diversity-based approach is less effective. Following our argument based on the semantics of DNNs and the testing goal, we derive $6$ metrics that can be used for DNN testing and carefully analyze their application scopes. We empirically show their efficacy in exposing bugs in DNNs compared to recent diversity-based metrics. Moreover, we also notice discrepancies between the practices of the software engineering (SE) community and the DL community. We point out some of these gaps, and hopefully, this can lead to bridging the SE practice and DL findings.
Abstract (translated)
深度神经网络(DNN)已经展示了非凡的能力，是现代软件系统的重要组成部分。然而，它们也面临着各种漏洞，如dversarial攻击和不公平。因此，测试深度学习系统(DL)是一个重要任务，以检测和缓解这些漏洞。基于传统软件测试的成功，通常采用多样性启发式方法，提出了各种DNN的多样性措施，以帮助 efficiently暴露DNN的 bug。在这项工作中，我们认为，许多DNN测试任务应该被视为定向测试问题，而不是通用的测试任务，因为这些任务是具体和明确的。因此，基于多样性的方法并不高效。基于DNN语义和测试目标，我们推导了六个可用于DNN测试的度量，并仔细分析了它们的应用范围。我们经验证地证明了它们相对于最近的基于多样性的度量在暴露DNN bug方面的有效性。此外，我们还注意到软件工程(SE)社区和深度学习社区的做法之间存在差异。我们指出了一些这些差距，希望这可以 bridging SE practice和DL findings。
URL
https://arxiv.org/abs/2305.15698
PDF
-
How do humans perceive adversarial text? A reality check on the validity and naturalness of word-based adversarial attacks
2023-05-24 21:52:13Salijona Dyrmishi, Salah Ghamizi, Maxime CordyAbstract
Natural Language Processing (NLP) models based on Machine Learning (ML) are susceptible to adversarial attacks -- malicious algorithms that imperceptibly modify input text to force models into making incorrect predictions. However, evaluations of these attacks ignore the property of imperceptibility or study it under limited settings. This entails that adversarial perturbations would not pass any human quality gate and do not represent real threats to human-checked NLP systems. To bypass this limitation and enable proper assessment (and later, improvement) of NLP model robustness, we have surveyed 378 human participants about the perceptibility of text adversarial examples produced by state-of-the-art methods. Our results underline that existing text attacks are impractical in real-world scenarios where humans are involved. This contrasts with previous smaller-scale human studies, which reported overly optimistic conclusions regarding attack success. Through our work, we hope to position human perceptibility as a first-class success criterion for text attacks, and provide guidance for research to build effective attack algorithms and, in turn, design appropriate defence mechanisms.
Abstract (translated)
基于机器学习(ML)的自然语言处理模型(NLP模型)容易被dversarial攻击所攻击，这些攻击者通过微小的变化来悄悄地修改输入文本，从而迫使模型做出错误的预测。然而，对这些攻击的评价却忽视了 imperceptibility 的特性，或者只在特定条件下进行了研究。这导致dversarial perturbations 无法通过任何人类质量门(human quality gate)通过，并且并不能对人工检查的NLP系统构成真正的威胁。为了绕过这个限制并正确评估(后来改进)NLP模型的鲁棒性，我们对378名人类参与者调查了最先进的方法所生成的文本dversarial examples 的可感知性。我们的结果显示，在人类参与的真实场景下，现有的文本攻击是不切实际的。这与之前较小的人类研究相比，它们 reporting 过份乐观的攻击成功结论。通过我们的工作，我们希望将人类感知作为文本攻击的成功标准之一，并为全球研究人员提供指导，以构建有效的攻击算法，并相应地设计适当的防御机制。
URL
https://arxiv.org/abs/2305.15587
PDF
-
Fast Adversarial CNN-based Perturbation Attack on No-Reference Image- and Video-Quality Metrics
2023-05-24 20:18:21Ekaterina Shumitskaya, Anastasia Antsiferova, Dmitriy VatolinAbstract
Modern neural-network-based no-reference image- and video-quality metrics exhibit performance as high as full-reference metrics. These metrics are widely used to improve visual quality in computer vision methods and compare video processing methods. However, these metrics are not stable to traditional adversarial attacks, which can cause incorrect results. Our goal is to investigate the boundaries of no-reference metrics applicability, and in this paper, we propose a fast adversarial perturbation attack on no-reference quality metrics. The proposed attack (FACPA) can be exploited as a preprocessing step in real-time video processing and compression algorithms. This research can yield insights to further aid in designing of stable neural-network-based no-reference quality metrics.
Abstract (translated)
现代基于神经网络的图像和视频质量指标的表现甚至可以达到全参考指标的水平。这些指标被广泛用于改善计算机视觉方法中的视觉质量和比较视频处理方法。然而，这些指标对于传统的dversarial攻击并不稳定，可能会导致错误的结果。我们的目标是研究无参考指标适用性的的边界，在这篇论文中，我们提出了一种快速dversarial perturbation攻击无参考质量指标。提出的攻击(FACPA)可以在实时视频处理和压缩算法中作为预处理步骤利用。这项研究可以带来见解，以进一步协助设计稳定的基于神经网络的无参考质量指标。
URL
https://arxiv.org/abs/2305.15544
PDF
-
Robust Classification via a Single Diffusion Model
2023-05-24 15:25:19Huanran Chen, Yinpeng Dong, Zhengyi Wang, Xiao Yang, Chengqi Duan, Hang Su, Jun ZhuAbstract
Recently, diffusion models have been successfully applied to improving adversarial robustness of image classifiers by purifying the adversarial noises or generating realistic data for adversarial training. However, the diffusion-based purification can be evaded by stronger adaptive attacks while adversarial training does not perform well under unseen threats, exhibiting inevitable limitations of these methods. To better harness the expressive power of diffusion models, in this paper we propose Robust Diffusion Classifier (RDC), a generative classifier that is constructed from a pre-trained diffusion model to be adversarially robust. Our method first maximizes the data likelihood of a given input and then predicts the class probabilities of the optimized input using the conditional likelihood of the diffusion model through Bayes' theorem. Since our method does not require training on particular adversarial attacks, we demonstrate that it is more generalizable to defend against multiple unseen threats. In particular, RDC achieves $73.24\%$ robust accuracy against $\ell_\infty$ norm-bounded perturbations with $\epsilon_\infty=8/255$ on CIFAR-10, surpassing the previous state-of-the-art adversarial training models by $+2.34\%$. The findings highlight the potential of generative classifiers by employing diffusion models for adversarial robustness compared with the commonly studied discriminative classifiers.
Abstract (translated)
近年来，扩散模型已经成功应用于改善图像分类器抗干扰能力，通过纯洁化dversarial噪声或生成真实的训练数据来实现。然而，基于扩散的纯洁化可以通过更强的自适应攻击来逃避，而抗干扰训练在未知的威胁下表现不佳，表现出这些方法不可避免的局限性。为了更好地利用扩散模型的表达力，在本文中，我们提出了 robust Diffusion Classifier (RDC)，是一种生成分类器，其训练模型为先前训练的扩散模型，具有抗干扰能力。我们的算法首先最大化给定输入的数据概率，然后使用扩散模型的条件概率来预测优化输入的分类概率，使用贝叶斯定理。由于我们不需要针对特定的dversarial攻击进行训练，我们证明它可以更广泛地应用于防御多个未知的威胁。特别是，RDC在CIFAR-10中通过$\ell_\infty$范数下Bounded的威胁对实现$73.24\%$的抗干扰准确性，比先前最先进的dversarial训练模型高出$2.34\%$。研究结果强调了使用扩散模型提高抗干扰能力相对于通常研究的辨别分类器的潜在能力。
URL
https://arxiv.org/abs/2305.15241
PDF
-
Relating Implicit Bias and Adversarial Attacks through Intrinsic Dimension
2023-05-24 14:40:23Lorenzo Basile, Nikos Karantzas, Alberto D'Onofrio, Luca Bortolussi, Alex Rodriguez, Fabio AnselmiAbstract
Despite their impressive performance in classification, neural networks are known to be vulnerable to adversarial attacks. These attacks are small perturbations of the input data designed to fool the model. Naturally, a question arises regarding the potential connection between the architecture, settings, or properties of the model and the nature of the attack. In this work, we aim to shed light on this problem by focusing on the implicit bias of the neural network, which refers to its inherent inclination to favor specific patterns or outcomes. Specifically, we investigate one aspect of the implicit bias, which involves the essential Fourier frequencies required for accurate image classification. We conduct tests to assess the statistical relationship between these frequencies and those necessary for a successful attack. To delve into this relationship, we propose a new method that can uncover non-linear correlations between sets of coordinates, which, in our case, are the aforementioned frequencies. By exploiting the entanglement between intrinsic dimension and correlation, we provide empirical evidence that the network bias in Fourier space and the target frequencies of adversarial attacks are closely tied.
Abstract (translated)
尽管神经网络在分类方面表现惊人，但它们仍然容易受到dversarial攻击。这些攻击是输入数据中的微小的扰动，旨在欺骗模型。自然地，一个问题出现了，即模型的结构、设置或属性与攻击的性质之间可能存在的潜在联系。在本研究中，我们旨在通过关注神经网络的隐含偏见来解决这个问题，隐含偏见是指其固有的倾向，以偏好特定的模式或结果。具体而言，我们研究了隐含偏见的一个方面，涉及准确图像分类所需的关键傅里叶频率。我们进行了测试，以评估这些频率和成功攻击所需的统计关系。为了深入探讨这个问题，我们提出了一种新方法，可以揭示一组坐标之间的非线性相关关系，这些坐标是指在我们的情况下，上述频率。通过利用内在维度和相关性的纠缠，我们提供了经验证据，表明在 Fourier 空间中的网络偏见和dversarial 攻击的目标频率之间是紧密联系的。
URL
https://arxiv.org/abs/2305.15203
PDF
-
Another Dead End for Morphological Tags? Perturbed Inputs and Parsing
Abstract
The usefulness of part-of-speech tags for parsing has been heavily questioned due to the success of word-contextualized parsers. Yet, most studies are limited to coarse-grained tags and high quality written content; while we know little about their influence when it comes to models in production that face lexical errors. We expand these setups and design an adversarial attack to verify if the use of morphological information by parsers: (i) contributes to error propagation or (ii) if on the other hand it can play a role to correct mistakes that word-only neural parsers make. The results on 14 diverse UD treebanks show that under such attacks, for transition- and graph-based models their use contributes to degrade the performance even faster, while for the (lower-performing) sequence labeling parsers they are helpful. We also show that if morphological tags were utopically robust against lexical perturbations, they would be able to correct parsing mistakes.
Abstract (translated)
词干标记对于解析的有用性一直受到严重质疑，因为这得益于 word-contextualization 解析器的成功。然而，大多数研究都局限于粒度粗的标记和高质量的写作内容；而我们对于面临词汇错误 production 模型的影响了解得很少。我们扩展这些 setups 并设计了一场对抗攻击来验证parsers 使用形态信息的作用：(i)是否会促进错误传播，(ii)如果他们能够发挥作用来纠正仅使用单词神经网络解析器犯下的错误。对14个不同的 UD 树库的结果表明，在这些攻击下，对于过渡和图形模型，他们的使用加速了性能的下降，而对于低表现序列标签解析器则它们是有益的。我们还表明，如果形态标记乌托邦地 robust 于词汇颠覆，它们能够纠正解析错误。
URL
https://arxiv.org/abs/2305.15119
PDF
-
Unpaired Image-to-Image Translation via Neural Schrödinger Bridge
2023-05-24 12:05:24Beomsu Kim, Gihyun Kwon, Kwanyoung Kim, Jong Chul YeAbstract
Diffusion models are a powerful class of generative models which simulate stochastic differential equations (SDEs) to generate data from noise. Although diffusion models have achieved remarkable progress in recent years, they have limitations in the unpaired image-to-image translation tasks due to the Gaussian prior assumption. Schrödinger Bridge (SB), which learns an SDE to translate between two arbitrary distributions, have risen as an attractive solution to this problem. However, none of SB models so far have been successful at unpaired translation between high-resolution images. In this work, we propose the Unpaired Neural Schrödinger Bridge (UNSB), which combines SB with adversarial training and regularization to learn a SB between unpaired data. We demonstrate that UNSB is scalable, and that it successfully solves various unpaired image-to-image translation tasks. Code: \url{this https URL}
Abstract (translated)
扩散模型是一种强大的生成模型，模拟随机微分方程(SDEs)从噪声中生成数据。尽管扩散模型近年来取得了显著进展，但由于高分辨率图像之间的非配对翻译任务依赖于高斯先验假设，它们在这些任务中具有限制。肖莱姆桥(SB)是一种学习两个任意分布之间的SDEs的模型，因此成为解决这个问题的一种有吸引力的解决方案。然而，迄今为止，所有SB模型都没有在配对高分辨率图像之间的非配对翻译任务中成功实现。在本文中，我们提出了未配对神经网络肖莱姆桥(UNSB)，它将SB与对抗训练和正则化相结合，以学习配对数据之间的UNSB。我们证明了UNSB是可扩展的，并且它成功地解决了各种配对图像到图像翻译任务。代码： \url{this https URL}
URL
https://arxiv.org/abs/2305.15086
PDF
-
Non-adversarial Robustness of Deep Learning Methods for Computer Vision
2023-05-24 10:21:31Gorana Gojić, Vladimir Vincan, Ognjen Kundačina, Dragiša Mišković, Dinu DraganAbstract
Non-adversarial robustness, also known as natural robustness, is a property of deep learning models that enables them to maintain performance even when faced with distribution shifts caused by natural variations in data. However, achieving this property is challenging because it is difficult to predict in advance the types of distribution shifts that may occur. To address this challenge, researchers have proposed various approaches, some of which anticipate potential distribution shifts, while others utilize knowledge about the shifts that have already occurred to enhance model generalizability. In this paper, we present a brief overview of the most recent techniques for improving the robustness of computer vision methods, as well as a summary of commonly used robustness benchmark datasets for evaluating the model's performance under data distribution shifts. Finally, we examine the strengths and limitations of the approaches reviewed and identify general trends in deep learning robustness improvement for computer vision.
Abstract (translated)
非自适应鲁棒性(也被称为自然鲁棒性)是深度学习模型的一种属性，使其能够在面临数据自然变异的情况下保持性能。然而，实现这种属性是挑战性的，因为难以在 advance 上预测可能发生的分布变异类型。为了应对这种挑战，研究人员提出了各种方法，其中一些方法能够预见潜在的分布变异，而另一些方法则利用已经发生的变异知识来提高模型的泛化能力。在本文中，我们将简要介绍最近用于提高计算机视觉方法鲁棒性的技术，并摘要介绍常用的鲁棒性基准数据集，用于评估模型在数据分布变异下的性能。最后，我们将审查所综述的方法的优点和局限性，并识别深度学习计算机视觉鲁棒性改进的一般趋势。
URL
https://arxiv.org/abs/2305.14986
PDF
-
Dual-Side Feature Fusion 3D Pose Transfer
Abstract
3D pose transfer solves the problem of additional input and correspondence of traditional deformation transfer, only the source and target meshes need to be input, and the pose of the source mesh can be transferred to the target mesh. Some lightweight methods proposed in recent years consume less memory but cause spikes and distortions for some unseen poses, while others are costly in training due to the inclusion of large matrix multiplication and adversarial networks. In addition, the meshes with different numbers of vertices also increase the difficulty of pose transfer. In this work, we propose a Dual-Side Feature Fusion Pose Transfer Network to improve the pose transfer accuracy of the lightweight method. Our method takes the pose features as one of the side inputs to the decoding network and fuses them into the target mesh layer by layer at multiple scales. Our proposed Feature Fusion Adaptive Instance Normalization has the characteristic of having two side input channels that fuse pose features and identity features as denormalization parameters, thus enhancing the pose transfer capability of the network. Extensive experimental results show that our proposed method has stronger pose transfer capability than state-of-the-art methods while maintaining a lightweight network structure, and can converge faster.
Abstract (translated)
3D 姿态转移解决了传统变形转移额外的输入和对应问题，只需要输入源和目标网格，可以将源网格的姿态转移到目标网格。近年来提出的一些轻量级方法虽然消耗较少内存，但对一些未知的姿态会引起尖点和扭曲，而另一些方法在训练时因为包含大型矩阵乘法和对抗网络而成本较高。此外，不同数量的顶点网格也增加了姿态转移的难度。在本研究中，我们提出了一种双重界面特征融合姿态转移网络，以提高轻量级方法的姿态转移精度。我们的方法将姿态特征作为侧面输入到解码网络中，并逐层将它们与目标网格的特征层融合。我们提出的特征融合自适应实例归一化具有两个侧面输入通道的特征，将姿态特征和身份特征作为归一化参数，从而增强网络的姿态转移能力。广泛的实验结果表明，我们提出的方法具有比当前方法更强的姿态转移能力，同时保持轻量级网络结构，并且可以更快地收敛。
URL
https://arxiv.org/abs/2305.14951
PDF
-
Adversarial Demonstration Attacks on Large Language Models
2023-05-24 09:40:56Jiongxiao Wang, Zichen Liu, Keun Hee Park, Muhao Chen, Chaowei XiaoAbstract
With the emergence of more powerful large language models (LLMs), such as ChatGPT and GPT-4, in-context learning (ICL) has gained significant prominence in leveraging these models for specific tasks by utilizing data-label pairs as precondition prompts. While incorporating demonstrations can greatly enhance the performance of LLMs across various tasks, it may introduce a new security concern: attackers can manipulate only the demonstrations without changing the input to perform an attack. In this paper, we investigate the security concern of ICL from an adversarial perspective, focusing on the impact of demonstrations. We propose an ICL attack based on TextAttack, which aims to only manipulate the demonstration without changing the input to mislead the models. Our results demonstrate that as the number of demonstrations increases, the robustness of in-context learning would decreases. Furthermore, we also observe that adversarially attacked demonstrations exhibit transferability to diverse input examples. These findings emphasize the critical security risks associated with ICL and underscore the necessity for extensive research on the robustness of ICL, particularly given its increasing significance in the advancement of LLMs.
Abstract (translated)
随着更强大的大型语言模型(LLM)的出现，如ChatGPT和GPT-4，上下文学习(ICL)在利用这些模型完成特定任务方面变得越来越重要。尽管包括演示可以显著提高LLM在各种任务中的表现，但它可能带来新的安全担忧：攻击者只能操纵演示，而无需改变输入来进行攻击。在本文中，我们从对抗性视角来研究ICL的安全关切，重点关注演示的影响。我们提出了基于TextAttack的ICL攻击，旨在仅操纵演示，而无需改变输入，误导模型。我们的结果显示，随着演示数量的增加，上下文学习的鲁棒性将减少。此外，我们还观察到，对抗性攻击演示具有可移植到多种输入示例的能力。这些发现强调了ICL相关的关键安全风险，并强调了深入研究ICL的鲁棒性的必要性，特别是考虑到LLM在语言模型发展中日益的重要性。
URL
https://arxiv.org/abs/2305.14950
PDF
-
Cross-lingual Data Augmentation for Document-grounded Dialog Systems in Low Resource Languages
Abstract
This paper proposes a framework to address the issue of data scarcity in Document-Grounded Dialogue Systems(DGDS). Our model leverages high-resource languages to enhance the capability of dialogue generation in low-resource languages. Specifically, We present a novel pipeline CLEM (Cross-Lingual Enhanced Model) including adversarial training retrieval (Retriever and Re-ranker), and Fid (fusion-in-decoder) generator. To further leverage high-resource language, we also propose an innovative architecture to conduct alignment across different languages with translated training. Extensive experiment results demonstrate the effectiveness of our model and we achieved 4th place in the DialDoc 2023 Competition. Therefore, CLEM can serve as a solution to resource scarcity in DGDS and provide useful guidance for multi-lingual alignment tasks.
Abstract (translated)
本文提出了一个框架来解决文档grounded对话系统(DGDS)中数据稀缺的问题。我们的模型利用高资源语言来增强低资源语言对话生成的能力。具体来说，我们提出了一种 novel pipeline CLEM(跨语言增强模型)，包括对抗训练检索(Retriever and Re-ranker)和 Fid(解码器中的融合)生成器。为了进一步利用高资源语言，我们还提出了一种创新架构，以通过翻译训练进行跨语言对齐。广泛的实验结果显示我们的模型的有效性，我们在2023年Dialdoc竞赛中获得了第四名。因此，CLEM可以作为DGDS中资源稀缺的解决方案，并为多语言对齐任务提供有用的指导。
URL
https://arxiv.org/abs/2305.14949
PDF
-
Chain-of-Questions Training with Latent Answers for Robust Multistep Question Answering
2023-05-24 08:55:08Wang Zhu, Jesse Thomason, Robin JiaAbstract
We train a language model (LM) to robustly answer multistep questions by generating and answering sub-questions. We propose Chain-of-Questions, a framework that trains a model to generate sub-questions and sub-answers one at a time by leveraging human annotated question decomposition meaning representation (QDMR). The key technical challenge is that QDMR only contains sub-questions but not answers to those sub-questions, so we treat sub-answers as latent variables and optimize them using a novel dynamic mixture of Hard-EM and MAPO. Chain-of-Questions greatly outperforms strong neuro-symbolic methods by 9.0 F1 on DROP contrast set, and outperforms GPT-3.5 by 24.3 F1 on HOTPOTQA adversarial set, thus demonstrating the effectiveness and robustness of our framework.
Abstract (translated)
我们训练了一个语言模型(LM)以通过生成和回答子问题来 robustly 回答多步骤问题。我们提出了 Chain-of-Questions 框架，该框架利用人类标注的问题分解意义表示(QDMR)来训练模型逐个生成子问题和子回答，从而实现了对多步骤问题的有效回答。然而，该框架的主要技术挑战是 QDMR 仅包含子问题，但不含对这些子问题的解答，因此我们将子回答视为潜在变量，并使用 Hard-EM 和 MAPO 的新型动态混合来优化它们。 Chain-of-Questions 在 DROP Contrast 集合上通过 9.0 F1 显著超越了强大的神经符号方法，而在 HOTpotQA 对抗性集合上比 GPT-3.5 提高了 24.3 F1，从而证明了我们框架的有效性和鲁棒性。
URL
https://arxiv.org/abs/2305.14901
PDF
