Abstract
Deep neural networks are applied in more and more areas of everyday life. However, they still lack essential abilities, such as robustly dealing with spatially transformed input signals. Approaches to mitigate this severe robustness issue are limited to two pathways: Either models are implicitly regularised by increased sample variability (data augmentation) or explicitly constrained by hard-coded inductive biases. The limiting factor of the former is the size of the data space, which renders sufficient sample coverage intractable. The latter is limited by the engineering effort required to develop such inductive biases for every possible scenario. Instead, we take inspiration from human behaviour, where percepts are modified by mental or physical actions during inference. We propose a novel technique to emulate such an inference process for neural nets. This is achieved by traversing a sparsified inverse transformation tree during inference using parallel energy-based evaluations. Our proposed inference algorithm, called Inverse Transformation Search (ITS), is model-agnostic and equips the model with zero-shot pseudo-invariance to spatially transformed inputs. We evaluated our method on several benchmark datasets, including a synthesised ImageNet test set. ITS outperforms the utilised baselines on all zero-shot test scenarios.
Abstract (translated)
深度神经网络在越来越多的日常应用领域得到应用。然而，它们仍然缺乏一些关键能力，如处理空间变换输入信号的鲁棒性。为减轻这种严重鲁棒性问题，我们仅有两个途径：通过增加样本多样性（数据增强）对模型进行隐式正则化，或者通过硬编码的归纳偏见对模型进行显式约束。前者的限制因素是数据空间的大小，使得足够的样本覆盖变得无法实现。后者则取决于为每个可能场景开发这种归纳偏见的工程努力。相反，我们从人类行为中获得了灵感，其中推理过程中会通过心理或物理动作对感知进行修改。我们提出了一种新颖的技术，用于模拟这种推理过程。这是通过在推理过程中遍历稀疏反变换树实现的，并使用并行能量为基础的评估进行评估。我们提出的推理算法称为反变换搜索（ITS）。与模型无关，它为模型提供了零散的逆变换搜索（ITS）伪invariance，以处理空间变换的输入。我们在多个基准数据集上评估了我们的方法，包括由合成的ImageNet测试集。ITS在所有零散测试场景上都优于使用的基线。
URL
https://arxiv.org/abs/2405.03730
