Abstract
While visual reasoning for simple analogies has received significant attention, compositional visual relations (CVR) remain relatively unexplored due to their greater complexity. To solve CVR tasks, we propose Predictive Reasoning with Augmented Anomaly Contrastive Learning (PR-A$^2$CL), \ie, to identify an outlier image given three other images that follow the same compositional rules. To address the challenge of modelling abundant compositional rules, an Augmented Anomaly Contrastive Learning is designed to distil discriminative and generalizable features by maximizing similarity among normal instances while minimizing similarity between normal and anomalous outliers. More importantly, a predict-and-verify paradigm is introduced for rule-based reasoning, in which a series of Predictive Anomaly Reasoning Blocks (PARBs) iteratively leverage features from three out of the four images to predict those of the remaining one. Throughout the subsequent verification stage, the PARBs progressively pinpoint the specific discrepancies attributable to the underlying rules. Experimental results on SVRT, CVR and MC$^2$R datasets show that PR-A$^2$CL significantly outperforms state-of-the-art reasoning models.
Abstract (translated)
尽管简单的类比视觉推理已经得到了广泛关注，但组合视觉关系（CVR）由于其更大的复杂性而相对较少被探索。为了解决CVR任务，我们提出了预测增强异常对比学习的预测推理(PR-A²CL)，即在给定三个遵循相同组合规则的图像的情况下识别一个离群图像。为了应对建模丰富组合规则的挑战，设计了增强异常对比学习以通过最大化正常实例之间的相似性同时最小化正常实例与异常离群之间的相似性来提取区分性和泛化的特征。更重要的是，引入了一种基于规则推理的预测和验证范式，在此过程中一系列预测异常推理模块(PARBs)迭代地利用四个图像中的三个图像的特征来预测剩余一个图像的特征。在随后的验证阶段，PARBs逐步确定归因于底层规则的具体差异。 实验结果表明，在SVRT、CVR 和 MC²R 数据集上，PR-A²CL 显著优于最先进的推理模型。
URL
https://arxiv.org/abs/2603.01125
