Abstract
Human vision organizes local cues into coherent global forms using Gestalt principles like closure, proximity, and figure-ground assignment -- functions reliant on global spatial structure. We investigate whether modern vision models show similar behaviors, and under what training conditions these emerge. We find that Vision Transformers (ViTs) trained with Masked Autoencoding (MAE) exhibit activation patterns consistent with Gestalt laws, including illusory contour completion, convexity preference, and dynamic figure-ground segregation. To probe the computational basis, we hypothesize that modeling global dependencies is necessary for Gestalt-like organization. We introduce the Distorted Spatial Relationship Testbench (DiSRT), which evaluates sensitivity to global spatial perturbations while preserving local textures. Using DiSRT, we show that self-supervised models (e.g., MAE, CLIP) outperform supervised baselines and sometimes even exceed human performance. ConvNeXt models trained with MAE also exhibit Gestalt-compatible representations, suggesting such sensitivity can arise without attention architectures. However, classification finetuning degrades this ability. Inspired by biological vision, we show that a Top-K activation sparsity mechanism can restore global sensitivity. Our findings identify training conditions that promote or suppress Gestalt-like perception and establish DiSRT as a diagnostic for global structure sensitivity across models.
Abstract (translated)
人类视觉通过诸如封闭性、邻近性和正负形分配等格式塔原则，将局部线索组织成连贯的整体形式。这些功能依赖于全局空间结构。我们研究了现代视觉模型是否表现出类似的特性，并探讨了在何种训练条件下它们会出现这种行为。 我们的发现表明，使用掩码自动编码（MAE）进行训练的Vision Transformers (ViTs) 展现出了与格式塔定律一致的激活模式，包括幻觉轮廓完成、凸性偏好和动态正负形分离。为了探究计算基础，我们假设建模全局依赖关系对于形成类似的格式塔组织是必要的。 为此，我们引入了扭曲空间关系测试台（Distorted Spatial Relationship Testbench, DiSRT），它评估模型对全局空间扰动的敏感度，同时保持局部纹理不变。通过使用DiSRT，我们展示了自监督模型（例如MAE、CLIP）在处理全球结构变化时的表现优于监督基准，并且有时甚至超越了人类表现。此外，经过MAE训练的ConvNeXt模型也表现出与格式塔兼容的表示形式，这表明这种敏感度可以在没有注意力架构的情况下产生。 然而，在进行分类微调后，这些能力会退化。受生物视觉启发，我们展示了一种Top-K激活稀疏机制可以恢复全局敏感性。我们的研究确定了促进或抑制类似格式塔感知的训练条件，并将DiSRT确立为跨模型检测全球结构敏感性的诊断工具。 简而言之，这项研究表明，在特定条件下训练的现代计算机视觉模型能够模拟人类视觉中的某些高级认知过程，特别是那些涉及从局部细节中提取全局信息的过程。这不仅加深了我们对这些模型如何工作的理解，还可能对开发更智能、更具适应性的机器学习系统具有重要影响。
URL
https://arxiv.org/abs/2506.00718
