Abstract
Vision Transformers (ViTs) and Convolutional Neural Networks (CNNs) face inherent challenges in image matting, particularly in preserving fine structural details. ViTs, with their global receptive field enabled by the self-attention mechanism, often lose local details such as hair strands. Conversely, CNNs, constrained by their local receptive field, rely on deeper layers to approximate global context but struggle to retain fine structures at greater depths. To overcome these limitations, we propose a novel Morpho-Aware Global Attention (MAGA) mechanism, designed to effectively capture the morphology of fine structures. MAGA employs Tetris-like convolutional patterns to align the local shapes of fine structures, ensuring optimal local correspondence while maintaining sensitivity to morphological details. The extracted local morphology information is used as query embeddings, which are projected onto global key embeddings to emphasize local details in a broader context. Subsequently, by projecting onto value embeddings, MAGA seamlessly integrates these emphasized morphological details into a unified global structure. This approach enables MAGA to simultaneously focus on local morphology and unify these details into a coherent whole, effectively preserving fine structures. Extensive experiments show that our MAGA-based ViT achieves significant performance gains, outperforming state-of-the-art methods across two benchmarks with average improvements of 4.3% in SAD and 39.5% in MSE.
Abstract (translated)
视觉变压器（ViTs）和卷积神经网络（CNNs）在图像抠图时会面临固有的挑战，特别是在保留细微结构细节方面。由于自注意力机制赋予的全局感受野，ViTs往往丢失如发丝这样的局部细节。相反，受限于其局部感受野的CNNs依赖更深的层来近似全局上下文，但在更深层次上难以保持细小结构。为了克服这些限制，我们提出了一种新的形态感知全局注意力（MAGA）机制，旨在有效捕捉细微结构的形态。MAGA采用像俄罗斯方块一样的卷积模式对齐局部精细结构的形状，确保最佳的局部对应关系同时保持对形态细节的敏感性。提取出的局部形态信息用作查询嵌入，并投影到全局键嵌入上以强调更广泛上下文中的局部细节。随后，通过投射到值嵌入上，MAGA无缝地将这些被强调的形态细节整合进统一的全局结构中。这一方法使MAGA能够同时关注局部形态并将这些细节融入一个连贯的整体之中，有效地保留了细小结构。广泛的实验表明，我们的基于MAGA的ViT实现了显著的性能提升，在两个基准测试中的平均改进分别为SAD指标4.3%和MSE指标39.5%，超越了现有最先进方法的表现。
URL
https://arxiv.org/abs/2411.10251
