Abstract
Slot attention is a powerful method for object-centric modeling in images and videos. However, its set-equivariance limits its ability to handle videos with a dynamic number of objects because it cannot break ties. To overcome this limitation, we first establish a connection between slot attention and optimal transport. Based on this new perspective we propose MESH (Minimize Entropy of Sinkhorn): a cross-attention module that combines the tiebreaking properties of unregularized optimal transport with the speed of regularized optimal transport. We evaluate slot attention using MESH on multiple object-centric learning benchmarks and find significant improvements over slot attention in every setting.
Abstract (translated)
空注意力是图像和视频中对象中心建模的强大方法。然而,其集齐性限制住了其处理具有动态数量的对象视频的能力,因为它无法打破关系。为了克服这一限制,我们首先建立了空注意力与最优传输之间的联系。基于这一新的视角,我们提出了Mesh(最小化Sinkhorn熵):一个交叉注意力模块,它将非正则化最优传输的解破键特性与正则化最优传输的速度相结合。我们使用Mesh对多个对象中心学习基准进行评估,并在每个设置中都发现了空注意力相对于空注意力的显著改进。
URL
https://arxiv.org/abs/2301.13197
