Abstract
We propose an optical flow-guided approach for semi-supervised video object segmentation. Optical flow is usually exploited as additional guidance information in unsupervised video object segmentation. However, its relevance in semi-supervised video object segmentation has not been fully explored. In this work, we follow an encoder-decoder approach to address the segmentation task. A model to extract the combined information from optical flow and the image is proposed, which is then used as input to the target model and the decoder network. Unlike previous methods where concatenation is used to integrate information from image data and optical flow, a simple yet effective attention mechanism is exploited in our work. Experiments on DAVIS 2017 and YouTube-VOS 2019 show that by integrating the information extracted from optical flow into the original image branch results in a strong performance gain and our method achieves state-of-the-art performance.
Abstract (translated)
我们提出了一种基于光学流的指导方法,用于半监督视频对象分割。光学流通常在 unsupervised 视频对象分割中作为额外的 guidance 信息而利用。然而,在半监督视频对象分割中,其相关性尚未 fully explored。在本研究中,我们采用编码-解码方法来解决分割任务。我们提出了一种方法,从光学流和图像中分别提取联合信息,并将其用作目标模型和解码网络的输入。与以前的方法和 concatenate 用于将图像数据和光学流信息整合起来不同,我们在研究中采用了一种简单但有效的注意力机制。在 Davis 2017 和 YouTube-VOS 2019 的实验中,结果表明,将从光学流中提取的信息整合到原始图像分支中可以获得显著的性能提升,我们的方法和方法达到了当前技术水平。
URL
https://arxiv.org/abs/2301.10492
