Abstract
Unsupervised video hashing usually optimizes binary codes by learning to reconstruct input videos. Such reconstruction constraint spends much effort on frame-level temporal context changes without focusing on video-level global semantics that are more useful for retrieval. Hence, we address this problem by decomposing video information into reconstruction-dependent and semantic-dependent information, which disentangles the semantic extraction from reconstruction constraint. Specifically, we first design a simple dual-stream structure, including a temporal layer and a hash layer. Then, with the help of semantic similarity knowledge obtained from self-supervision, the hash layer learns to capture information for semantic retrieval, while the temporal layer learns to capture the information for reconstruction. In this way, the model naturally preserves the disentangled semantics into binary codes. Validated by comprehensive experiments, our method consistently outperforms the state-of-the-arts on three video benchmarks.
Abstract (translated)
无监督视频哈希通常通过学习重构输入视频来优化二进制码。这样的重构约束在帧级时间上下文变化上花费了很大的精力，而没有重点关注对于检索更有用的视频级别的全局语义。因此，我们通过将视频信息分解为重构依赖于和语义依赖于信息来解决这个问题。具体来说，我们首先设计了一个简单的双流结构，包括一个时间层和一个哈希层。然后，在从自监督中获得语义相似性知识的帮助下，哈希层学会了捕捉语义检索所需的信息，而时间层学会了捕捉重构所需的信息。在这种情况下，模型自然地将二进制码中的分离语义保留在编码中。通过全面的实验验证，我们的方法在三个视频基准测试中始终超越了最先进的水平。
URL
https://arxiv.org/abs/2310.08009
