Abstract
Recent optical flow estimation methods often employ local cost sampling from a dense all-pairs correlation volume. This results in quadratic computational and memory complexity in the number of pixels. Although an alternative memory-efficient implementation with on-demand cost computation exists, this is slower in practice and therefore prior methods typically process images at reduced resolutions, missing fine-grained details. To address this, we propose a more efficient implementation of the all-pairs correlation volume sampling, still matching the exact mathematical operator as defined by RAFT. Our approach outperforms on-demand sampling by up to 90% while maintaining low memory usage, and performs on par with the default implementation with up to 95% lower memory usage. As cost sampling makes up a significant portion of the overall runtime, this can translate to up to 50% savings for the total end-to-end model inference in memory-constrained environments. Our evaluation of existing methods includes an 8K ultra-high-resolution dataset and an additional inference-time modification of the recent SEA-RAFT method. With this, we achieve state-of-the-art results at high resolutions both in accuracy and efficiency.
Abstract (translated)
最近的光流估计方法通常采用从密集的所有成对相关体积中抽取局部成本的方法。这导致计算和内存复杂度随像素数量呈二次增长。虽然存在一种记忆高效的实现方式，可以在需要时进行成本计算，但这种方法在实践中速度较慢，因此以前的方法通常会以降低图像分辨率的方式来处理图像，从而忽略了细粒度的细节。为了解决这个问题，我们提出了一种更高效的全部成对相关体积采样实现方法，在保持与RAFT定义的确切数学操作一致的同时，比按需成本计算方式快最多90%，同时保持低内存使用率，并且相比默认实现方式内存消耗减少了高达95%。由于成本采样占整个运行时间的重要部分，这可以在内存受限的环境中将总体端到端模型推理节省多达50%的内存。 我们对现有方法的评估包括一个8K超高分辨率的数据集以及最近SEA-RAFT方法在推断时的一个额外修改版本。通过这种方法，在高分辨率下，我们在准确性和效率方面均达到了最先进的成果。
URL
https://arxiv.org/abs/2505.16942
