Abstract
Transmission latency significantly affects users' quality of experience in real-time interaction and actuation. As latency is principally inevitable, video prediction can be utilized to mitigate the latency and ultimately enable zero-latency transmission. However, most of the existing video prediction methods are computationally expensive and impractical for real-time applications. In this work, we therefore propose real-time video prediction towards the zero-latency interaction over networks, called IFRVP (Intermediate Feature Refinement Video Prediction). Firstly, we propose three training methods for video prediction that extend frame interpolation models, where we utilize a simple convolution-only frame interpolation network based on IFRNet. Secondly, we introduce ELAN-based residual blocks into the prediction models to improve both inference speed and accuracy. Our evaluations show that our proposed models perform efficiently and achieve the best trade-off between prediction accuracy and computational speed among the existing video prediction methods. A demonstration movie is also provided at this http URL.
Abstract (translated)
传输延迟显著影响用户在实时互动和操作中的体验质量。由于延迟不可避免，可以利用视频预测技术来减少延迟，并最终实现零延迟传输。然而，现有的大多数视频预测方法计算成本高昂，在实时应用中不切实际。为此，我们提出了一种名为 IFRVP（中间特征细化视频预测）的方法，旨在通过网络实现零延迟互动。 首先，我们为视频预测提出了三种训练方法，这些方法基于帧插值模型进行扩展，并使用了一个简单的仅包含卷积的帧插值网络，该网络基于IFRNet。其次，我们在预测模型中引入了基于ELAN（Efficient Lightweight Architecture Network）的残差块，以同时提高推理速度和准确性。 我们的评估表明，所提出的模型在现有的视频预测方法中表现出色，在预测准确性和计算速度之间实现了最佳权衡。此外，我们还提供了一个展示电影，地址为[此处请手动访问提供的URL]。 这个研究的目标是通过有效的视频预测技术来减少网络传输中的延迟问题，并且提高用户实时互动的体验质量。
URL
https://arxiv.org/abs/2503.23185
