Abstract
Video processing is generally divided into two main categories: processing of the entire video, which typically yields optimal classification outcomes, and real-time processing, where the objective is to make a decision as promptly as possible. Although the models dedicated to the processing of entire videos are typically well-defined and clearly presented in the literature, this is not the case for online processing, where a~plethora of hand-devised methods exist. To address this issue, we present PrAViC, a novel, unified, and theoretically-based adaptation framework for tackling the online classification problem in video data. The initial phase of our study is to establish a mathematical background for the classification of sequential data, with the potential to make a decision at an early stage. This allows us to construct a natural function that encourages the model to return a result much faster. The subsequent phase is to present a straightforward and readily implementable method for adapting offline models to the online setting using recurrent operations. Finally, PrAViC is evaluated by comparing it with existing state-of-the-art offline and online models and datasets. This enables the network to significantly reduce the time required to reach classification decisions while maintaining, or even enhancing, accuracy.
Abstract (translated)
视频处理通常分为两大类：一类是对整个视频的处理，这类方法通常能获得最佳分类效果；另一类是实时处理，在这种情况下目标是在尽可能短的时间内作出决策。尽管针对整个视频处理的研究在文献中已有明确和完善的定义，但对于在线处理的情况则存在许多手工设计的方法，并且没有统一的标准。为了解决这一问题，我们提出了一种新的、基于理论的适应框架PrAViC，旨在解决视频数据中的实时分类难题。 我们的研究首先建立了一个数学基础，用于对序列数据进行早期决策分类。这使得我们可以构造一个自然函数来鼓励模型以更快的速度返回结果。接下来，我们将展示一种简单且易于实现的方法，通过使用递归操作将离线模型适应到在线环境中。最后，我们通过与现有的最先进的离线和在线模型以及数据集进行比较来评估PrAViC的表现。 这种方法使得网络能够在保持或甚至提高分类准确性的同时显著减少做出决策所需的时间。
URL
https://arxiv.org/abs/2406.11443
