Abstract
Video quality significantly affects video classification. We found this problem when we classified Mild Cognitive Impairment well from clear videos, but worse from blurred ones. From then, we realized that referring to Video Quality Assessment (VQA) may improve video classification. This paper proposed Self-Supervised Learning-based Video Vision Transformer combined with No-reference VQA for video classification (SSL-V3) to fulfill the goal. SSL-V3 leverages Combined-SSL mechanism to join VQA into video classification and address the label shortage of VQA, which commonly occurs in video datasets, making it impossible to provide an accurate Video Quality Score. In brief, Combined-SSL takes video quality score as a factor to directly tune the feature map of the video classification. Then, the score, as an intersected point, links VQA and classification, using the supervised classification task to tune the parameters of VQA. SSL-V3 achieved robust experimental results on two datasets. For example, it reached an accuracy of 94.87% on some interview videos in the I-CONECT (a facial video-involved healthcare dataset), verifying SSL-V3's effectiveness.
Abstract (translated)
视频质量显著影响视频分类的效果。我们在对轻度认知障碍（Mild Cognitive Impairment）进行分类时发现，从清晰的视频中可以较好地识别，但从模糊的视频中则表现较差。因此我们意识到借鉴视频质量评估（VQA）的方法可能有助于改进视频分类任务。本文提出了一种基于自监督学习的视频视觉变换器结合无参考VQA的视频分类方法（SSL-V3），以实现这一目标。 SSL-V3利用了Combined-SSL机制，将VQA与视频分类结合起来，并解决了VQA中常见的标签短缺问题，在大多数视频数据集中难以提供准确的视频质量分数。简而言之，Combined-SSL通过将视频质量评分作为一个因素直接调整视频分类模型中的特征图来实现这一点。同时，该评分作为交点位置，连接了VQA和分类任务，使用监督学习下的分类任务来调整VQA参数。 在两个数据集上，SSL-V3取得了稳健的实验结果。例如，在I-CONECT（一个涉及面部视频的医疗健康数据集）中的某些访谈视频上，其准确率达到了94.87%，证明了SSL-V3的有效性。
URL
https://arxiv.org/abs/2603.10965
