Paper Reading AI Learner

DGME-T: Directional Grid Motion Encoding for Transformer-Based Historical Camera Movement Classification

2025-10-17 15:14:11
Tingyu Lin, Armin Dadras, Florian Kleber, Robert Sablatnig

Abstract

Camera movement classification (CMC) models trained on contemporary, high-quality footage often degrade when applied to archival film, where noise, missing frames, and low contrast obscure motion cues. We bridge this gap by assembling a unified benchmark that consolidates two modern corpora into four canonical classes and restructures the HISTORIAN collection into five balanced categories. Building on this benchmark, we introduce DGME-T, a lightweight extension to the Video Swin Transformer that injects directional grid motion encoding, derived from optical flow, via a learnable and normalised late-fusion layer. DGME-T raises the backbone's top-1 accuracy from 81.78% to 86.14% and its macro F1 from 82.08% to 87.81% on modern clips, while still improving the demanding World-War-II footage from 83.43% to 84.62% accuracy and from 81.72% to 82.63% macro F1. A cross-domain study further shows that an intermediate fine-tuning stage on modern data increases historical performance by more than five percentage points. These results demonstrate that structured motion priors and transformer representations are complementary and that even a small, carefully calibrated motion head can substantially enhance robustness in degraded film analysis. Related resources are available at this https URL.

Abstract (translated)

基于当代高质量视频训练的相机运动分类(CMC)模型在应用于存档影片时通常会出现性能下降,因为噪声、缺失帧和低对比度会遮蔽运动线索。为解决这一问题,我们建立了一个统一基准测试平台,该平台整合了两个现代数据集,并将其重组为四种典型类别,同时将HISTORIAN集合重新组织成五个平衡的类别。基于此基准,我们推出了DGME-T,这是Video Swin Transformer的一个轻量级扩展版本,通过一个可学习和规范化的晚期融合层注入来自光流的方向网格运动编码(DGME)。DGME-T使骨干网络在现代视频片段上的顶级准确率从81.78%提升至86.14%,宏平均F1分数从82.08%提高到87.81%。同时,它还能将对二战时期影片的准确性从83.43%提升至84.62%,宏平均F1分数则从81.72%增加到82.63%。跨域研究进一步显示,在现代数据上进行中间微调阶段可以使历史性能提高超过五个百分点。这些结果表明,结构化的运动先验和变换表示是互补的,并且即使是一个小而精心校准的运动头部也能显著提升在降级影片分析中的鲁棒性。 相关资源可在以下链接获取:[此链接](请将"[此链接]"替换为实际提供的URL)。

URL

https://arxiv.org/abs/2510.15725

PDF

https://arxiv.org/pdf/2510.15725.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot