Abstract
Event cameras capture per-pixel brightness changes with microsecond resolution, offering continuous motion information lost between RGB frames. However, existing event-based motion estimators depend on large-scale synthetic data that often suffers from a significant sim-to-real gap. We propose TETO (Tracking Events with Teacher Observation), a teacher-student framework that learns event motion estimation from only $\sim$25 minutes of unannotated real-world recordings through knowledge distillation from a pretrained RGB tracker. Our motion-aware data curation and query sampling strategy maximizes learning from limited data by disentangling object motion from dominant ego-motion. The resulting estimator jointly predicts point trajectories and dense optical flow, which we leverage as explicit motion priors to condition a pretrained video diffusion transformer for frame interpolation. We achieve state-of-the-art point tracking on EVIMO2 and optical flow on DSEC using orders of magnitude less training data, and demonstrate that accurate motion estimation translates directly to superior frame interpolation quality on BS-ERGB and HQ-EVFI.
Abstract (translated)
事件相机以微秒级分辨率捕捉像素级亮度变化，提供RGB帧间丢失的连续运动信息。然而，现有基于事件的运动估计器依赖于大规模合成数据，常受显著的仿真到现实差距影响。我们提出TETO（通过教师观测追踪事件），一个师生框架，仅通过从预训练RGB追踪器进行知识蒸馏，就能从约25分钟未标注的真实世界记录中学习事件运动估计。我们提出的运动感知数据整理与查询采样策略，通过将物体运动与主导的自我运动解耦，最大化有限数据的学习效果。所得估计器联合预测点轨迹和密集光流，我们将其作为显式运动先验，用于调节预训练的视频扩散Transformer以实现帧插值。我们在EVIMO2上实现最先进的点追踪，在DSEC上实现最先进的光流，训练数据量减少数个数量级，并证明准确运动估计可直接提升BS-ERGB和HQ-EVFI上的帧插值质量。
URL
https://arxiv.org/abs/2603.23487
