Abstract
Domain Generalized Video Semantic Segmentation (DGVSS) is trained on a single labeled driving domain and is directly deployed on unseen domains without target labels and test-time adaptation while maintaining temporally consistent predictions over video streams. In practice, both domain shift and temporal-sampling shift break correspondence-based propagation and fixed-stride temporal aggregation, causing severe frame-to-frame flicker even in label-stable regions. We propose Time2General, a DGVSS framework built on Stability Queries. Time2General introduces a Spatio-Temporal Memory Decoder that aggregates multi-frame context into a clip-level spatio-temporal memory and decodes temporally consistent per-frame masks without explicit correspondence propagation. To further suppress flicker and improve robustness to varying sampling rates, the Masked Temporal Consistency Loss is proposed to regularize temporal prediction discrepancies across different strides, and randomize training strides to expose the model to diverse temporal gaps. Extensive experiments on multiple driving benchmarks show that Time2General achieves a substantial improvement in cross-domain accuracy and temporal stability over prior DGSS and VSS baselines while running at up to 18 FPS. Code will be released after the review process.
Abstract (translated)
领域泛化的视频语义分割（Domain Generalized Video Semantic Segmentation, DGVSS）是在单一标注的驾驶域上训练，并直接在无目标标签和测试时间适应的未知域上部署，同时保持视频流中一致的时间预测。实践中，无论是领域的转变还是采样率的变化都会打破基于对应传播和固定步幅的时间聚合，即使在标签稳定的区域也会导致严重的帧间闪烁问题。 我们提出了Time2General框架，它建立在稳定性查询的基础上，并针对DGVSS设计了一个时空记忆解码器（Spatio-Temporal Memory Decoder），该解码器可以将多帧上下文信息聚集为一段视频级别的时空内存，并无需显式的对应传播来解码出一致的时间预测的每帧掩模。为了进一步抑制闪烁并提高对不同采样率变化的鲁棒性，我们提出了遮罩时间一致性损失（Masked Temporal Consistency Loss），以规范不同时步中的时间预测差异，并随机化训练步骤以使模型接触各种各样的时间间隔。 在多个驾驶基准测试中进行的广泛实验表明，Time2General在跨域精度和时间稳定性上相对于之前的DGSS和VSS基线取得了显著改进，并且可以在高达18 FPS的速度下运行。代码将在审查过程后发布。
URL
https://arxiv.org/abs/2602.09648
