Abstract
We develop an automated video colorization framework that minimizes the flickering of colors across frames. If we apply image colorization techniques to successive frames of a video, they treat each frame as a separate colorization task. Thus, they do not necessarily maintain the colors of a scene consistently across subsequent frames. The proposed solution includes a novel deep recurrent encoder-decoder architecture which is capable of maintaining temporal and contextual coherence between consecutive frames of a video. We use a high-level semantic feature extractor to automatically identify the context of a scenario including objects, with a custom fusion layer that combines the spatial and temporal features of a frame sequence. We demonstrate experimental results, qualitatively showing that recurrent neural networks can be successfully used to improve color consistency in video colorization.
Abstract (translated)
我们开发了一个自动化的视频色彩校正框架，以减少帧之间的色彩闪烁。如果我们将图像色彩校正技术应用于视频的相邻帧，它们将每个帧视为独立的色彩校正任务。因此，它们不一定保持场景的颜色在后续的帧中保持一致。 proposed solution 包括一种新颖的深度循环编码-解码架构，它能够在视频的连续帧之间保持时间和实践一致性。我们使用高级别的语义特征提取器自动识别包括物体的场景上下文，并使用自定义融合层将帧序列的空间和时间特征进行组合。我们展示了实验结果，定性地证明循环神经网络可以成功地用于改善视频色彩校正中的颜色一致性。
URL
https://arxiv.org/abs/2305.13704
