Abstract
The primary challenge of video streaming is to balance high video quality with smooth playback. Traditional codecs are well tuned for this trade-off, yet their inability to use context means they must encode the entire video data and transmit it to the client. This paper introduces ELVIS (End-to-end Learning-based VIdeo Streaming Enhancement Pipeline), an end-to-end architecture that combines server-side encoding optimizations with client-side generative in-painting to remove and reconstruct redundant video data. Its modular design allows ELVIS to integrate different codecs, inpainting models, and quality metrics, making it adaptable to future innovations. Our results show that current technologies achieve improvements of up to 11 VMAF points over baseline benchmarks, though challenges remain for real-time applications due to computational demands. ELVIS represents a foundational step toward incorporating generative AI into video streaming pipelines, enabling higher quality experiences without increased bandwidth requirements.
Abstract (translated)
视频流传输的主要挑战在于平衡高质量的视频与流畅播放之间的关系。传统编解码器在这一权衡中表现良好，但由于它们无法利用上下文信息，必须对整个视频数据进行编码并将其传送到客户端。本文介绍了 ELVIS（基于端到端学习的视频流增强管道），这是一种结合了服务器端编码优化和客户端生成式修复技术来移除和重建冗余视频数据的端到端架构。ELVIS 的模块化设计使其能够整合不同的编解码器、修补模型以及质量指标，从而适应未来的创新需求。我们的结果表明，当前的技术相比基准测试可提升高达 11 个 VMAF 分数（一种视频质量评估标准），尽管实时应用仍面临计算资源的挑战。ELVIS 标志着将生成式人工智能融入视频流传输管道的一个基础性步骤，能够在不增加带宽需求的情况下提供更高质量的观看体验。
URL
https://arxiv.org/abs/2512.14185
