Abstract
Generating high-quality 360° panoramic videos from perspective input is one of the crucial applications for virtual reality (VR), whereby high-resolution videos are especially important for immersive experience. Existing methods are constrained by computational limitations of vanilla diffusion models, only supporting $\leq$ 1K resolution native generation and relying on suboptimal post super-resolution to increase resolution. We introduce CubeComposer, a novel spatio-temporal autoregressive diffusion model that natively generates 4K-resolution 360° videos. By decomposing videos into cubemap representations with six faces, CubeComposer autoregressively synthesizes content in a well-planned spatio-temporal order, reducing memory demands while enabling high-resolution output. Specifically, to address challenges in multi-dimensional autoregression, we propose: (1) a spatio-temporal autoregressive strategy that orchestrates 360° video generation across cube faces and time windows for coherent synthesis; (2) a cube face context management mechanism, equipped with a sparse context attention design to improve efficiency; and (3) continuity-aware techniques, including cube-aware positional encoding, padding, and blending to eliminate boundary seams. Extensive experiments on benchmark datasets demonstrate that CubeComposer outperforms state-of-the-art methods in native resolution and visual quality, supporting practical VR application scenarios. Project page: this https URL
Abstract (translated)
从视角输入生成高质量的360°全景视频是虚拟现实（VR）的关键应用之一，而高分辨率视频对于沉浸式体验尤为重要。现有的方法受限于传统扩散模型在计算上的限制，只能支持原生生成不超过1K分辨率的视频，并且依赖于次优的后处理超分辨率技术来提升分辨率。我们介绍了CubeComposer，这是一种新颖的空间-时间自回归扩散模型，能够直接生成4K分辨率的360°视频。通过将视频分解为具有六个面的立方图表示，CubeComposer按照精心规划的空间-时间顺序自回归地合成内容，在减少内存需求的同时支持高分辨率输出。 具体而言，为了应对多维自回归中的挑战，我们提出了以下解决方案： 1. 一种空间-时间自回归策略，通过协调立方体各个面和时间窗口的360°视频生成过程，实现连贯的合成； 2. 一个配备稀疏上下文注意力设计的立方图面上下文管理机制，以提高效率； 3. 连续性感知技术，包括位置编码、填充和平滑处理，以消除边界接缝。 在基准数据集上的广泛实验表明，CubeComposer在原生分辨率和视觉质量方面超越了现有最先进的方法，支持实际的VR应用场景。项目主页：[此链接](this%20https%20URL)
URL
https://arxiv.org/abs/2603.04291
