Abstract
As augmented reality and virtual reality applications gain popularity, image processing for OmniDirectional Images (ODIs) has attracted increasing attention. OmniDirectional Image Super-Resolution (ODISR) is a promising technique for enhancing the visual quality of ODIs. Before performing super-resolution, ODIs are typically projected from a spherical surface onto a plane using EquiRectangular Projection (ERP). This projection introduces latitude-dependent geometric distortion in ERP images: distortion is minimal near the equator but becomes severe toward the poles, where image content is stretched across a wider area. However, existing ODISR methods have limited sampling ranges and feature extraction capabilities, which hinder their ability to capture distorted patterns over large areas. To address this issue, we propose a novel Multi-level Distortion-aware Deformable Network (MDDN) for ODISR, designed to expand the sampling range and receptive field. Specifically, the feature extractor in MDDN comprises three parallel branches: a deformable attention mechanism (serving as the dilation=1 path) and two dilated deformable convolutions with dilation rates of 2 and 3. This architecture expands the sampling range to include more distorted patterns across wider areas, generating dense and comprehensive features that effectively capture geometric distortions in ERP images. The representations extracted from these deformable feature extractors are adaptively fused in a multi-level feature fusion module. Furthermore, to reduce computational cost, a low-rank decomposition strategy is applied to dilated deformable convolutions. Extensive experiments on publicly available datasets demonstrate that MDDN outperforms state-of-the-art methods, underscoring its effectiveness and superiority in ODISR.
Abstract (translated)
随着增强现实和虚拟现实应用的普及，全景图像（OmniDirectional Images, ODIs）的图像处理技术越来越受到关注。OmniDirectional Image Super-Resolution (ODISR) 是一种有前景的技术，用于提高 ODIs 的视觉质量。在进行超分辨率处理之前，通常会将 ODIs 从球面投影到平面上，采用的是等矩形投影（EquiRectangular Projection, ERP）。这种投影会在 ERP 图像中引入纬度相关的几何失真：在赤道附近失真较小，而在两极地区则显著增加，导致图像内容在更广泛的区域内拉伸。然而，现有的 ODISR 方法具有有限的采样范围和特征提取能力，这限制了它们捕捉大范围内失真模式的能力。为了解决这一问题，我们提出了一种新颖的多级扭曲感知变形网络（Multi-level Distortion-aware Deformable Network, MDDN），旨在扩大采样范围和感受野。具体而言，MDDN 中的特征提取器包括三个并行分支：一个可变形注意力机制（作为 dilation=1 的路径）以及两个具有扩张率为 2 和 3 的可变形膨胀卷积。这种架构扩展了采样范围，能够覆盖更广泛的失真模式，生成密集且全面的特征，有效捕捉 ERP 图像中的几何扭曲。从这些可变形特征提取器中抽取的表示在多级特征融合模块中自适应地进行融合。此外，为了减少计算成本，在扩张可变形卷积上应用了低秩分解策略。在公共数据集上的广泛实验表明，MDDN 在 ODISR 方面优于最先进的方法，证明了其有效性和优越性。
URL
https://arxiv.org/abs/2512.17343
