Abstract
Multispectral Sentinel-2 images are a valuable source of Earth observation data, however spatial resolution of their spectral bands limited to 10 m, 20 m, and 60 m ground sampling distance remains insufficient in many cases. This problem can be addressed with super-resolution, aimed at reconstructing a high-resolution image from a low-resolution observation. For Sentinel-2, spectral information fusion allows for enhancing the 20 m and 60 m bands to the 10 m resolution. Also, there were attempts to combine multitemporal stacks of individual Sentinel-2 bands, however these two approaches have not been combined so far. In this paper, we introduce DeepSent -- a new deep network for super-resolving multitemporal series of multispectral Sentinel-2 images. It is underpinned with information fusion performed simultaneously in the spectral and temporal dimensions to generate an enlarged multispectral image. In our extensive experimental study, we demonstrate that our solution outperforms other state-of-the-art techniques that realize either multitemporal or multispectral data fusion. Furthermore, we show that the advantage of DeepSent results from how these two fusion types are combined in a single architecture, which is superior to performing such fusion in a sequential manner. Importantly, we have applied our method to super-resolve real-world Sentinel-2 images, enhancing the spatial resolution of all the spectral bands to 3.3 m nominal ground sampling distance, and we compare the outcome with very high-resolution WorldView-2 images. We will publish our implementation upon paper acceptance, and we expect it will increase the possibilities of exploiting super-resolved Sentinel-2 images in real-life applications.
Abstract (translated)
彩色Sentinel-2图像是一个重要的地球观测数据源,然而它们的光谱 Band 的空间分辨率只能达到10 m、20 m和60 m的地面采样距离,在许多情况下仍然不足以满足要求。解决这个问题可以通过超分辨率来解决,旨在从低分辨率观察中提取高分辨率图像。对于Sentinel-2,光谱信息融合可以增强20 m和60 mBand的空间分辨率到10 m。此外,也尝试过将单个Sentinel-2Band的多个时间序列 stack 合并起来,然而目前这两种方法还没有被合并。在本文中,我们介绍了 DeepSent - 一个用于超分辨率解决多光谱Sentinel-2图像的时间序列系列。它通过在光谱和时间维度上的信息融合生成放大的多光谱图像的基础。在我们的广泛的实验研究中,我们证明了我们的解决方案比实现多时间或多光谱数据融合的现有先进技术更好。此外,我们表明 DeepSent 的优势在于这两个融合类型的组合方式,这种方法比Sequentially 进行这种融合更好。重要的是,我们已将我们的方法应用于超分辨率处理真实的Sentinel-2图像,将所有光谱 Band 的空间分辨率提高到3.3 m的正式地面采样距离,并将结果与非常高质量的WorldView-2图像进行比较。我们将在论文接受后发布我们的实现,我们期望这将增加利用超分辨率处理真实的Sentinel-2图像在实际应用中的机会。
URL
https://arxiv.org/abs/2301.11154
