Abstract
In recent years, remarkable advancements have been achieved in the field of image generation, primarily driven by the escalating demand for high-quality outcomes across various image generation subtasks, such as inpainting, denoising, and super resolution. A major effort is devoted to exploring the application of super-resolution techniques to enhance the quality of low-resolution images. In this context, our method explores in depth the problem of ship image super resolution, which is crucial for coastal and port surveillance. We investigate the opportunity given by the growing interest in text-to-image diffusion models, taking advantage of the prior knowledge that such foundation models have already learned. In particular, we present a diffusion-model-based architecture that leverages text conditioning during training while being class-aware, to best preserve the crucial details of the ships during the generation of the super-resoluted image. Since the specificity of this task and the scarcity availability of off-the-shelf data, we also introduce a large labeled ship dataset scraped from online ship images, mostly from ShipSpotting\footnote{\url{this http URL}} website. Our method achieves more robust results than other deep learning models previously employed for super resolution, as proven by the multiple experiments performed. Moreover, we investigate how this model can benefit downstream tasks, such as classification and object detection, thus emphasizing practical implementation in a real-world scenario. Experimental results show flexibility, reliability, and impressive performance of the proposed framework over state-of-the-art methods for different tasks. The code is available at: this https URL .
Abstract (translated)
近年来，在图像生成领域取得了显著的进步，主要受到高质量图像成果需求的不断增长推动，尤其是在修复、去噪和超分辨率等图像生成子任务方面。大量精力致力于探讨将超分辨率技术应用于增强低分辨率图像质量。在这种情况下，我们的方法深入研究了船舶图像超分辨率问题，这对沿海和港口监视至关重要。我们研究了对于文本到图像扩散模型的增长兴趣，利用其已经获得的知识。特别是，我们提出了一个基于扩散模型的架构，在训练过程中利用文本条件，以保留超分辨率图像中船舶的关键细节。由于这项任务的独特性和可用数据的稀缺性，我们还引入了一个从在线图像网站如ShipSpotting网站收集的大规模标注船舶数据集。我们的方法在超分辨率模型的应用中实现了比之前使用的更稳健的结果，这是通过多次实验证明的。此外，我们还研究了这种模型如何为下游任务（如分类和目标检测）带来好处，从而强调在现实场景中的实际实现。实验结果表明，该框架具有灵活性、可靠性和令人印象深刻的性能，超过目前最先进的方法。代码可在此处下载：https://this URL 。
URL
https://arxiv.org/abs/2403.18370
