Abstract
Transformer-based models have achieved strong performance in remote sensing image captioning by capturing long-range dependencies and contextual information. However, their practical deployment is hindered by high computational costs, especially in multi-modal frameworks that employ separate transformer-based encoders and decoders. In addition, existing remote sensing image captioning models primarily focus on high-level semantic extraction while often overlooking fine-grained structural features such as edges, contours, and object boundaries. To address these challenges, a lightweight transformer architecture is proposed by reducing the dimensionality of the encoder layers and employing a distilled version of GPT-2 as the decoder. A knowledge distillation strategy is used to transfer knowledge from a more complex teacher model to improve the performance of the lightweight network. Furthermore, an edge-aware enhancement strategy is incorporated to enhance image representation and object boundary understanding, enabling the model to capture fine-grained spatial details in remote sensing images. Experimental results demonstrate that the proposed approach significantly improves caption quality compared to state-of-the-art methods.
Abstract (translated)
基于Transformer的模型通过捕捉长距离依赖和上下文信息，在遥感图像描述生成中取得了优异的成绩。然而，由于计算成本高，尤其是在使用独立的Transformer编码器和解码器的多模态框架中，这些模型的实际部署受到了阻碍。此外，现有的遥感图像描述生成模型主要关注高层次语义提取，而往往忽视了边缘、轮廓和对象边界等细粒度结构特征。 为了解决这些问题，提出了一种轻量级的Transformer架构，通过减少编码器层的维度并使用GPT-2的精简版本作为解码器来实现。采用知识蒸馏策略，将更复杂的教师模型的知识转移到轻量级网络中以提高性能。此外，还引入了边缘感知增强策略，用以提升图像表示和对象边界理解能力，使模型能够捕捉到遥感图像中的细粒度空间细节。 实验结果表明，所提出的方法在图像描述质量方面显著优于现有的先进方法。
URL
https://arxiv.org/abs/2506.09429
