Abstract
Monocular 6-DoF pose estimation plays an important role in multiple spacecraft missions. Most existing pose estimation approaches rely on single images with static keypoint localisation, failing to exploit valuable temporal information inherent to space operations. In this work, we adapt a deep learning framework from human pose estimation to the spacecraft pose estimation domain that integrates motion-aware heatmaps and optical flow to capture motion dynamics. Our approach combines image features from a Vision Transformer (ViT) encoder with motion cues from a pre-trained optical flow model to localise 2D keypoints. Using the estimates, a Perspective-n-Point (PnP) solver recovers 6-DoF poses from known 2D-3D correspondences. We train and evaluate our method on the SPADES-RGB dataset and further assess its generalisation on real and synthetic data from the SPARK-2024 dataset. Overall, our approach demonstrates improved performance over single-image baselines in both 2D keypoint localisation and 6-DoF pose estimation. Furthermore, it shows promising generalisation capabilities when testing on different data distributions.
Abstract (translated)
单目六自由度姿态估计在多航天器任务中扮演着重要角色。大多数现有的姿态估计算法依赖于仅包含静态关键点定位的单一图像，未能充分利用空间操作固有的宝贵时间信息。在这项工作中，我们将一个源自人体姿态估计算法的深度学习框架进行了改进，并将其应用于航天器的姿态估计领域，该框架集成了运动感知热图和光学流以捕捉运动动力学。我们的方法结合了来自视觉变压器（ViT）编码器的图像特征与预训练光学流模型提供的运动线索来定位2D关键点。利用这些估算结果，透视-n-点(PnP)求解器可以从已知的二维三维对应关系中恢复出六自由度姿态。我们在SPADES-RGB数据集上对我们的方法进行了训练和评估，并进一步在来自SPARK-2024的数据集中真实和合成数据上的泛化能力进行了测试。总体而言，与单图像基线相比，我们提出的方法在二维关键点定位和六自由度姿态估计方面都表现出更优的性能；同时，它还展示了在不同数据分布下的良好泛化能力。
URL
https://arxiv.org/abs/2509.06000
