Abstract
The task of reassembly is a significant challenge across multiple domains, including archaeology, genomics, and molecular docking, requiring the precise placement and orientation of elements to reconstruct an original structure. In this work, we address key limitations in state-of-the-art Deep Learning methods for reassembly, namely i) scalability; ii) multimodality; and iii) real-world applicability: beyond square or simple geometric shapes, realistic and complex erosion, or other real-world problems. We propose ReassembleNet, a method that reduces complexity by representing each input piece as a set of contour keypoints and learning to select the most informative ones by Graph Neural Networks pooling inspired techniques. ReassembleNet effectively lowers computational complexity while enabling the integration of features from multiple modalities, including both geometric and texture data. Further enhanced through pretraining on a semi-synthetic dataset. We then apply diffusion-based pose estimation to recover the original structure. We improve on prior methods by 55% and 86% for RMSE Rotation and Translation, respectively.
Abstract (translated)
重组任务在考古学、基因组学和分子对接等多个领域都面临着重大挑战，需要精确地定位和定向各个元素以重建原始结构。在这项工作中，我们解决了最先进的深度学习方法在重组任务中的三个关键限制：i) 可扩展性；ii) 多模态处理能力；iii) 现实世界适用性（即超出正方形或其他简单几何形状的范围，包括现实世界的复杂侵蚀问题等）。为此，我们提出了一种名为ReassembleNet的方法。该方法通过将每个输入块表示为一组轮廓关键点，并利用图神经网络池化技术来学习选择最具信息量的关键点，从而降低了任务的复杂性。这种方法在减少计算复杂度的同时，还支持多模态特征（包括几何和纹理数据）的集成。通过半合成数据集上的预训练进一步增强ReassembleNet的能力后，我们接着应用基于扩散的方法进行姿态估计以恢复原始结构。我们的方法分别使旋转和平移方向上均方根误差（RMSE）相对于先前的方法提高了55% 和86%。
URL
https://arxiv.org/abs/2505.21117
