Abstract
We present SPRITETOMESH, a fully automatic pipeline for converting 2D game sprite images into triangle meshes compatible with skeletal animation frameworks such as Spine2D. Creating animation-ready meshes is traditionally a tedious manual process requiring artists to carefully place vertices along visual boundaries, a task that typically takes 15-60 minutes per sprite. Our method addresses this through a hybrid learned-algorithmic approach. A segmentation network (EfficientNet-B0 encoder with U-Net decoder) trained on over 100,000 sprite-mask pairs from 172 games achieves an IoU of 0.87, providing accurate binary masks from arbitrary input images. From these masks, we extract exterior contour vertices using Douglas-Peucker simplification with adaptive arc subdivision, and interior vertices along visual boundaries detected via bilateral-filtered multi-channel Canny edge detection with contour-following placement. Delaunay triangulation with mask-based centroid filtering produces the final mesh. Through controlled experiments, we demonstrate that direct vertex position prediction via neural network heatmap regression is fundamentally not viable for this task: the heatmap decoder consistently fails to converge (loss plateau at 0.061) while the segmentation decoder trains normally under identical conditions. We attribute this to the inherently artistic nature of vertex placement - the same sprite can be meshed validly in many different ways. This negative result validates our hybrid design: learned segmentation where ground truth is unambiguous, algorithmic placement where domain heuristics are appropriate. The complete pipeline processes a sprite in under 3 seconds, representing a speedup of 300x-1200x over manual creation. We release our trained model to the game development community.
Abstract (translated)
我们介绍了SPRITETOMESH，这是一种全自动流程，用于将2D游戏精灵图像转换为与Spine2D等骨骼动画框架兼容的三角网格。传统的做法是手动创建适合动画制作的网格，这个过程既耗时又繁琐，需要艺术家仔细地沿着视觉边界放置顶点，通常每个精灵花费15到60分钟的时间。我们的方法通过混合学习算法的方法解决了这个问题。 该流程包括一个经过训练的分割网络（使用EfficientNet-B0编码器和U-Net解码器），它在一个包含超过10万对游戏遮罩图像的数据库上进行训练，涉及来自172个不同游戏的数据集，获得了0.87的交并比（IoU），能够从任意输入图像中生成准确的二值掩膜。基于这些掩膜，我们利用道格拉斯-普克简化算法结合自适应弧细分来提取外部轮廓顶点，并通过双边滤波多通道Canny边缘检测与跟踪放置方法沿着视觉边界找到内部顶点。最后，在使用遮罩中心点过滤的基础上进行Delaunay三角剖分生成最终网格。 在受控实验中，我们证明了直接通过神经网络热图回归预测顶点位置对于此任务是不可行的：在相同的条件下，热图解码器始终无法收敛（损失函数停滞在0.061），而分割解码器正常训练。我们认为这源于顶点放置本质上是一种艺术性的过程——同一个精灵可以通过多种不同的方式被合理地网格化。这一结果验证了我们混合设计的有效性：对于地面实况明确的情况下采用学习分割方法，而对于领域启发式适用的场景则使用算法放置的方法。 整个流程可以在不到3秒内处理完一个精灵图像，相比于手动创建而言速度提高了300到1200倍。我们向游戏开发社区开放了训练好的模型。
URL
https://arxiv.org/abs/2602.21153
