Abstract
Large image generation and vision models, combined with differentiable rendering technologies, have become powerful tools for generating paths that can be drawn or painted by a robot. However, these tools often overlook the intrinsic physicality of the human drawing/writing act, which is usually executed with skillful hand/arm gestures. Taking this into account is important for the visual aesthetics of the results and for the development of closer and more intuitive artist-robot collaboration scenarios. We present a method that bridges this gap by enabling gradient-based optimization of natural human-like motions guided by cost functions defined in image space. To this end, we use the sigma-lognormal model of human hand/arm movements, with an adaptation that enables its use in conjunction with a differentiable vector graphics (DiffVG) renderer. We demonstrate how this pipeline can be used to generate feasible trajectories for a robot by combining image-driven objectives with a minimum-time smoothing criterion. We demonstrate applications with generation and robotic reproduction of synthetic graffiti as well as image abstraction.
Abstract (translated)
大型图像生成和视觉模型，结合可微渲染技术，已经成为生成机器人可以绘制或绘画路径的强大工具。然而，这些工具往往忽略了人类绘图/书写行为内在的物理特性，这种行为通常通过熟练的手臂手势来执行。考虑到这一点对于结果的视觉美学以及更接近、更具直观性的艺术家与机器人的协作场景的发展非常重要。我们提出了一种方法，该方法通过在图像空间中定义成本函数的基础上进行基于梯度的优化，使得自然的人类手/臂动作得以实现，从而弥合了这一差距。为此，我们使用了人类手/臂运动的sigma-对数正态模型，并做了一些适应性修改，使其能够与可微向量图形（DiffVG）渲染器结合使用。我们演示了如何通过将图像驱动的目标与最小时间平滑标准相结合来利用这条管道生成机器人的可行轨迹。我们展示了合成涂鸦的生成和机器人复制以及图像抽象的应用实例。
URL
https://arxiv.org/abs/2507.03166
