Abstract
Enabling robots to execute novel manipulation tasks zero-shot is a central goal in robotics. Most existing methods assume in-distribution tasks or rely on fine-tuning with embodiment-matched data, limiting transfer across platforms. We present NovaFlow, an autonomous manipulation framework that converts a task description into an actionable plan for a target robot without any demonstrations. Given a task description, NovaFlow synthesizes a video using a video generation model and distills it into 3D actionable object flow using off-the-shelf perception modules. From the object flow, it computes relative poses for rigid objects and realizes them as robot actions via grasp proposals and trajectory optimization. For deformable objects, this flow serves as a tracking objective for model-based planning with a particle-based dynamics model. By decoupling task understanding from low-level control, NovaFlow naturally transfers across embodiments. We validate on rigid, articulated, and deformable object manipulation tasks using a table-top Franka arm and a Spot quadrupedal mobile robot, and achieve effective zero-shot execution without demonstrations or embodiment-specific training. Project website: this https URL.
Abstract (translated)
使机器人能够在没有演示的情况下执行新的操作任务是机器人学的一个核心目标。目前大多数现有方法要么假设任务在数据分布内，要么依赖于与实体匹配的数据进行微调，这限制了不同平台之间的迁移能力。我们介绍了NovaFlow，这是一个自主操作框架，它能够将任务描述转换为针对目标机器人的可执行计划，而无需任何演示数据。 给定一个任务描述，NovaFlow 使用视频生成模型合成一段视频，并利用现成的感知模块将其转化为3D可操作对象流（object flow）。根据该对象流，NovaFlow 计算刚性物体的相对姿态，并通过抓取提议和轨迹优化将这些姿态实现为机器人动作。对于柔性物体，这种流动作为基于粒子的动力学模型的模型化规划中的跟踪目标。 通过将任务理解与低级控制解耦，NovaFlow 自然地实现了跨实体的迁移能力。我们在桌面Franka机械臂和Spot四足移动机器人的刚性、关节式和柔性物体操作任务上进行了验证，并在没有演示数据或特定于实体的训练的情况下实现了有效的零样本执行。 项目网站：[请参阅原文链接获取具体网址]
URL
https://arxiv.org/abs/2510.08568
