Abstract
Modern AI systems, especially those interacting with the physical world, increasingly require real-time performance. However, the high latency of state-of-the-art generalist models, including recent vision-language action models (VLAs), poses a significant challenge. While action chunking has enabled temporal consistency in high-frequency control tasks, it does not fully address the latency problem, leading to pauses or out-of-distribution jerky movements at chunk boundaries. This paper presents a novel inference-time algorithm that enables smooth asynchronous execution of action chunking policies. Our method, real-time chunking (RTC), is applicable to any diffusion- or flow-based VLA out of the box with no re-training. It generates the next action chunk while executing the current one, "freezing" actions guaranteed to execute and "inpainting" the rest. To test RTC, we introduce a new benchmark of 12 highly dynamic tasks in the Kinetix simulator, as well as evaluate 6 challenging real-world bimanual manipulation tasks. Results demonstrate that RTC is fast, performant, and uniquely robust to inference delay, significantly improving task throughput and enabling high success rates in precise tasks $\unicode{x2013}$ such as lighting a match $\unicode{x2013}$ even in the presence of significant latency. See this https URL for videos.
Abstract (translated)
现代人工智能系统，特别是那些与物理世界互动的系统，越来越需要实时性能。然而，包括最近出现的视觉-语言-动作模型（VLAs）在内的最新通用型模型的高延迟带来了重大挑战。虽然动作分块已经使高频控制任务中实现了时间一致性，但它并未完全解决延迟问题，导致在分块边界处会出现暂停或不规则的动作。 本文介绍了一种新颖的推理时算法，该算法能够实现动作分块策略的平滑异步执行。我们的方法称为实时分块（RTC），可以在无需重新训练的情况下应用于任何基于扩散或流的VLA模型。该方法在执行当前动作的同时生成下一个动作片段，“冻结”那些确定要执行的动作，并“填充”剩余部分。 为了测试RTC，我们引入了一个新的基准，在Kinetix模拟器中包含了12项动态任务，以及评估了6项具有挑战性的现实世界双臂操作任务。实验结果表明，RTC速度较快、性能优良且独特地对推理延迟有强健性，显著提高了任务吞吐量，并在精确任务（如点燃火柴）中实现了高成功率——即使存在较大的延迟也能做到这一点。 有关本文介绍的视频，请参阅此链接：[此处应插入实际的URL或文档参考]
URL
https://arxiv.org/abs/2506.07339
