Abstract
We introduce $\infty$-THOR, a new framework for long-horizon embodied tasks that advances long-context understanding in embodied AI. $\infty$-THOR provides: (1) a generation framework for synthesizing scalable, reproducible, and unlimited long-horizon trajectories; (2) a novel embodied QA task, Needle(s) in the Embodied Haystack, where multiple scattered clues across extended trajectories test agents' long-context reasoning ability; and (3) a long-horizon dataset and benchmark suite featuring complex tasks that span hundreds of environment steps, each paired with ground-truth action sequences. To enable this capability, we explore architectural adaptations, including interleaved Goal-State-Action modeling, context extension techniques, and Context Parallelism, to equip LLM-based agents for extreme long-context reasoning and interaction. Experimental results and analyses highlight the challenges posed by our benchmark and provide insights into training strategies and model behaviors under long-horizon conditions. Our work provides a foundation for the next generation of embodied AI systems capable of robust, long-term reasoning and planning.
Abstract (translated)
我们介绍了$\infty$-THOR，这是一个新的框架，旨在处理具身任务中的长时间跨度问题，并在具身人工智能中推进长上下文理解。$\infty$-THOR提供了以下内容： 1. 一个生成框架，用于合成可扩展、可重复且无限的长时间跨度轨迹； 2. 一个新的具身问答任务，“针在具身干草堆里”，其中遍布于延长轨迹中的多个散落线索测试代理的长上下文推理能力； 3. 一套包含复杂任务的长时间跨度数据集和基准套件，每个任务跨越数百个环境步骤，并配以真实动作序列。 为了实现这一功能，我们探索了架构调整，包括交错的目标-状态-行动建模、上下文扩展技术以及上下文并行性，以便为基于大语言模型（LLM）的代理提供极端长上下文推理和交互的能力。实验结果和分析突显了我们的基准带来的挑战，并提供了关于长时间跨度条件下训练策略及模型行为的见解。 我们这项工作为下一代能够进行稳健、长期推理与规划的具身人工智能系统奠定了基础。
URL
https://arxiv.org/abs/2505.16928
