arXiv_RO
-
Dual-arm Cooperation for Tabletop Wiping with Temporal Object Removal
2026-10-14 03:45:19Pengqi Zhang, Longchang Yu, Daohan Li, Liang Yan, Hangxin LiuAbstract
We devise a dual-arm cooperative planning framework for wiping table surface in the presence of scattered objects. Unlike conventional surface-cleaning strategies that only cover free areas and detour around obstacles, our framework explicitly assigns one arm to rearrange objects while the other performs coverage wiping, thereby enabling access to previously obstructed regions. Specifically, we first construct a set of wiping and object-rearrangement action primitives. Based on these primitives, we introduce Geo-MILP, a geometry-aware task planner formulated as Mixed-Integer Linear Programming (MILP). By integrating Logic-Based Benders Decomposition (LBBD), Geo-MILP iteratively optimizes action sequencingand timing to resolve geometric conflicts, promoting efficient cooperation between wiping and object rearrangement. We then employ K-Order Markov Optimization (KOMO) to generate smooth, collision-free motions to ensure feasible execution. Simulation and physical experiments demonstrate that the proposed framework not only increases reachable coverage by enabling access to obstructed areas, but also significantly improves time efficiency compared to Task and Motion Planning (TAMP) baselines, reducing planning time by up to 9 times and execution makespan by approximately 40%.
Abstract (translated)
URL
https://arxiv.org/abs/2610.19527
PDF
-
Dungeon and Visibility World Generator
Abstract
Robotic exploration, coverage, and multi-agent deployment algorithms require reproducible environments with controllable geometry and visibility. This paper presents the Dungeon and Visibility World Generator, an open-source framework for constructing two- and three-dimensional environments and evaluating geometric line-of-sight. The framework supports orthogonal, polygonal or polyhedral, and curvilinear geometries, with six construction families covering scattered rooms, clusters, winding passages, caverns, branches, and loops. Room dimensions, passage structures, obstacles, and spatial placement can be controlled through configurable parameters and probability distributions. Each generated environment provides continuous free-space geometry and a corresponding discrete occupancy representation. A Rust-based visibility engine supports line-of-sight queries in both representations, enabling the evaluation of observable regions from arbitrary agent positions. The framework records configurations, generated geometries, occupancy grids, and visibility results to support reproducible experiments. We demonstrate the generator across all six geometry classes, examine how geometry and spatial resolution affect visibility, and evaluate generation time and reproducibility. The resulting framework provides a configurable foundation for evaluating robotic exploration, coverage, and multi-agent deployment algorithms in diverse environments.
Abstract (translated)
URL
https://arxiv.org/abs/2610.19523
PDF
-
HiFun: a Hierarchical Framework for Efficient Functional Dexterous Manipulation Learning
2026-10-14 03:37:15Linyi Huang, Guowei Huai, Weibin Liu, Shulong Jiang, Ping Tan, Weixuan Zhang, Hui Zhang, Jie SongAbstract
While multi-fingered dexterous robot hands have the potential to provide human-level dexterity, using them for functional manipulation remains difficult because task success depends on precise contact, coordinated arm-hand motion, and reactive recovery from execution errors. Precise contact depends on complex contact dynamics that are difficult to model accurately in simulation, motivating real-world learning. However, real-world demonstrations in the full arm-hand control space are costly and noisy because each trajectory must coordinate arm motion with high-DoF multi-finger contact. Small execution deviations can further cause slipped contact or premature force application, making recovery from off-nominal contact states important for robust execution. To deal with these challenges, we propose HiFun, a hierarchical real-world RL framework for functional dexterous manipulation tasks. Our key insight is to decouple the control complexity into a functional hand skill for precise contact execution and an arm-hand coordination policy for arm control and hand-skill activation. This design enables effective real-time human intervention for reactive recovery learning and efficient data collection with Dynamic Movement Primitive (DMP) augmentation. Across six real-world tasks on two dexterous hands, HiFun achieves 98.3% average success within one hour of online training and remains robust to dynamic disturbances. The project page is available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2610.19518
PDF
-
VLRLA: Deployment-Time Adaptation of Vision-Language-Action Models through In-Context Reinforcement Learning
2026-10-14 03:35:08Diyuan Shi, Yiqi Tang, Hongyin Zhang, You Wu, Donglin WangAbstract
Vision-language-action (VLA) models connect visual perception and language understanding with robot control, providing a foundation for instruction-conditioned manipulation. However, conventional pretrained VLA policies lack a mechanism for gradual adaptation through accumulated interaction experience during deployment. We therefore propose VLRLA, a general framework that equips existing VLA models with \textit{in-context reinforcement learning} capabilities for deployment-time adaptation. Our approach augments a VLA model with a reinforcement learning module that learns from historical observations, actions, rewards, and termination signals. During deployment, the policy gradually refines its action generation and progressively improves task success while keeping model parameters fixed. Our approach is also designed for general applicability and compatibility with diverse learning history construction strategies and model training pipelines. Finally, through extensive experiments, we systematically investigate the learning properties of VLRLA in simulation and demonstrate its ability to adapt to unseen scenarios during real-world deployment.
Abstract (translated)
URL
https://arxiv.org/abs/2610.19517
PDF
-
Test-Time Compute for Humanoid Behavior Foundation Models
2026-10-14 03:12:07Weiji Xie, Junsong Wu, Zibo Zhou, Yueru Chen, Xingyi Wang, Zhenhua Xiong, Weinan Zhang, Chenjia BaiAbstract
Humanoid Behavior Foundation Models (BFMs) can execute impressive motions, but downstream applications such as loco-manipulation demand high whole-body positional accuracy. Existing BFMs suffer from a systematic command-execution gap, where executed motions deviate from desired ones. We show that additional test-time compute can improve execution without retraining the BFM. First, we introduce Test-Time Command Optimization (TTCO), which adapts the input command to the specific BFM to produce an executed trajectory closer to the desired motion. TTCO finds such commands by sampling-based optimization through closed-loop simulations of the BFM and robot dynamics. Across three BFMs on the AMASS dataset, TTCO reduces mean global body-position error by 79-84% in simulation. Second, we analyze test-time compute scaling and derive an approximate compute-optimal strategy that can sustain performance gains as the compute budget increases. Third, TTCO can generate BFM-specific action labels from recorded robot trajectories without requiring the original teleoperation body-motion commands. Simulation and real-robot experiments show that these labels are effective for training high-level loco-manipulation policies. This method also supports reusing existing demonstrations across BFMs.
Abstract (translated)
URL
https://arxiv.org/abs/2610.14896
PDF
-
IMU-Bench: Benchmarking Wrist IMU Representations Across Kinematic, Semantic and Clinical Tasks
2026-10-14 02:58:38Yiquan Li, Aidan Bradshaw, Chenfeng Gao, Karan AhujaAbstract
Wrist-worn inertial measurement units (IMUs) provide a common sensing modality for tasks ranging from activity and gesture recognition to pose estimation, inertial odometry and clinical motor assessment. Existing wearable benchmarks evaluate performance across multiple tasks, but leave open how supervision for one task affects transfer to others and whether these tasks benefit from a shared representation. We present IMU-Bench, a benchmark of ten tasks from eight datasets for studying IMU multi-task performance and cross-task transfer. We compare six self-supervised methods pretrained on unlabeled recorded and simulated wrist motion, evaluating classification, pose and velocity estimation, and IMU-language retrieval. We find that our masked autoencoder, InrtlWrist, achieves the best or joint-best frozen-probe performance on nine of ten tasks, and fine-tuning improves on training from scratch on nine of ten tasks, including full-body pose, odometry and retrieval. Across transfer and joint-training experiments, supervision that benefits one target can reduce performance on others, with supervised sources transferring within kinematic estimation while, under frozen probing, none approaches the pretrained encoder on language retrieval, and joint-training gains that come at a cost to the paired task. These findings distinguish broad downstream usefulness from complementary task supervision. Code, task cards and a leaderboard are publicly available.
Abstract (translated)
URL
https://arxiv.org/abs/2610.19481
PDF
-
Active Generative Simulation: Meta-Learning What to Simulate for Sim-and-Real Policy Co-Training
2026-10-14 02:57:04Liqian Ma, Vaibhav Saxena, Utkarsh Aashu Mishra, Shuo Cheng, Yongxin Chen, Danfei XuAbstract
Sim-and-real co-training is an effective way to scale robot learning by augmenting limited real-world demonstrations with simulation data. However, existing approaches typically treat simulators as fixed data sources, relying on uniform generation, offline retrieval, or manual parameter alignment by domain experts. We formulate \textit{Active Generative Simulation}: optimizing a simulator's data-generation distribution to maximize downstream real-world performance. To solve this problem, we propose MetaGenSim, a meta-learning framework that trains an auxiliary rater network via meta-gradients to automatically discover high-utility data-generation distributions within a user-specified simulation parameter space, without manual parameter alignment. Across simulation and real-world manipulation tasks, MetaGenSim outperforms uniform generation and retrieval baselines, achieves a 150% relative improvement in real-world full-task success over uniform generation, and matches expert manual alignment. We further show that distributions curated with a lightweight proxy policy transfer to VLA fine-tuning, and that MetaGenSim can exploit partial alignment through atomic skill stitching and object-receptacle composition when exact sim-and-real task alignment is unavailable. Project website: this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2610.19479
PDF
-
Spacecraft Pose Tracking via Kinematic Prediction and Selective Event Correction
2026-10-14 02:46:18Yonglong Zhang, Guanning Yao, Shuo Zhang, Yifan Yang, Yang Liu{\dag}, Zongwu XieAbstract
In initialized spacecraft tracking, a short-horizon motion prediction remains available as event support changes. Observation-Aligned Event Innovation (OA-EI) uses events to correct selected components of this prediction. A constant-body-twist model propagates the complete pose; the event encoder proposes rotational and normalized image-plane translation corrections, retaining propagated axial depth. Candidate magnitudes come from a bounded Scale Gate and a deterministic activity factor, with empty windows returning the prediction unchanged. Evaluation uses four FRESH physical-event trajectories, twenty rollout starts, and three training seeds. Each rollout starts from two reference poses and then uses self-estimates. At 7.7 seconds, rotation and translation errors fall from 41.63°/57.28 mm for prediction alone to 15.73°/22.41 mm for OA-EI, reductions of 62.2% and 60.9%. Failure rates fall from 20.0% to 1.67%. Matched ablations favor the rotation-bearing interface for overall pose accuracy; combining scaling with activity support gives the lowest errors at every tested event-retention level. On Jetson Orin Nano, the measured in-memory update takes 44.56 milliseconds at the median and 58.77 milliseconds at P95.
Abstract (translated)
URL
https://arxiv.org/abs/2610.19465
PDF
-
Paraphrase Consensus Is Not Self-Consistency: Instruction Reliance and Robustness in VLA Policies
2026-10-14 02:30:09Hanlong Li, Sihan Tan, Takeshi Ashizawa, Benjamin Yen, Kazuhiro NakadaiAbstract
Vision-language-action (VLA) policies fine-tuned on LIBERO are trained and evaluated on the same ten instruction strings, and rewording the instruction changes base OpenVLA's success by -38.0 points. Test-time prompt ensembling, which improves language models, does not recover this loss in any setting we ran. Paraphrase consensus (the medoid action over K paraphrases) costs base OpenVLA 16.8 points when the training instruction is a member. Without it, the medoid of the paraphrases scores -32.4 points relative to the training instruction, and confidence-based selection scores -28.0. The same medoid over samples of one paraphrase improves on greedy decoding but stays -27.6 points below the training instruction, consistent with aggregation reducing variance without removing the bias the paraphrases share. A wrong-instruction probe measures how much of a policy's success depends on the instruction. Across three policies and four suites, paraphrases cost a policy only where this share is high, and the stronger policies lose a much smaller fraction of it than base OpenVLA. LoRA post-training on paraphrased instructions reduces the one-paraphrase effect to +2.0 points, against -32.0 for a matched control, reproduces across two seeds and on OpenVLA-OFT, transfers to withheld phrasings, and leaves the policy's reliance on the instruction unchanged.
Abstract (translated)
URL
https://arxiv.org/abs/2610.19438
PDF
-
$S^3$-Nav: Structured Spatial-Semantic Decision Making for Language-Guided Robot Navigation
Abstract
This paper presents $S^3$-Nav, a structured spatial-semantic decision-making framework for language-guided robot navigation under partial observability. Rather than assuming a complete environment representation in advance, $S^3$-Nav incrementally constructs persistent spatial-semantic state while exposing only a capacity-controlled, task-conditioned view to the high-level policy. This separation decouples what the robot must retain from what the policy needs to inspect, allowing decision complexity to be controlled without discarding grounded evidence required for subsequent reasoning and validation. High-level navigation outputs remain provisional and are independently grounded and validated against persistent semantic evidence and current geometric observations before physical execution or task termination. Evaluations across multiple environments and object- and relation-goal tasks demonstrate that separating persistent state, decision-facing evidence, and execution authority improves closed-loop navigation reliability while reducing policy reasoning cost.
Abstract (translated)
URL
https://arxiv.org/abs/2610.19431
PDF
-
A VR Training System for Robot-Assisted Vascular Intervention with Dual-Mode Control and Model-Based Interaction Response
2026-10-14 01:59:33Yining Shang, Shuxiang Guo, Sheng Cao, Pengcheng Li, Bin Wang, Mingchao DingAbstract
Virtual reality training for robot-assisted vascular intervention requires accurate robotic control and effective feedback on guidewire--vessel interactions. This paper presents a VR training system combining asynchronous dual-mode command mapping with sensorless interaction estimation. The system supports velocity-controlled navigation and position-driven incremental manipulation. Guidewire deformation and a Hunt--Crossley-type contact response are combined and temporally filtered to estimate interaction intensity, while trajectory-based visualization supports post-procedure review. Tests confirmed distinct command-dispatch patterns across three velocity gears and the position mode. During hybrid-scene navigation, the estimated response remained low during straight insertion and increased as the guidewire bent within the vascular model. Trajectory analysis yielded a model-derived cumulative trauma work (CTW) value of 15.166 in normalized units and localized regions of elevated interaction. In a preliminary study with 15 novices, physical-phantom task success increased from 53.3% to 93.3% after VR training, while mean completion time decreased from 132.4 s to 96.7 s. These results demonstrate the system's dual-mode operation, deformation-sensitive feedback, and potential to improve subsequent physical-phantom task performance.
Abstract (translated)
URL
https://arxiv.org/abs/2610.19413
PDF
-
Topology-Preserving Multi-Usage Pipe Centerline Refinement under Permanent Tubular Occupancy
Abstract
Discrete multi-group pipe routers place channels on a grid or coarse graph. The resulting centerlines are topologically feasible but typically contain many axis-aligned corners. For permanently occupied fluid tubes---as in additively manufactured hydraulic manifolds---those corners are undesirable: they degrade geometric fairness and, in practice, flow quality. This technical report studies the downstream geometric stage that starts from a fixed routing topology and a feasible coarse layout, and refines tubular centerlines under permanent clearance. Unlike classical robotic path smoothing, which usually fair one temporary open trajectory, our setting is a branched manifold with declared terminal usages, finite tube radii, and static cross-group occupancy. We formalize the refinement problem, highlight the multi-usage junction conflict that a single shared branch point cannot resolve, and outline a hybrid trunk-share / junction-split parameterization with soft geometric costs and hard clearance acceptance. On a multi-group suite ($8$ groups $\times$ $8$ terminals, $r{=}2$) with capsule clearance, the refine reaches full hard-accept success, $\kappa r\lesssim 1$, and about a $95\%$ relative drop in the soft objective. Physics-based pressure-drop optimization is intentionally deferred to a later PDE stage; here the objective remains geometric.
Abstract (translated)
URL
https://arxiv.org/abs/2610.19403
PDF
-
A Master-to-Simulation Interface with Demonstration-Trajectory Guidance for Vascular Intervention
2026-10-14 01:41:13Mingyu Zou, Shuxiang Guo, Sheng Cao, Pengcheng Li, Jian Guo, Bin Wang, Mingchao DinAbstract
Robotic vascular intervention can reduce radiation exposure to physicians and improve manipulation stability, but effective guidance remains challenging because conventional fluoroscopic images provide limited spatial information. This paper developed a vascular intervention interactive system that connects a real robotic master controller with the SOFA(Simulation Open Framework Architecture) physical simulation platform and reused successful operation trajectories as visual guidance in physical phantom experiments. Experiments were conducted to validate the master-to-SOFA control loop and to compare three guidance conditions: no guidance, centerline guidance, and successful-trajectory guidance. The control validation results showed reliable command delivery under velocity-mode operation. In the physical phantom experiment, centerline guidance showed no obvious improvement over no guidance. In contrast, successful trajectory guidance provided a clear performance benefit. Compared with the no-guidance condition, it reduced task completion time, collision number, rollback number, and rotation number by approximately 66.8%, 34.3%, 55.6%, and 66.7%, respectively. These results indicate that successful trajectories obtained under instrument-vessel interaction can provide practical guidance cues, offering a simple and effective visual assistance strategy for robotic vascular intervention experiments.
Abstract (translated)
URL
https://arxiv.org/abs/2610.19395
PDF
-
Beyond Kinematic Tracking: Preserving Expressive Characteristics in Humanoid Whole-Body Control
2026-10-14 01:39:14Lingfan Bao, Hanshang Zhu, Yan Pan, Jincheng Wang, Jingyang Yu, Tianhu Peng, Yiyang Jia, Dimitrios Kanoulas, Chengxu ZhouAbstract
Whole-body control plays a crucial role in enabling humanoid robots to execute expressive motions. Faithful execution requires preserving both the intended action and its expressive characteristics. Existing whole-body trackers typically balance joint and keypoint tracking with physical stability across diverse motions, but these objectives do not necessarily prioritize the expressive characteristics of each action. Our analysis shows that lower average positional tracking error does not guarantee better preservation of dynamic and postural motion characteristics. To address this gap, we propose a residual learning framework with compensation objectives targeting amplitude, energy, body expansion, body lean, and pelvis height. To concentrate compensation while limiting interference with the underlying tracking behavior, motion-aware gating allocates correction capacity according to body-part activity, preserving larger correction ranges in more active parts and restricting them elsewhere. For each frozen tracker, we train motion-specific residual policies and distill them into a unified policy shared across motion categories. Simulation and physical experiments on the Unitree G1 across eight motion categories demonstrate superior preservation of dynamic and postural motion characteristics compared with the corresponding baselines.
Abstract (translated)
URL
https://arxiv.org/abs/2610.19393
PDF
-
Probabilistic Physics Digital Twin for Human-Robot Interaction
2026-10-14 00:22:40Yushen Hu, Harshvardhan Chandirasekar, Apurv Kushwaha, Jonathan King, Zhu-Tian ChenAbstract
Digital twins are increasingly used in robotics to simulate interactions, predict action outcomes, and support planning. However, constructing accurate digital twins requires precise estimates of physical properties such as mass, friction, and center of mass, which are often difficult to obtain in real-world environments. We propose probabilistic digital twins, which represent uncertain physical properties as probability distributions rather than fixed values. By simulating actions across these distributions, the twin predicts a range of possible outcomes and their likelihoods. These probabilistic simulations can guide both human and robot actions by identifying actions that are effective despite physical uncertainty. We demonstrate this framework through human-robot interaction scenarios, including interfaces that communicate simulated outcomes to users and robot planning strategies that explicitly account for uncertainty.
Abstract (translated)
URL
https://arxiv.org/abs/2610.19336
PDF
-
Learning Keypoint Constraints from Visual Demonstrations for Safe Motion Planning
2026-10-13 23:15:47Zheng Qiu, Zhouyu Zhang, Ninghan Zhong, Glen ChouAbstract
We present Keypoint Constraint Learning (KCL), a framework for learning task-relevant motion constraints from visual demonstrations and for planning safely with the extracted constraints. To make constraint learning tractable from RGB-D observations, we represent demonstrations using sparse 3D object-centric keypoints and semantic mask centroids extracted through segmentation, visual feature clustering, and point tracking. In this reduced geometric space, we learn keypoint dynamics and infer distance- and orientation-based constraints that make the expert trajectories feasible and approximately locally optimal by enforcing KKT optimality conditions in the reduced state space. We then plan with the learned constraints under uncertainty, using robustness margins derived from accumulated prediction error, and provide a theoretical analysis of how keypoint perception error and learned dynamics error affect the recovered constraints. We evaluate KCL on visual demonstrations from navigation to rigid/deformable manipulation, showing that it recovers multi-stage geometric constraints and outperforms baselines in downstream safe planning.
Abstract (translated)
URL
https://arxiv.org/abs/2610.19295
PDF
-
MIND: Instruction-Conditioned Incremental Grounding for Robots
2026-10-13 22:33:02Mohamed Abouagour, Junhyun Nam, Wonse Jo, Byung-Cheol MinAbstract
Robots following persistent language instructions must decide which cached room judgments remain valid as their environment changes. We present MIND (Maintenance via INstruction-conditioned Dependencies), a framework for maintaining target-room decisions without re-evaluating every room after each event. A language model compiles each instruction into dependencies for semantic qualification and target selection. The controller intersects semantic dependencies with event footprints, refreshes affected judgments, and updates reachability and selection separately. For fixed deterministic evaluation, planning, and selection, conservative dependency and footprint coverage preserves the decisions of full recomputation. On 240 five-event ProcTHOR sequences with an exact benchmark predicate, MIND reduces mean semantic evaluations from 30.6 to 11.2 while preserving all 1,200 post-event decisions. Frozen-evaluator experiments with Phi-4 and Qwen3-14B also preserve all 1,200 decisions, without improving the evaluators' underlying task accuracy. In 32 closed-loop AI2-THOR episodes using exact-predicate qualification and metadata-derived footprints, MIND matches the final target decision of full recomputation in every episode. Unitree G1 experiments demonstrate target reuse and reselection across room transitions without additional semantic evaluations under MIND, and abstention after a barrier and an object removal at 4 evaluations against 15 to 18. Additional materials are available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2610.19262
PDF
-
Full-Order Safety for Nonlinear Reduced-Order MPC
2026-10-13 22:06:52Luis Pabon, Johannes K\"{o}hler, Revant Kasichainula, Marco PavoneAbstract
Reduced-order models (ROMs) make model predictive control (MPC) of high-dimensional nonlinear systems tractable, but the resulting modeling error can lead to constraint violations on the real system. We introduce CoRE-MPC, a robust MPC formulation that plans with a nonlinear ROM and guarantees recursive feasibility and constraint satisfaction for the full-order system. It can be applied with a broad class of nonlinear model reduction techniques and the resulting MPC does not require evaluating the full-order model online, thus ensuring computational efficiency. The key insight is in how the ROM affects the dynamics of the prediction error. We account for this with two error radii that are propagated alongside the ROM and provide a guaranteed bound on the prediction error. The approach is validated in simulation on an 80-state oscillator chain, a Burgers equation, and a 9,768-state finite-element soft robot, where a two-state ROM with data-driven bounds keeps all 1,628 nodes clear of a wall at a median solve time below 1 ms for a 20 ms control period. Across these examples, CoRE-MPC is 13 to 61 times faster than full-order robust MPC at only 27 to 35% higher closed-loop cost, whereas existing certified schemes are either infeasible or at least twice as conservative. Code: [released upon publication].
Abstract (translated)
URL
https://arxiv.org/abs/2610.19250
PDF
-
PACER: Process-Aware VLA Policy Learning from Human Corrections for Robotic Disassembly
2026-10-13 22:04:07Chang Liu, Sibo Tian, Litian Gong, Zhiwen Fan, Jiachen Li, Xiao Liang, Minghui ZhengAbstract
Robotic disassembly of end-of-life (EoL) products requires robots to accommodate product variation, follow prerequisite sequences, and approach components precisely for contact-rich manipulation. Supervised fine-tuning (SFT) provides a practical path for adaptation, but limited and costly disassembly data cannot capture the distribution shifts arising from changing environments and product uncertainties. The policy's own rollouts record its approach attempts, and human corrections demonstrate recovery from its errors, but turning these experiences into better policy performance remains challenging. We introduce PACER, a Process-Aware Correction and Evidence Reweighting framework that learns from these experiences through SFT, using progress toward the target and each trace's source and outcome to weight valid action chunks in the native supervised action loss. PACER trains one policy per weighting setting and selects one of the resulting policies through held-out validation. We evaluate PACER on UR5e motherboard disassembly across five component classes, where it achieves 87/100 successful approach-and-handoff trials on the held-out benchmark, compared with 54/100 for vanilla post-SFT.
Abstract (translated)
URL
https://arxiv.org/abs/2610.19248
PDF
-
End-to-End Vision-Language-Action Policies for Autonomous Lunar Rover Docking
2026-10-13 21:43:06Sitan Huang, Grace GaoAbstract
Surface navigation on the Moon is subject to extreme illumination contrast and deployment-time perceptual disturbances. While modular navigation pipelines have long supported planetary rover navigation, robustness is compromised by measurement errors that propagate through hand-engineered components. End-to-end learned policies are an alternative approach to navigation, which maps observations directly to control actions without intermediate components. Vision-language-action models, in particular, have driven significant breakthroughs in robotics and autonomous vehicle research. By leveraging internet-scale vision-language pretraining, these models can be exploited for their commonsense reasoning and visual understanding to provide robust generalization with limited task-specific data. However, their applicability and performance have yet to be characterized for lunar surface operations. In this study, we design a vision-language-action policy for autonomous lunar rover docking and characterize its performance relative to a modular navigation pipeline. We develop a scalable training framework that generates safety-aware expert demonstrations in a high-fidelity lunar simulation environment. In closed-loop evaluations, our policy performs competitively relative to the modular baseline under perceptual disturbances unseen during training. To address the tradeoff between compute consumption and closed-loop safety, the policy adapts its execution horizon by determining how far the rover may safely travel in open loop. Adaptive-horizon execution reduces inference frequency while limiting hazard-contact rates relative to fixed horizons in the evaluated scenarios.
Abstract (translated)
URL
https://arxiv.org/abs/2610.19224
PDF
