Trajectory prediction is a cornerstone in autonomous driving (AD), playing a critical role in enabling vehicles to navigate safely and efficiently in dynamic environments. To address this task, this paper presents a novel trajectory prediction model tailored for accuracy in the face of heterogeneous and uncertain traffic scenarios. At the heart of this model lies the Characterized Diffusion Module, an innovative module designed to simulate traffic scenarios with inherent uncertainty. This module enriches the predictive process by infusing it with detailed semantic information, thereby enhancing trajectory prediction accuracy. Complementing this, our Spatio-Temporal (ST) Interaction Module captures the nuanced effects of traffic scenarios on vehicle dynamics across both spatial and temporal dimensions with remarkable effectiveness. Demonstrated through exhaustive evaluations, our model sets a new standard in trajectory prediction, achieving state-of-the-art (SOTA) results on the Next Generation Simulation (NGSIM), Highway Drone (HighD), and Macao Connected Autonomous Driving (MoCAD) datasets across both short and extended temporal spans. This performance underscores the model's unparalleled adaptability and efficacy in navigating complex traffic scenarios, including highways, urban streets, and intersections.
轨迹预测是自动驾驶(AD)中的关键技术,在使车辆在动态环境中安全高效地导航方面发挥了重要作用。为解决这个问题,本文提出了一种专门针对异质和不确定交通场景的轨迹预测模型。这个模型的核心是基于特征扩散模块,这是一种创新的模块,旨在通过模拟固有不确定性的交通场景来提高预测准确性。通过向这个模型注入详细语义信息,从而增强了轨迹预测的准确性。此外,我们的空间-时间(ST)交互模块有效地捕捉了交通场景对车辆动力学的影响,在时间和空间维度上实现了对车辆动态的微小影响。通过详尽评估,我们的模型在轨迹预测方面达到了新的标准,在Next Generation Simulation(NGSIM)、高速公路无人机(HighD)和澳门 Connected Autonomous Driving(MoCAD)数据集上取得了最先进的(SOTA)结果。这种性能凸显了模型的无与伦比的适应性和有效性,使其能够应对复杂的交通场景,包括高速公路、城市街道和交叉口。
https://arxiv.org/abs/2405.02145
Collective decision-making enables multi-robot systems to act autonomously in real-world environments. Existing collective decision-making mechanisms suffer from the so-called speed versus accuracy trade-off or rely on high complexity, e.g., by including global communication. Recent work has shown that more efficient collective decision-making mechanisms based on artificial neural networks can be generated using methods from evolutionary computation. A major drawback of these decision-making neural networks is their limited interpretability. Analyzing evolved decision-making mechanisms can help us improve the efficiency of hand-coded decision-making mechanisms while maintaining a higher interpretability. In this paper, we analyze evolved collective decision-making mechanisms in detail and hand-code two new decision-making mechanisms based on the insights gained. In benchmark experiments, we show that the newly implemented collective decision-making mechanisms are more efficient than the state-of-the-art collective decision-making mechanisms voter model and majority rule.
集体决策使多机器人系统能够自主地在现实环境中行动。现有的集体决策机制存在所谓的速度与精度权衡,或者依赖于高复杂性,例如通过包括全局通信。最近的工作表明,基于人工神经网络的更高效集体决策机制可以通过进化计算方法来生成。这些决策神经网络的一个主要缺点是它们的可解释性有限。分析进化的决策机制可以帮助我们提高手编决策机制的效率,同时保持更高的可解释性。在本文中,我们详细分析了进化的集体决策机制,并基于所获得的见解手编了两个新的决策机制。在基准实验中,我们证明了新实施的共同决策机制比最先进的共同决策选民模型和多数规则更加高效。
https://arxiv.org/abs/2405.02133
Autonomous locomotion for mobile ground robots in unstructured environments such as waypoint navigation or flipper control requires a sufficiently accurate prediction of the robot-terrain interaction. Heuristics like occupancy grids or traversability maps are widely used but limit actions available to robots with active flippers as joint positions are not taken into account. We present a novel iterative geometric method to predict the 3D pose of mobile ground robots with active flippers on uneven ground with high accuracy and online planning capabilities. This is achieved by utilizing the ability of signed distance fields to represent surfaces with sub-voxel accuracy. The effectiveness of the presented approach is demonstrated on two different tracked robots in simulation and on a real platform. Compared to a tracking system as ground truth, our method predicts the robot position and orientation with an average accuracy of 3.11 cm and 3.91°, outperforming a recent heightmap-based approach. The implementation is made available as an open-source ROS package.
自治移动地面机器人在非结构化环境中(如路径规划或翻转控制)实现自主移动需要对机器人与地面之间的相互作用进行足够准确的预测。类似于占用网格或可穿越性地图等启发式方法被广泛使用,但它们限制了具有活动翻板的机器人的可用动作,因为它们没有考虑到关节位置。我们提出了一种新颖的迭代几何方法,可以预测带有活动翻板的移动地面机器人在不平滑地面上的3D姿态,具有高精度和在线规划能力。这是通过利用签名距离场表示具有子像素准确度的表面来实现的。所提出的方法的有效性在模拟中和真实平台上进行了演示。与跟踪系统作为地面真实情况相比,我们的方法预测机器人的位置和方向具有平均准确度为3.11cm和3.91°,超过了最近基于高图的方法的性能。该实现可作为开源ROS包提供。
https://arxiv.org/abs/2405.02121
We consider a set of challenging sequential manipulation puzzles, where an agent has to interact with multiple movable objects and navigate narrow passages. Such settings are notoriously difficult for Task-and-Motion Planners, as they require interdependent regrasps and solving hard motion planning problems. In this paper, we propose to search over sequences of easier pick-and-place subproblems, which can lead to the solution of the manipulation puzzle. Our method combines a heuristic-driven forward search of subproblems with an optimization-based Task-and-Motion Planning solver. To guide the search, we introduce heuristics to generate and prioritize useful subgoals. We evaluate our approach on various manually designed and automatically generated scenes, demonstrating the benefits of auxiliary subproblems in sequential manipulation planning.
我们考虑一组具有挑战性的序列操作谜题,其中智能体需要与多个可移动的对象进行交互并穿越狭窄的通道。对于任务和动作规划器来说,这样的设置通常是困难的,因为它们需要相互依存的规则和解决困难的动作规划问题。在本文中,我们提出了一个搜索更容易的挑选和放置子问题的序列的方法,这些问题可以导致操作谜题的解决。我们的方法结合了以启发式驱动的前向搜索子问题以及基于优化的任务和动作规划器的优化方法。为了引导搜索,我们引入了启发式来生成和优先考虑有用的子目标。我们在各种手动设计和自动生成的场景中评估了我们的方法,证明了辅助子问题在序列操作规划中的优势。
https://arxiv.org/abs/2405.02053
Monitoring large scale environments is a crucial task for managing remote alpine environments, especially for hazardous events such as avalanches. One key information for avalanche risk forecast is imagery of released avalanches. As these happen in remote and potentially dangerous locations this data is difficult to obtain. Fixed-wing vehicles, due to their long range and travel speeds are a promising platform to gather aerial imagery to map avalanche activities. However, operating such vehicles in mountainous terrain remains a challenge due to the complex topography, regulations, and uncertain environment. In this work, we present a system that is capable of safely navigating and mapping an avalanche using a fixed-wing aerial system and discuss the challenges arising when executing such a mission. We show in our field experiments that we can effectively navigate in steep terrain environments while maximizing the map quality. We expect our work to enable more autonomous operations of fixed-wing vehicles in alpine environments to maximize the quality of the data gathered.
监控大型环境对于管理远程 Alpine 环境至关重要,尤其是在可能引发山洪等危险事件的环境中。预测雪灾的一种关键信息是释放雪崩的影像。由于这些事件发生在远程且可能危险的位置,因此很难获得这些数据。固定翼车辆由于其长航程和高速旅行,是一个有前途的平台,用于收集高空影像以绘制雪崩活动。然而,在山区操作这些车辆仍然具有挑战性,由于复杂的地质、法规和不确定的环境。在这项工作中,我们提出了一个系统,使用固定翼空中系统安全地导航和绘制雪崩活动。并讨论了在执行此任务时出现的挑战。我们通过现场实验证明,在陡峭的地形环境中,我们既能保证最高地图质量,又能安全地导航。我们预计,我们的工作将使固定翼车辆在 Alpine 环境中实现更自主的操作,从而提高收集到的数据的质量。
https://arxiv.org/abs/2405.02011
Augmented reality (AR) has the potential to improve the immersion and efficiency of computer-assisted orthopaedic surgery (CAOS) by allowing surgeons to maintain focus on the operating site rather than external displays in the operating theatre. Successful deployment of AR to CAOS requires a calibration that can accurately calculate the spatial relationship between real and holographic objects. Several studies attempt this calibration through manual alignment or with additional fiducial markers in the surgical scene. We propose a calibration system that offers a direct method for the calibration of AR head-mounted displays (HMDs) with CAOS systems, by using infrared-reflective marker-arrays widely used in CAOS. In our fast, user-agnostic setup, a HoloLens 2 detected the pose of marker arrays using infrared response and time-of-flight depth obtained through sensors onboard the HMD. Registration with a commercially available CAOS system was achieved when an IR marker-array was visible to both devices. Study tests found relative-tracking mean errors of 2.03 mm and 1.12° when calculating the relative pose between two static marker-arrays at short ranges. When using the calibration result to provide in-situ holographic guidance for a simulated wire-insertion task, a pre-clinical test reported mean errors of 2.07 mm and 1.54° when compared to a pre-planned trajectory.
增强现实(AR)通过让外科医生将注意力集中在手术现场而不是外部显示屏上,从而改善了计算机辅助骨科手术(CAOS)的沉浸感和效率。成功地将AR应用于CAOS需要进行校准,以准确计算真实和全息物体之间的空间关系。几项研究通过手动对齐或使用手术场景中的附加引导标记来尝试进行这种校准。我们提出了一个通过使用广泛用于CAOS的IR反射型标记阵列直接校准AR头盔显示器(HMD)与CAOS系统的校准系统。在我们的快速、用户友好的设置中,HoloLens 2使用红外响应和通过HMD上的传感器获得的时间飞行深度来检测标记阵列的姿态。在与两个设备可见的IR标记阵列进行相对对齐时,研究测试发现了2.03mm和1.12°的相对跟踪平均误差。当使用校准结果为模拟电线插入任务提供现场全息指导时,一个早期临床试验报告了与预先规划轨迹相比较的2.07mm和1.54°的平均误差。
https://arxiv.org/abs/2405.01999
Stability and reliable operation under a spectrum of environmental conditions is still an open challenge for soft and continuum style manipulators. The inability to carry sufficient load and effectively reject external disturbances are two drawbacks which limit the scale of continuum designs, preventing widespread adoption of this technology. To tackle these problems, this work details the design and experimental testing of a modular, tendon driven bead-style continuum manipulator with tunable stiffness. By embedding the ability to independently control the stiffness of distinct sections of the structure, the manipulator can regulate it's posture under greater loads of up to 1kg at the end-effector, with reference to the flexible state. Likewise, an internal routing scheme vastly improves the stability of the proximal segment when operating the distal segment, reducing deviations by at least 70.11%. Operation is validated when gravity is both tangential and perpendicular to the manipulator backbone, a feature uncommon in previous designs. The findings presented in this work are key to the development of larger scale continuum designs, demonstrating that flexibility and tip stability under loading can co-exist without compromise.
在各种环境条件下保持稳定可靠的操作仍然是软性和连续式操作器的开放挑战。无法承受足够的负载并有效拒绝外部干扰是两个限制连续设计规模的因素,这阻止了这种技术的大规模应用。为解决这些问题,本文详细描述了具有可调刚度的模块化索具驱动的珠子式连续操作器的的设计和实验测试。通过嵌入能够独立控制结构不同部分的刚度,操作器可以在末端效应器上调节其姿态,达到超过1kg的负载。同样,内部路由方案在操作远端段时极大地提高了近端段的稳定性,将偏差减少至少70.11%。当重力既与操作器主干成角度又与操作器底部成垂直时,操作被验证。本文的工作成果对大型连续设计的发展至关重要,表明在施加负载下,灵活性和尖端稳定性可以共存而不妥协。
https://arxiv.org/abs/2405.01925
Modular reconfigurable manipulators enable quick adaptation and versatility to address different application environments and tailor to the specific requirements of the tasks. Task performance significantly depends on the manipulator's mounted pose and morphology design, therefore posing the need of methodologies for selecting suitable modular robot configurations and mounted pose that can address the specific task requirements and required performance. Morphological changes in modular robots can be derived through a discrete optimization process involving the selective addition or removal of modules. In contrast, the adjustment of the mounted pose operates within a continuous space, allowing for smooth and precise alterations in both orientation and position. This work introduces a computational framework that simultaneously optimizes modular manipulators' mounted pose and morphology. The core of the work is that we design a mapping function that \textit{implicitly} captures the morphological state of manipulators in the continuous space. This transformation function unifies the optimization of mounted pose and morphology within a continuous space. Furthermore, our optimization framework incorporates a array of performance metrics, such as minimum joint effort and maximum manipulability, and considerations for trajectory execution error and physical and safety constraints. To highlight our method's benefits, we compare it with previous methods that framed such problem as a combinatorial optimization problem and demonstrate its practicality in selecting the modular robot configuration for executing a drilling task with the CONCERT modular robotic platform.
模块可重构操纵器允许快速适应和多样性以应对不同的应用环境,并专门满足任务的特定要求。任务性能很大程度上取决于操纵器安装的姿态和形态设计,因此需要方法来选择合适的模块化机器人配置和安装姿势来满足特定任务要求和性能需求。通过离散优化过程,可以获得模块化机器人的形态变化。相反,安装姿势的调整在连续空间中进行,允许在方向和位置上进行平滑和精确的修改。本工作介绍了一个计算框架,同时优化模块化操纵器的安装姿势和形态。工作的核心是我们设计了一个映射函数,隐含地捕捉了连续空间中操纵器的形态状态。这个变换函数将安装姿势和形态的优化在连续空间中统一起来。此外,我们的优化框架包括一系列性能度量,如最小关节努力和最大可操作性,以及轨迹执行误差和物理和安全性考虑。为了突出我们方法的优点,我们将它与之前的方法进行了比较,这些方法将类似问题视为组合优化问题,并展示了其在选择使用CONCERT模块化机器人平台执行钻井任务时的实用性。
https://arxiv.org/abs/2405.01923
The detection of traversable regions on staircases and the physical modeling constitutes pivotal aspects of the mobility of legged robots. This paper presents an onboard framework tailored to the detection of traversable regions and the modeling of physical attributes of staircases by point cloud data. To mitigate the influence of illumination variations and the overfitting due to the dataset diversity, a series of data augmentations are introduced to enhance the training of the fundamental network. A curvature suppression cross-entropy(CSCE) loss is proposed to reduce the ambiguity of prediction on the boundary between traversable and non-traversable regions. Moreover, a measurement correction based on the pose estimation of stairs is introduced to calibrate the output of raw modeling that is influenced by tilted perspectives. Lastly, we collect a dataset pertaining to staircases and introduce new evaluation criteria. Through a series of rigorous experiments conducted on this dataset, we substantiate the superior accuracy and generalization capabilities of our proposed method. Codes, models, and datasets will be available at this https URL.
楼梯上的可通行区域的检测和楼梯的物理建模构成了机器人运动的关键方面。本文提出了一种专为检测可通行区域和建模楼梯物理属性通过点云数据而定制的车载框架。为了减轻光照变化和数据集差异导致的过拟合问题,一系列数据增强技术被引入以提高基本网络的训练。我们提出了一个曲率抑制交叉熵(CSCE)损失来降低在可通行和非可通行区域边界上的预测不确定性。此外,基于楼梯姿态估计的测量校正方法被引入,以校准受到倾斜视角影响的原始建模的输出。最后,我们还收集了一个楼梯数据的集合,并引入了新的评估标准。通过在对这个数据集的严谨实验过程中,我们证明了我们提出的方法的优越准确性和泛化能力。代码、模型和数据集将在这个[https:// URL]处提供。
https://arxiv.org/abs/2405.01918
Robotic applications across industries demand advanced navigation for safe and smooth movement. Smooth path planning is crucial for mobile robots to ensure stable and efficient navigation, as it minimizes jerky movements and enhances overall performance Achieving this requires smooth collision-free paths. Partial Swarm Optimization (PSO) and Potential Field (PF) are notable path-planning techniques, however, they may struggle to produce smooth paths due to their inherent algorithms, potentially leading to suboptimal robot motion and increased energy consumption. In addition, while PSO efficiently explores solution spaces, it generates long paths and has limited global search. On the contrary, PF methods offer concise paths but struggle with distant targets or obstacles. To address this, we propose Smoothed Partial Swarm Optimization with Improved Potential Field (SPSO-IPF), combining both approaches and it is capable of generating a smooth and safe path. Our research demonstrates SPSO-IPF's superiority, proving its effectiveness in static and dynamic environments compared to a mere PSO or a mere PF approach.
机器人应用在多个行业中需要先进的导航来实现安全和平稳的运动。平滑路径规划对于移动机器人来说至关重要,因为它可以最小化剧烈运动并提高整体性能。要实现这一点,需要平滑的冲突free路径。部分聚类优化(PSO)和势场(PF)是著名的路径规划技术,然而,由于其固有算法,它们可能无法产生平滑的路径,从而导致机器人运动 suboptimal 和能源消耗增加。此外,尽管PSO有效地探索解决方案空间,但它生成长路径,全局搜索有限。相反,PF方法提供简洁的路径,但与远距离目标或障碍物 struggle。为了应对这个问题,我们提出了平滑部分聚类优化和改进势场(SPSO-IPF)的方法,结合两种方法,它能够生成平滑和安全路径。我们的研究证明了SPSO-IPF的优越性,证明了与仅仅使用PSO或仅仅使用PF方法相比,其在静态和动态环境中的有效性。
https://arxiv.org/abs/2405.01794
Autonomous wheeled-legged robots have the potential to transform logistics systems, improving operational efficiency and adaptability in urban environments. Navigating urban environments, however, poses unique challenges for robots, necessitating innovative solutions for locomotion and navigation. These challenges include the need for adaptive locomotion across varied terrains and the ability to navigate efficiently around complex dynamic obstacles. This work introduces a fully integrated system comprising adaptive locomotion control, mobility-aware local navigation planning, and large-scale path planning within the city. Using model-free reinforcement learning (RL) techniques and privileged learning, we develop a versatile locomotion controller. This controller achieves efficient and robust locomotion over various rough terrains, facilitated by smooth transitions between walking and driving modes. It is tightly integrated with a learned navigation controller through a hierarchical RL framework, enabling effective navigation through challenging terrain and various obstacles at high speed. Our controllers are integrated into a large-scale urban navigation system and validated by autonomous, kilometer-scale navigation missions conducted in Zurich, Switzerland, and Seville, Spain. These missions demonstrate the system's robustness and adaptability, underscoring the importance of integrated control systems in achieving seamless navigation in complex environments. Our findings support the feasibility of wheeled-legged robots and hierarchical RL for autonomous navigation, with implications for last-mile delivery and beyond.
自动驾驶轮式机器人具有潜力彻底改变物流系统,提高操作效率和适应城市环境的灵活性。然而,在导航城市环境中还存在独特的挑战,对机器人的运动和导航提出了创新解决方案。这些挑战包括在各种地形上进行自适应运动以及高效地围绕复杂动态障碍物进行导航。本文介绍了一种集成系统,包括自适应运动控制、面向移动性的局部路径规划和城市规模路径规划。我们使用基于模型无关强化学习(RL)技术和优先学习方法开发了一个多功能的运动控制器。该控制器在各种崎岖不平的地面上实现高效的稳健运动,得益于平滑的步行和驾驶模式之间的转换。它与通过分层的RL框架集成的学习导航控制器紧密集成,使机器人能够有效通过具有挑战性的地形和各种障碍物的高速导航。我们的控制器被集成到大型城市导航系统中,并通过瑞士苏黎世和西班牙塞维利亚等地进行的自主、公里级导航任务进行了验证。这些任务突显了系统的稳健性和适应性,进一步强调了集成控制系统在复杂环境中实现无缝导航的重要性。我们的研究结果支持轮式机器人的可行性和层次式RL在自主导航方面的应用,这对末端交付和更广阔的应用领域都有重要的意义。
https://arxiv.org/abs/2405.01792
Multi-Robot-Arm Motion Planning (M-RAMP) is a challenging problem featuring complex single-agent planning and multi-agent coordination. Recent advancements in extending the popular Conflict-Based Search (CBS) algorithm have made large strides in solving Multi-Agent Path Finding (MAPF) problems. However, fundamental challenges remain in applying CBS to M-RAMP. A core challenge is the existing reliance of the CBS framework on conservative "complete" constraints. These constraints ensure solution guarantees but often result in slow pruning of the search space -- causing repeated expensive single-agent planning calls. Therefore, even though it is possible to leverage domain knowledge and design incomplete M-RAMP-specific CBS constraints to more efficiently prune the search, using these constraints would render the algorithm itself incomplete. This forces practitioners to choose between efficiency and completeness. In light of these challenges, we propose a novel algorithm, Generalized ECBS, aimed at removing the burden of choice between completeness and efficiency in MAPF algorithms. Our approach enables the use of arbitrary constraints in conflict-based algorithms while preserving completeness and bounding sub-optimality. This enables practitioners to capitalize on the benefits of arbitrary constraints and opens a new space for constraint design in MAPF that has not been explored. We provide a theoretical analysis of our algorithms, propose new "incomplete" constraints, and demonstrate their effectiveness through experiments in M-RAMP.
多机器人手臂运动规划(M-RAMP)是一个具有复杂单智能体规划和多智能体协调的具有挑战性的问题。最近,扩展流行的基于冲突的搜索(CBS)算法的进展已经大大解决了 Multi-Agent Path Finding(MAPF)问题。然而,将 CBS 应用于 M-RAMP 仍然存在一些基本挑战。核心挑战是 CBS 框架现有对保守“完整”约束的依赖。这些约束确保了解决方案保证,但通常会导致搜索空间中单次规划的重复且代价昂贵的调用。因此,即使可以利用领域知识和设计不完整的 M-RAMP 特定的 CBS 约束以更有效地剪枝搜索,但使用这些约束会使算法本身不完整。这迫使实践者必须在效率和完整性之间做出选择。鉴于这些挑战,我们提出了一个名为 Generalized ECBS 的新颖算法,旨在消除在 MAPF 算法中选择完整性和效率之间的负担。我们的方法允许在基于冲突的算法中使用任意约束,同时保留完整性和约束下的逼近最优解。这使得实践者能够利用任意约束的优势,并为 MAPF 中的新约束设计打开了一个新的空间。我们对算法进行了理论分析,提出了新的“不完整”约束,并通过在 M-RAMP 上的实验验证了它们的有效性。
https://arxiv.org/abs/2405.01772
The increased deployment of multi-robot systems (MRS) in various fields has led to the need for analysis of system-level performance. However, creating consistent metrics for MRS is challenging due to the wide range of system and environmental factors, such as team size and environment size. This paper presents a new analytical framework for MRS based on dimensionless variable analysis, a mathematical technique typically used to simplify complex physical systems. This approach effectively condenses the complex parameters influencing MRS performance into a manageable set of dimensionless variables. We form dimensionless variables which encapsulate key parameters of the robot team and task. Then we use these dimensionless variables to fit a parametric model of team performance. Our model successfully identifies critical performance determinants and their interdependencies, providing insight for MRS design and optimization. The application of dimensionless variable analysis to MRS offers a promising method for MRS analysis that effectively reduces complexity, enhances comprehension of system behaviors, and informs the design and management of future MRS deployments.
多机器人系统(MRS)在各种领域的广泛应用导致了系统级性能分析的需求。然而,为MRS创建一致的度量标准具有挑战性,由于涉及系统大小和环境大小的广泛范围因素。本文基于无度变量分析(维度无关变量分析)提出了一种新的MRS分析框架,这是一种通常用于简化复杂物理系统的数学技术。这种方法有效地将影响MRS性能的复杂参数压缩成可管理的一组无度变量。我们创建了包含机器人团队和任务关键参数的无度变量。然后,我们使用这些无度变量来拟合一个参数模型,该模型成功识别了关键绩效决定因素及其相互依赖关系,为MRS设计和优化提供了洞察。将维度无关变量分析应用于MRS提供了一种有前途的MRS分析方法,有效减少了复杂性,增强了系统行为的理解,并为未来MRS部署的设计和管理提供了指导。
https://arxiv.org/abs/2405.01771
Traditional optimization-based planners, while effective, suffer from high computational costs, resulting in slow trajectory generation. A successful strategy to reduce computation time involves using Imitation Learning (IL) to develop fast neural network (NN) policies from those planners, which are treated as expert demonstrators. Although the resulting NN policies are effective at quickly generating trajectories similar to those from the expert, (1) their output does not explicitly account for dynamic feasibility, and (2) the policies do not accommodate changes in the constraints different from those used during training. To overcome these limitations, we propose Constraint-Guided Diffusion (CGD), a novel IL-based approach to trajectory planning. CGD leverages a hybrid learning/online optimization scheme that combines diffusion policies with a surrogate efficient optimization problem, enabling the generation of collision-free, dynamically feasible trajectories. The key ideas of CGD include dividing the original challenging optimization problem solved by the expert into two more manageable sub-problems: (a) efficiently finding collision-free paths, and (b) determining a dynamically-feasible time-parametrization for those paths to obtain a trajectory. Compared to conventional neural network architectures, we demonstrate through numerical evaluations significant improvements in performance and dynamic feasibility under scenarios with new constraints never encountered during training.
传统的优化规划器虽然在效果上很有效,但计算成本很高,导致轨迹生成速度较慢。成功减少计算时间的方法之一是使用模仿学习(IL)从这些规划器中开发快速神经网络(NN)策略,将它们视为专家演示者。尽管生成的NN策略在快速生成类似于专家轨迹方面非常有效,但(1)它们的输出没有明确考虑到动态可行性,(2)这些策略没有考虑到训练过程中约束的变化。为了克服这些限制,我们提出了约束引导扩散(CGD),一种新型的IL-基轨迹规划方法。CGD利用了一种结合扩散策略和代理高效优化问题的混合学习/在线优化方案,使得可以生成无碰撞、动态可行轨迹。CGD的关键思想包括将专家通过 IL 解决的原始具有挑战性的优化问题划分为两个更容易管理子问题:(a)高效地找到无碰撞路径,(b)为这些路径确定一个动态可行的时间参数,以获得轨迹。与传统的神经网络架构相比,我们通过数值评估展示了在训练过程中从未遇到过的新的约束条件下,性能和动态可行性都有显著的提高。
https://arxiv.org/abs/2405.01758
Adaptive Cruise Control ACC can change the speed of the ego vehicle to maintain a safe distance from the following vehicle automatically. The primary purpose of this research is to use cutting-edge computing approaches to locate and track vehicles in real time under various conditions to achieve a safe ACC. The paper examines the extension of ACC employing depth cameras and radar sensors within Autonomous Vehicles AVs to respond in real time by changing weather conditions using the Car Learning to Act CARLA simulation platform at noon. The ego vehicle controller's decision to accelerate or decelerate depends on the speed of the leading ahead vehicle and the safe distance from that vehicle. Simulation results show that a Proportional Integral Derivative PID control of autonomous vehicles using a depth camera and radar sensors reduces the speed of the leading vehicle and the ego vehicle when it rains. In addition, longer travel time was observed for both vehicles in rainy conditions than in dry conditions. Also, PID control prevents the leading vehicle from rear collisions
自适应巡航控制(ACC)可以根据预设的速度自动改变车辆的自适应速度,以保持与后车安全距离。本研究的主要目的是利用尖端计算方法在各种情况下实时定位和跟踪车辆,以实现安全ACC。论文检查了在自动驾驶车辆(AV)中使用深度相机和雷达传感器扩展ACC,通过使用Car Learning to Act CARLA仿真平台在中午实时响应天气条件。自车控制器决定加速或减速取决于前车的速度和与该车辆的安全距离。仿真结果表明,使用深度相机和雷达传感器的自动驾驶车辆在下雨时,自车和前车的速度都会降低。此外,在雨天观察到的车辆行驶时间比干燥条件下更长。此外,PID控制还可以防止前车发生碰撞。
https://arxiv.org/abs/2405.01504
Bilateral teleoperation of an aerial manipulator facilitates the execution of industrial missions thanks to the combination of the aerial platform's maneuverability and the ability to conduct complex tasks with human supervision. Heretofore, research on such operations has focused on flying without any physical interaction or exerting a pushing force on a contact surface that does not involve abrupt changes in the interaction force. In this paper, we propose a human reaction time compensating haptic-based bilateral teleoperation strategy for an aerial manipulator extracting a wedged object from a static structure (i.e., plug-pulling), which incurs an abrupt decrease in the interaction force and causes additional difficulty for an aerial platform. A haptic device composed of a 4-degree-of-freedom robotic arm and a gripper is made for the teleoperation of aerial wedged object-extracting tasks, and a haptic-based teleoperation method to execute the aerial manipulator by the haptic device is introduced. We detect the extraction of the object by the estimation of the external force exerted on the aerial manipulator and generate reference trajectories for both the aerial manipulator and the haptic device after the extraction. As an example of the extraction of a wedged object, we conduct comparative plug-pulling experiments with a quadrotor-based aerial manipulator. The results validate that the proposed bilateral teleoperation method reduces the overshoot in the aerial manipulator's position and ensures fast recovery to its initial position after extracting the wedged object.
双边遥控操作航空手爪能够通过结合航空平台的可操纵性和在人类监督下执行复杂任务的特性,促进工业任务的执行。迄今为止,关于这种操作的研究主要集中在没有身体交互或对接触表面施加推力的情况下飞行。在本文中,我们提出了一个基于触觉反馈的双边遥控操作策略,用于从静态结构中提取楔形物体(即插销拉动)的航空手爪,该操作会导致相互作用力的急剧下降,并为航空平台带来额外的困难。 为了进行遥控操作,我们设计了一个4自由度机器人手臂和夹具组成的触觉装置,并引入了基于触觉的遥控方法来执行航空手爪。在提取楔形物体的过程中,我们通过估计航空手爪受到的外力来检测物体的提取,并为航空手爪和触觉装置生成参考轨迹。 以楔形物体的提取为例,我们与基于四旋翼的航空手爪进行了比较插销拉动实验。结果证实了所提出的双边遥控方法可以降低航空手爪的位置超差,并在提取楔形物体后确保其迅速恢复到初始位置。
https://arxiv.org/abs/2405.01361
Understanding user enjoyment is crucial in human-robot interaction (HRI), as it can impact interaction quality and influence user acceptance and long-term engagement with robots, particularly in the context of conversations with social robots. However, current assessment methods rely solely on self-reported questionnaires, failing to capture interaction dynamics. This work introduces the Human-Robot Interaction Conversational User Enjoyment Scale (HRI CUES), a novel scale for assessing user enjoyment from an external perspective during conversations with a robot. Developed through rigorous evaluations and discussions of three annotators with relevant expertise, the scale provides a structured framework for assessing enjoyment in each conversation exchange (turn) alongside overall interaction levels. It aims to complement self-reported enjoyment from users and holds the potential for autonomously identifying user enjoyment in real-time HRI. The scale was validated on 25 older adults' open-domain dialogue with a companion robot that was powered by a large language model for conversations, corresponding to 174 minutes of data, showing moderate to good alignment. Additionally, the study offers insights into understanding the nuances and challenges of assessing user enjoyment in robot interactions, and provides guidelines on applying the scale to other domains.
理解用户的喜爱在人机交互(HRI)中至关重要,因为它可能会影响交互质量和影响用户对机器的接受程度以及与机器的长期参与,特别是在与社交机器人的对话中。然而,目前的评估方法仅依赖自我报告问卷,无法捕捉交互动态。这项工作介绍了一个名为人机交互聊天机器人用户喜爱量表(HRI CUES)的新量表,用于从外部角度评估用户在机器人对话中的喜爱。通过与具有相关专业知识的三位注释者的深入讨论和严格的评估,该量表构建了一个结构化的框架,用于评估每个对话交流(回合)的喜爱程度以及整个交互水平。该量表旨在补充来自用户的自我报告喜爱,并具有在实时HRI中自动识别用户喜好的潜力。 该量表在25名年龄较大的成年人与一台由大型语言模型驱动的伴侣机器人进行开放领域的对话上进行了验证,对话持续了174分钟,显示出中等至良好的相关性。此外,这项研究揭示了评估用户喜爱在机器人交互中的细微问题和挑战,并为其他领域提供了应用该量表的指导。
https://arxiv.org/abs/2405.01354
Simulation is a fundamental tool in developing autonomous vehicles, enabling rigorous testing without the logistical and safety challenges associated with real-world trials. As autonomous vehicle technologies evolve and public safety demands increase, advanced, realistic simulation frameworks are critical. Current testing paradigms employ a mix of general-purpose and specialized simulators, such as CARLA and IVRESS, to achieve high-fidelity results. However, these tools often struggle with compatibility due to differing platform, hardware, and software requirements, severely hampering their combined effectiveness. This paper introduces BlueICE, an advanced framework for ultra-realistic simulation and digital twinning, to address these challenges. BlueICE's innovative architecture allows for the decoupling of computing platforms, hardware, and software dependencies while offering researchers customizable testing environments to meet diverse fidelity needs. Key features include containerization to ensure compatibility across different systems, a unified communication bridge for seamless integration of various simulation tools, and synchronized orchestration of input and output across simulators. This framework facilitates the development of sophisticated digital twins for autonomous vehicle testing and sets a new standard in simulation accuracy and flexibility. The paper further explores the application of BlueICE in two distinct case studies: the ICAT indoor testbed and the STAR campus outdoor testbed at the University of Delaware. These case studies demonstrate BlueICE's capability to create sophisticated digital twins for autonomous vehicle testing and underline its potential as a standardized testbed for future autonomous driving technologies.
模拟是在发展自动驾驶车辆中的一种基本工具,它允许在不需要与现实世界试验相关的物流和安全性挑战的情况下进行严格的测试。随着自动驾驶技术的发展和公共安全需求的增长,先进的、逼真的模拟框架至关重要。当前的测试范式采用通用和专用模拟器,如CARLA和IVRESS,以实现高保真度的结果。然而,由于不同平台、硬件和软件需求的不同,这些工具往往难以兼容,严重地阻碍了它们的综合效果。本文介绍了一种名为BlueICE的高级框架,以解决这些挑战。BlueICE创新的设计允许在计算平台、硬件和软件依赖之间进行解耦,并为研究人员提供可定制的测试环境,以满足不同的保真度需求。关键特点包括容器化以确保不同系统之间的兼容性,统一的通信桥实现各种模拟工具的无缝集成,以及模拟器之间同步操作输入和输出。这个框架促进了自动驾驶车辆测试中复杂数字孪生的开发,为模拟准确性和灵活性设定了新的标准。本文进一步探讨了BlueICE在两个不同案例研究中的应用:美国马里兰大学ICAT室内测试区和大学 of Delaware的STAR校园户外测试区。这些案例研究展示了BlueICE在创建自动驾驶车辆测试中的复杂数字孪生方面的能力,并强调了其在未来自动驾驶技术标准化测试床上的潜力。
https://arxiv.org/abs/2405.01328
Uncertainty in LiDAR measurements, stemming from factors such as range sensing, is crucial for LIO (LiDAR-Inertial Odometry) systems as it affects the accurate weighting in the loss function. While recent LIO systems address uncertainty related to range sensing, the impact of incident angle on uncertainty is often overlooked by the community. Moreover, the existing uncertainty propagation methods suffer from computational inefficiency. This paper proposes a comprehensive point uncertainty model that accounts for both the uncertainties from LiDAR measurements and surface characteristics, along with an efficient local uncertainty analytical method for LiDAR-based state estimation problem. We employ a projection operator that separates the uncertainty into the ray direction and its orthogonal plane. Then, we derive incremental Jacobian matrices of eigenvalues and eigenvectors w.r.t. points, which enables a fast approximation of uncertainty propagation. This approach eliminates the requirement for redundant traversal of points, significantly reducing the time complexity of uncertainty propagation from $\mathcal{O} (n)$ to $\mathcal{O} (1)$ when a new point is added. Simulations and experiments on public datasets are conducted to validate the accuracy and efficiency of our formulations. The proposed methods have been integrated into a LIO system, which is available at this https URL.
来自因素如测距感知的LiDAR测量的不确定性对LIO(LiDAR-Inertial Odometry)系统至关重要,因为它会影响损失函数的准确加权。虽然最近的一些LIO系统解决了与测距感有关的不确定性,但通常忽视了入射角对不确定性的影响。此外,现有的不确定性传播方法存在计算效率低下的问题。本文提出了一种全面的点不确定性模型,考虑了来自测距感的不确定性和表面特征,以及用于LiDAR基于状态估计问题的有效局部不确定性分析方法。我们采用一个投影操作将不确定性分解为光线方向和其垂直平面。然后,我们求解关于点的增量雅可比矩阵,使得不确定性传播变得更加快速。这种方法消除了冗余的点遍历,从而显著减少了不确定传播的时间复杂度从$\mathcal{O}(n)$到$\mathcal{O}(1)$,当添加新点时。在公开数据集上进行模拟和实验验证了我们的公式的准确性和效率。所提出的方法已集成到LIO系统中,该系统可用于此链接:https://。
https://arxiv.org/abs/2405.01316
The existing Motion Imitation models typically require expert data obtained through MoCap devices, but the vast amount of training data needed is difficult to acquire, necessitating substantial investments of financial resources, manpower, and time. This project combines 3D human pose estimation with reinforcement learning, proposing a novel model that simplifies Motion Imitation into a prediction problem of joint angle values in reinforcement learning. This significantly reduces the reliance on vast amounts of training data, enabling the agent to learn an imitation policy from just a few seconds of video and exhibit strong generalization capabilities. It can quickly apply the learned policy to imitate human arm motions in unfamiliar videos. The model first extracts skeletal motions of human arms from a given video using 3D human pose estimation. These extracted arm motions are then morphologically retargeted onto a robotic manipulator. Subsequently, the retargeted motions are used to generate reference motions. Finally, these reference motions are used to formulate a reinforcement learning problem, enabling the agent to learn a policy for imitating human arm motions. This project excels at imitation tasks and demonstrates robust transferability, accurately imitating human arm motions from other unfamiliar videos. This project provides a lightweight, convenient, efficient, and accurate Motion Imitation model. While simplifying the complex process of Motion Imitation, it achieves notably outstanding performance.
现有的运动模仿模型通常需要通过MoCap设备获得的专家数据,但需要的训练数据量巨大,很难获得,这需要大量的时间和财务资源。本项目将3D人体姿态估计与强化学习相结合,提出了一种将运动模仿简化为强化学习中关节角度预测问题的全新模型。这使得对大量训练数据的依赖程度显著降低,使得代理可以从几秒钟的视频中仅学习几个关节的模仿策略,并表现出强大的泛化能力。它能够快速将学习到的策略应用于不熟悉的视频中的模仿人类手臂运动。首先,使用3D人体姿态估计从给定的视频中提取人体的骨骼运动。然后,这些提取的运动动作被拓扑重构到机器人操作器上。接下来,重构的运动动作用于生成参考动作。最后,这些参考动作被用于构成强化学习问题,使得代理能够学习模仿人类手臂运动的策略。本项目在模仿任务中表现优异,并展示了稳健的泛化能力,准确地将不熟悉的视频中的人类手臂运动模仿出来。本项目提供了一个轻量、方便、高效和准确的动态模仿模型。尽管简化了运动模仿的复杂过程,但取得了显著的优异性能。
https://arxiv.org/abs/2405.01284