Abstract
We propose a novel gradient-based online optimization framework for solving stochastic programming problems that frequently arise in the context of cyber-physical and robotic systems. Our problem formulation accommodates constraints that model the evolution of a cyber-physical system, which has, in general, a continuous state and action space, is nonlinear, and where the state is only partially observed. We also incorporate an approximate model of the dynamics as prior knowledge into the learning process and show that even rough estimates of the dynamics can significantly improve the convergence of our algorithms. Our online optimization framework encompasses both gradient descent and quasi-Newton methods, and we provide a unified convergence analysis of our algorithms in a non-convex setting. We also characterize the impact of modeling errors in the system dynamics on the convergence rate of the algorithms. Finally, we evaluate our algorithms in simulations of a flexible beam, a four-legged walking robot, and in real-world experiments with a ping-pong playing robot.
Abstract (translated)
我们提出了一个新颖的基于梯度的在线优化框架，用于解决在计算机物理和机器人系统中经常出现的随机规划问题。我们的问题建模容纳了描述计算机物理系统演变的一般连续状态和动作空间、非线性的状态，并且仅部分观察到的状态。我们还将近似模型动态作为先验知识纳入学习过程，并证明了即使对动态的粗略估计，也可以显著提高算法的收敛。我们的在线优化框架包括梯度下降和准Newton方法，并在非凸设置中提供对算法收敛分析的统一。我们还研究了系统动态建模误差对算法收敛率的影响。最后，我们在柔性梁、四足行走机器人和实世界乒乓球机器人上进行了算法的仿真评估。
URL
https://arxiv.org/abs/2404.05318
