Abstract
This letter proposes a data-driven method for learning convergent control policies from offline data using Contraction theory. Contraction theory enables constructing a policy that makes the closed-loop system trajectories inherently convergent towards a unique trajectory. At the technical level, identifying the contraction metric, which is the distance metric with respect to which a robot's trajectories exhibit contraction is often non-trivial. We propose to jointly learn the control policy and its corresponding contraction metric while enforcing contraction. To achieve this, we learn an implicit dynamics model of the robotic system from an offline data set consisting of the robot's state and input trajectories. We propose a data augmentation algorithm for learning contraction policies using this learned dynamics model. We randomly generate samples in the state space and propagate them forward in time through the learned dynamics model to generate auxiliary sample trajectories. We then learn both the control policy and the contraction metric such that the distance between the trajectories from the offline data set and our generated auxiliary sample trajectories decreases over time. We evaluate the performance of our proposed framework on simulated robotic goal-reaching tasks and demonstrate that enforcing contraction results in faster convergence and greater robustness of the learned policy.
Original language | English (US) |
---|---|
Pages (from-to) | 2905-2912 |
Number of pages | 8 |
Journal | IEEE Robotics and Automation Letters |
Volume | 7 |
Issue number | 2 |
DOIs | |
State | Published - Apr 1 2022 |
Keywords
- Convergence
- Deep Learning Methods
- Heuristic algorithms
- Machine Learning for Robot Control
- Measurement
- Reinforcement Learning
- Robots
- Robustness
- System dynamics
- Trajectory
- reinforcement learning
- Deep learning methods
- machine learning for robot control
ASJC Scopus subject areas
- Mechanical Engineering
- Control and Optimization
- Artificial Intelligence
- Human-Computer Interaction
- Control and Systems Engineering
- Computer Vision and Pattern Recognition
- Biomedical Engineering
- Computer Science Applications