One important robotics problem is “How can one program a robot to perform a task”? Classical robotics solves this problem by manually engineering modules for state estimation, planning, and control. In contrast, robot learning solely relies on black-box models and data. This book shows that these two approaches of classical engineering and black-box machine learning are not mutually exclusive. To solve tasks with robots, one can transfer insights from classical robotics to deep networks and obtain better learning algorithms for robotics and control. To highlight that incorporating existing knowledge as inductive biases in machine learning algorithms improves performance, this book covers different approaches for learning dynamics models and learning robust control policies. The presented algorithms leverage the knowledge of Newtonian Mechanics, Lagrangian Mechanics as well as the Hamilton-Jacobi-Isaacs differential equation as inductive bias and are evaluated on physical robots.
Author(s): Michael Lutter
Publisher: Springer
Year: 2023
Language: English
Pages: 131
Series Editor’s Foreword
Preface
Acknowledgements
Contents
Acronyms
1 Introduction
1.1 Contribution
1.1.1 Differentiable Newton–Euler Algorithm
1.1.2 Deep Lagrangian Networks
1.1.3 Robust Fitted Value Iteration
1.2 Book Outline
References
2 A Differentiable Newton–Euler Algorithm for Real-World Robotics
2.1 Introduction
2.1.1 Naming Convention
2.1.2 Outline
2.2 Dynamics Model Representations
2.2.1 Black-Box Models
2.2.2 White-Box Models
2.2.3 Differentiable Simulators
2.3 Differentiable Newton–Euler Algorithm
2.3.1 Rigid-Body Physics and Holonomic Constraints
2.3.2 Virtual Physical Parameters
2.3.3 Rigid-Body Physics and Non-holonomic Constraints
2.3.4 Actuator Models
2.4 Experiments
2.4.1 Experimental Setup
2.4.2 Experiment 1—Trajectory Prediction
2.4.3 Experiment 2—Offline Reinforcement Learning
2.5 Conclusion
References
3 Combining Physics and Deep Learning for Continuous-Time Dynamics Models
3.1 Introduction
3.1.1 Outline
3.2 Related Work
3.2.1 Physics-Inspired Deep Networks
3.2.2 Continuous-Time Models and Neural ODEs
3.3 Preliminaries
3.3.1 Learning Dynamics Models
3.3.2 Lagrangian Mechanics
3.3.3 Hamiltonian Mechanics
3.4 Physics-Inspired Deep Networks
3.4.1 Deep Lagrangian Networks (DeLaN)
3.4.2 Hamiltonian Neural Networks (HNN)
3.4.3 Variations of DeLaN and HNN
3.5 Experiments
3.5.1 Experimental Setup
3.5.2 Model Prediction Experiments
3.5.3 Model-Based Control Experiments
3.6 Conclusion
3.6.1 Open Challenges
3.6.2 Summary
References
4 Continuous-Time Fitted Value Iteration for Robust Policies
4.1 Introduction
4.1.1 Outline
4.2 Problem Statement
4.2.1 Reinforcement Learning
4.2.2 Adversarial Reinforcement Learning
4.3 Deriving the Optimal Policy
4.3.1 Action Constraints
4.3.2 Optimal Adversary Actions
4.4 Continuous Fitted Value Iteration
4.4.1 N-Step Value Function Target
4.4.2 Dataset
4.4.3 Admissible Set
4.4.4 Value Function Representation
4.5 Experiments
4.5.1 Experimental Setup
4.5.2 Experimental Results
4.6 Related Work
4.6.1 Continuous-Time Reinforcement Learning
4.6.2 Fitted Value Iteration
4.6.3 Robust Policies for Sim2Real
4.7 Discussion
4.7.1 Worst Case Optimization
4.7.2 State Distribution and Dimensionality
4.7.3 Exploration
4.7.4 Known Dynamics Model
4.8 Summary
4.9 Appendix
4.9.1 Optimal Policy Proof—Theorem 4.1
4.9.2 Optimal Adversary Proof—Theorem 4.2
References
5 Conclusion
5.1 Summary
5.2 Open Problems and Future Work
5.2.1 Learning Dynamics Models
5.2.2 Learning Robust Policies
References
