Model Predictive Control and Reinforcement Learning

Block Course, 04.10.2022 - 10.10.2022, 9:00-17:00, HS 1098, Kollegiengebäude I, Platz der Universität 3, 79098 Freiburg i.Br.

 

This block course is intended for master and PhD students from engineering, computer science, mathematics, physics, and other mathematical sciences. The aim is that participants understand the main concepts of model predictive control (MPC) and reinforcement learning (RL) as well the similarities and differences between the two approaches. In hands-on exercises and project work they learn to apply the methods to practical optimal control problems from science and engineering. 

The course consists of lectures, exercises, and project work. The lectures in are given by Prof. Dr. Joschka Boedecker and Prof. Dr. Moritz Diehl from the University of Freiburg.

Topics include

  • Optimal Control Problem (OCP) formulations - constrained, infinite horizon, discrete time, stochastic, robust
  • Markov Decision Processes (MDP)
  • From continuous to discrete: discretization in space and time
  • Dynamic Programming (DP) concepts and algorithms - value iteration and policy iteration
  • Linear Quadratic Regulator (LQR) and Riccati equations
  • Convexity considerations in DP for constrained linear systems
  • Model predictive control (MPC) formulations and stability guarantees
  • MPC algorithms - quadratic programming, direct multiple shooting, Gauss-Newton, real-time iterations
  • Differential Dynamic Programming (DDP) for the solution of unconstrained MPC problems 
  • Reinforcement Learning (RL) formulations and approaches  
  • Model-free RL: Monte Carlo, temporal differences, model learning, direct policy search
  • RL with function approximation
  • Model-based RL and combinations of model-based and model-free methods
  • Similarities and differences between MPC and RL

Prerequisites

To be able to treat the contents on a sufficiently high level, we will assume some basic knowledge of Numerical Optimal Control and Reinforcement learning. Students should be familiar with the following contents before the start of the course, namely the first 5 lectures of last year's MPCRL course:

 

More information as well as a course schedule coming soon.