Numerical Optimal Control

Lectures: Prof. Dr. Moritz Diehl,          Exercises: Armin Nurkanović

The course’s aim is to give an introduction into numerical methods for the solution of optimal control problems in science and engineering. The focus is on both discrete time and continuous time optimal control in continuous state spaces. It is intended for a mixed audience of students from mathematics, engineering, and computer science.

*** This page is intended for the course as taught in the summer semester 2023 at the University of Freiburg. For a timeless version with a focus on self study of the material see here. ***

Contact: moritz.diehl@imtek.uni-freiburg.dearmin.nurkanovic@imtek.uni-freiburg.de

 

Important announcement: Since the lecturing room HS II is not available due to a faculty internal event, the exercise on the 06.06. will be held online via Zoom.

Link

Topic: NOC Meeting Time: Jun 6, 2023 14:00 Amsterdam, Berlin, Rome, Stockholm, Vienna

Join Zoom Meeting https://uni-freiburg.zoom.us/j/62490711979?pwd=V1k4L2NHZ0FjY2xjcGdqalAvOFNJZz09

Meeting ID: 624 9071 1979 Passcode: vZ7mMtgL6

 

Structure of the course

This course is organized as inverted classroom and we provide recordings of the lecture and of the exercise solutions. We will meet once a week to discuss lecture or exercises. The course has 6 ECTS credits. It is possible to do a project to get an additional 3 ECTS, i.e., a total of 9 ECTS for course+project.

Meetings: We will meet every Tuesday, 14:05 to 15:45 in Room HS II, Albertstr. 23b (Institutsviertel)

These meetings are alternatingly dedicated to either Q&A sessions with Prof. Diehl or exercise sessions with the teaching assistant (see below). and will not be recorded.

Ilias: There is also an Ilias course, though most material will be published on the page you are currently viewing. In Ilias, we provide a forum for discussion of any questions you have related to the course, be it organization, content or exercises. Please feel free to open new topics and to answer questions of your fellow students. Further, the mid term quiz will be published on Ilias (see below).

 Link to join Ilias course.

Lecture recordings: The lecture recordings were already created in a past semester. There are 20 lectures of approximately 90 minutes each, which amounts to a lecture load of about 2 lectures or 3 hours per week. You can find a recommended schedule for watching them in the calendar below.

Course manuscript: The lectures are accompanied by a detailed course manuscript, which you may find in the materials section below. Please note that it is in general more detailed than the lectures and that we skip some of the chapters. We will distribute printed copies.

Exercises: The exercises are mainly computer based. Computers with (Matlab or Python) and CasADi installed are required to solve them (see below for details). There will be a total of 10 exercises. They will be published throughout the semester, after some time delay followed by a solution manuscript as well as a video recording. The exercises are voluntary (though of course we strongly recommend to solve them). Nonetheless we offer the possibility to hand them in to receive feedback, but for this please respect the deadlines you can find in the calendar below. If you would like feedback on a specific part of the exercise especially, you can state so on your solution sheet.

Q&A sessions: Every second week there will be a virtual Q&A session with Prof. Diehl, where you can ask any questions about the course content. The format is meant to be highly interactive and depends strongly on your participation. We would recommend that while watching the video lectures or reading the course script, you write down any questions that come to your mind, such that you have them readily available for the Q&A sessions.

Exercise sessions: Every other week we will meet for the exercise sessions. They will not be used to show the solutions, but to discuss any questions related to the exercises. These can either be questions about the current exercise sheet or questions about the solution to the last sheet. As the Q&A sessions, this format depends heavily on your participation.

Mid term quiz: Some time during the semester, we will publish a quiz on Ilias, with questions covering the course contents so far. It is obligatory that you pass this quiz until a deadline (see calendar)but you have infinitely many trials and at least one week for doing so and will receive instant feedback by auto-grading. Note that the questions will not necessarily be representative of an exam.

Final evaluation: The final exam is a written closed book exam. Only pen, paper, a calculator and two A4 sheets (i.e., 4 pages) of self-chosen content are allowed (handwritten). For students from M.Sc. MSE / ESE and B.Sc. Math, this exam is graded. Students from the M.Sc. Math need to pass the written exam in order to take the graded 11ECTS oral exam. Unmentioned special cases: Everyone who wants ECTS for this course needs to pass the exam.

Projects (more detail in a section below): The optional project (3 ECTS) consists in the formulation and implementation of a self-chosen problem of Numerical Optimal Control, resulting in documented computer code, a project report, and a public presentation. Project work starts in the last third of the semester. For students from the faculty of engineering the project is graded independently from the 6ECTS lecture. For students from the B.Sc. Math, the grade for the lecture&project 9ECTS module is solely determined by the written exam. For students from the M.Sc. Math the project is again a prerequisite to the graded 11ECTS oral exam.

 

Calendar

 

Date Format Content Watch this week Prepare Deadlines
18.04. Intro - Lec. 1 -  
25.04 Q&A up to including Chap. 2 Lec. 2, 3 one question Ex 1 (voluntary)
02.05 Ex Ex 2, 3, sol ex 1 Lec. 4, 5 one question  
09.05 Q&A up to including Chap. 4 Lec. 6, 7 one question Ex 2, 3 (voluntary)
16.05 Ex Ex 4,5 ,sol ex 2, 3 Lec. 8, 9 **  
23.05 Q&A up to including Chap. 8.2, project pitches Lec. 10, 11 one question Ex 4, 5 (voluntary)
30.05 ** ** pentecost break ** *** **  
06.06 Ex project finding exerciese (via Zoom) Lec. 12, 13 ** Ex 6 (voluntary)
13.06 Ex Ex 6, 7; sol ex 4, 5 Lec. 14, 15 one question mid term quiz (obligatory, 16.6. until 23:59)
20.06 Q&A projects; up to including Chap. 12 Lec. 16, 17 one question Ex 7, 8 (voluntary); project commitments
27.06 Ex proj; Ex 8, 9, 10; sol ex 6,7 Lec. 18, 19 one question  
04.07 Q&A proj; All course content Lec. 20 one question Ex 9, 10 (voluntary)
11.07 Ex proj; Sol ex 9, 10 (early feedback on reports and presentations)   one question  
18.07 pres project presentations     project presentations
25.07 none no session, just a deadline     project reports (until 23:59)
15.08 exam Location: TBA      

 

Manuscript

 

Lectures

 Topic  Chapters
 Lecture 1 - Welcome and introduction Part 1Part 2* -
 Lecture 2 - Introduction (cont.) and Newton-type methods 1.2 - 2.1
 Lecture 3 - Newton-type methods (cont.) 2.2 - 2.5
 Lecture 4 - Nonlinear optimization** 3 - 3.2
 Lecture 5 - Nonlinear optimization (cont) and Newton-type optimization algorithms 3.3 - 4.1
 Lecture 6 - SOSC (recall) and Newton-type optimization algorithms 3.3 - 4.2
 Lecture 7 - Newton-type optimization algorithms (Inequalities and globalization) 4.3 - 4.4
 Lecture 8 - Calculating derivatives 5
 Lecture 9 - Discrete time optimal control 7 - 7.3
 Lecture 10 - Discrete time optimal control: sparsity structure (cont.) 7.3
 Lecture 11 - Dynamic Programming 8 - 8.3
 Lecture 12 - Dynamic Programming (cont.) 8.3 - 8.8
 Lecture 13 - Differential Dynamic Programming*** -
 Lecture 14 - Continous time optimal control and numerical simulation 9 - 10.2
 Lecture 15 - Numerical simulation (cont.) 10.2 - 10.5
 Lecture 16 - Talk by Michael Neunert and Hamilton Jacobi Bellman Equation 11
 Lecture 17 - Pontryagin and the indirect approach 12
 Lecture 18 - Direct Approaches 13
 Lecture 19 - Model predictive control 15
 Lecture 20 - Parametric Nonlinear Optimization 16

* In part 2 there were some issues with the sound, but if you put your volume on maximum, you should be able to understand everything.
** Unfortunately, the microphone battery died at the end, so the last 10 minutes are mute.
*** Not yet covered by the lecture manuscript. Instead, please refer to Section 8.8.6 of Rawlings, Mayne, Diehl 2017. Model Predictive Control

Exercises

 Sheet (pdf)  Material (code) Solution (video)  Solution (material)
 Exercise 1 - Initial value problems ex1.zip sol ex1 ex1_sol.zip
 Exercise 2 - Nonlinear optimization and Newton-type methods ex2.zip sol ex2 ex2_sol.zip
 Exercise 3 - Equality constrained optimization ex3.zip sol ex3 ex3_sol.zip
 Exercise 4 - Inequality constrained optimization ex4.zip sol ex4 ex4_sol.zip
 Exercise 5 - Algorithmic differentiation - sol ex5 ex5_sol.zip
Exercise 6 - Optimal control formulations ex6.zip    
Exercise 7 - Dynamic Programming ex7.zip    
Exericse 8 - Continous time optimal control -    
       
  - -  

 

Further Material

  • Books
    • Rawlings, J. B., Mayne D. Q., Diehl, M., Model Predictive Control, 2nd Edition, Nobhill Publishing, 2017 (free PDF here)
    • Biegler, L. T., Nonlinear Programming, SIAM, 2010
    • Betts, J., Practical Methods for Optimal Control and Estimation Using Nonlinear Programming, SIAM, 2010
  • Sample exam
  • See the first 10 minute of this talk for a short introduction to embedded optimization.

 

Project

You can find the project guidelines here.

 

Software

The exercises lean heavily on the open source tool CasADi, which offers an interface to Python, Matlab, and Octave. The computer exercise templates as well as the code solutions will be published in Matlab and Python and we offer support for both. The solution videos however are based on the Matlab version.

Python is one of the major programming languages and via libraries such as NumPy and SciPy a common choice for scientific computing. Apart from CasADi, we will use the libraries NumPy, SciPy and Matplotlib. If you are completly new to Python, you may want to check out the Anaconda distribution.

Matlab is an environment for numerical computing based on a proprietary language that allows one to easily manipulate matrices and visualize data. The University of Freiburg offers a free-of-cost license to students and staff which can be obtained following the instructions here.

CasADi is a symbolic framework for algorithmic differentiation and numerical optimization. In order to install CasADi, follow the instructions here.
Matlab: Download the binaries for your platform and, after having extracted them, add their location to MATLAB's path. To test your installation run the simple example described at the provided link. If successful, save the path by executing the command savepath. In this way, the location of the binaries will be known even after restarting MATLAB.
Python: If you are using pip, you can simply pip install casadi in your preferred environmentIf you are using Anaconda / conda for managing your environments, we suggest to first conda install pip inside a conda environment, followed by pip install casadi.