Lectures: Prof. Dr. Moritz Diehl (email),
Exercises: Léo Simpson (email)
*** This page is intended for the course as taught in the summer semester 2026 at the University of Freiburg. **
The course provides an introduction to numerical methods for solving optimization problems in science and engineering. The focus is on continuous nonlinear optimization in finite dimensions, covering both convex and nonconvex problems. It is intended for a mixed audience of students from mathematics, engineering, and computer science.
The course is organized as an inverted classroom and is based on four pillars:
- a course manuscript
- lecture recordings, and
- exercises that are accompanied by solutions and solution recordings, and
- weekly alternating Q&A and exercise sessions to discuss the course contents
Contact: moritz.diehl@imtek.uni-freiburg.de, leo.simpson@imtek.uni-freiburg.de
Organization of the course
**************************************************************************************************************
Exam date: 9th of September (to be confirmed) | Rooms: ...
**************************************************************************************************************
The course is organized as a flipped classroom. We provide recordings of the lecture and will meet once a week to discuss the course content. This course has 6 ECTS credits. It is possible to undertake a project to earn an additional 3 ECTS, bringing the total to 9 ECTS for the course and project. For more information, please get in touch with Léo Simpson.
Meetings: We will meet every Tuesday, 14:10 to 15:45 in Room HS II, Albertstraße 23b.
These meetings are alternatingly dedicated to either Q&A sessions with Prof. Diehl or exercise sessions with the teaching assistant (see calendar below), and will not be recorded.
Ilias: Even though most of the material will be published on this page, there is also an Ilias course-page. Official communication will be conducted via the Ilias forum. You can also use the forum for course-related discussions and to ask questions (content- or organization-related). Please feel free to open new topics and to answer questions from your fellow students. Furthermore, a mid-term quiz will be posted on Ilias (see the section on the midterm quiz below).
Lecture recordings: The lecture recordings were already created in a past semester. There are 24 lectures, each approximately 90 minutes long. You can find the recordings in the materials section below, and a recommended schedule on the calendar.
Course manuscript: The lectures closely follow a course manuscript: Manuscript for Numerical optimization (2026 version). We also provide printed versions.
Exercises: The exercises combine programming and theoretical parts. Computers with Python and CasADi installed are required to solve the programming parts (see section on Software at the end for details). The exercises are voluntary, but we strongly recommend that you solve them. We offer the option to hand them in to receive feedback, but please respect the deadlines listed in the calendar below. If you would like feedback on a specific part of the exercise, you can state so on your solution sheet. To hand them in, send them to Léo Simpson by email.
The table with the exercise sheets and the solutions (when available) is available in the section Exercise below.
Q&A sessions: Every second week, there will be a virtual/(in-person?) Q&A session with Prof. Diehl, where you can ask any questions about the course content. The format is meant to be highly interactive and depends strongly on your participation. We recommend that, while watching the video lectures or reading the course script, you write down any questions that come to mind, so you have them readily available for the Q&A sessions.In the Q&A sessions, this format depends heavily on your participation.
Exercise sessions: We will meet every other week for exercise sessions. They will be used to discuss the solutions and any questions related to the exercises. These can be either questions about the current exercise sheet or about the solution to the last sheet.
Mandatory computer exercise: There will be one obligatory, computer-based exercise sheet. In order to pass, you need to do a live demonstration of your resulting code to one of the teachers. You should be able to a) explain your code b) execute your code live c) do spontaneous modifications based on requests by the teachers. The sheet will be published before the holidays. It is mandatory that you demonstrate your code either on June 2nd or June 9th. You can also use the June 2nd exercise session to ask questions, get preliminary feedback, etc.
Here you can find the description of the programming exercise: programming-exercise.pdf . We also provide a Python code that you can use to visualize your solutions to the problem: visualization_circles.zip
Midterm quiz: now available on the Ilias course-page.
It is obligatory that you pass this quiz by 05.06.2026 (23:59). In order to pass, you need to get at least 80% of the points. Note that you have an infinite number of trials to do so and will receive instant feedback by auto-grading.
Also note that the questions will not necessarily be representative of an exam.
Projects: (project guidelines will be available later) The optional project (3 ECTS) consists of the formulation and implementation of a self-chosen optimization problem or numerical solution method, resulting in documented computer code, a project report, and a public presentation. Project work starts in the last third of the semester. For students in the faculty of engineering, the project is graded independently of the 6 ECTS lecture. For students from the B.Sc. In Math, the grade for the lecture and project 9ECTS module is determined solely by the written exam. For students in the M.Sc. in Mathematics program, the project is again a prerequisite for the graded 11 ECTS oral exam.
Final evaluation: The final exam is a written exam. Only pen, paper, a non-programmable calculator, and two A4 cheat-sheets (i.e., 4 pages) of handwritten content are allowed. For students from M.Sc. MSE / ESE and B.Sc. Math, this exam is graded. Students in the M.Sc. in mathematics program must pass the written exam to take the graded 11ECTS oral exam. Everyone who wants ECTS for this course needs to pass the exam.
To help you studying for the exam, we provide sample exams below in Section
Overview of course requirements
Below, the core components of the course are listed. For both modules, in order to get the ECTS you need to pass each of the listed requirements in some form or another. Whether the exam and the project report are SL or PL (graded or ungraded), and whether you can do the 6 and 3 ECTS modules independently, or only combined as 9 ECTS module, depends on the details of your M.Sc. (B.Sc.) program.
For the details of each requirement, see above.
6 ECTS lecture module
"Exercise"-Studienleistung (SL)
Mid-term quiz
Mandatory programming exercise
Written exam (PL or SL for some study programs)
3 ECTS project module
Project presentation ("exercise"-SL)
Project report
For M. Sc. Math it is also possible to do an additional oral exam in order to get an overall 11 ECTS. Please contact us for exam dates.
Calendar
| Date | Format | Content | Watch until this week | Read until this week | Prepare | Deadlines | Comments |
| 21.04 | Introduction | Q&A | Lec. 1, 2 | Chap. 1&2 | - | ||
| 28.04 | Exercise | Solution to Ex 1 | Lec. 3 | Chap. 3 | one exercise question | Ex 1 (voluntary) | |
| 05.05 | Q&A | Q&A | Lec. 4,5 | Chap . 4 & 5 | one lecture question | ||
| 12.05 | Exercise | Solution to Ex 2 | Lec. 6,7 | Chap. 6 | one exercise question | Ex 2 (voluntary) | |
| 19.05 | Q&A | .Q&A | Lec. 8,9 | Chap. 7 & 8 | one lecture question | ||
| 26.05 | *** Lecture break *** | ||||||
| 02.06 | Exercise |
| Lec. 10,11 | Chap. 9 |
|
| |
| 09.06 | Q&A |
| Lec. 12,13 | Chap. 10 |
| Complete the programming exercise (II) | |
| 16.06 | Q&A | Q&A | Lec. 14 | Chap. 11 | one lecture question | ||
| 23.06 | Exercise | Solution to Ex 4 | Lec. 15, 16, 17 | Chap. 12 | one exercise question |
| |
| 30.06 | Q&A | Q&A + project discussion | Lec. 18, 19,20 | Chap. 13 |
| ||
| 07.07 | Exercise | Solution to Ex 5 | Lec. 21,22 | Chap. 14 & 15 | one exercise question | Ex 5 (voluntary) | |
| 14.07 | Q&A + Project work | Project work | Lec. 23,24 | Project questions | |||
| 21.07 | Project presentation | Project presentation | |||||
| 05.08 | (no session - just a deadline) | project report | project report deadline | ||||
Lectures
The video recordings correspond to approximately 90 minutes each and comprise 24 lectures in total. A recommended schedule for watching can be found in the calendar above.
| Lecture | Content |
| Lecture 1 | Introduction to Section 1.3 (Mathematical formulation) |
| Lecture 2 | Section 1.4 (Definitions) to 2.7 (Mixed-Integer-Programming) |
| Lecture 3 | Section 3.1 (How to check convexity) to 3.5 (Standard form of convex opt. problems) |
| Lecture 4 | Section 3.6 (Semidefinite Programming) to Example 4.2 (Dual of LP) |
| Lecture 5 | Example 4.3 (Dual decomposition) to Chapter 6 introduction |
| Lecture 6 | Section 6.1 (Linear Least Squares) to 6.5 (L1-Estimation) |
| Lecture 7 | Section 6.6 (Gauss-Newton method) to 7.2 (Local convergence rates) |
| Lecture 8 | Section 7.3 (Newton-Type methods) to 8.1 (Local contraction) |
| Lecture 9 | Section 8.2 (Affine invariance) to 9.1 (Line search) |
| Lecture 10 | Section 9.2 (Wolfe conditions) to 9.3 (Global convergence of line search) |
| Lecture 11 | Section 9.4 (Trust-Region methods) to 9.5 (The Cauchi-Point) |
| Lecture 12 | Section 10.1 (Algorithmic Differentiation) to 10.3 (Backward AD) |
| Lecture 13 | Section 10.4 to Chapter 11 introduction |
| Lecture 14 | Section 11.1 (LICQ and linearized feasible cone) to 11.2 (SONC) |
| Lecture 15 | Section 12.1 (Optimality conditions) to 12.5 (Constrained Gauss-Newton) |
| Lecture 16 | Section 11.3 (Perturbation analysis) and 12.7 (Local convergence) |
| Lecture 17 | Section 12.6 (General constrained NT-Algorithm) to 12.9 (Careful BFGS updating) |
| Lecture 18 | Section 13.1 to 13.2 (Active constraints and LICQ) |
| Lecture 19 | Section 13.3 (Convex Problems) |
| Lecture 20 | Section 13.4 (Complementarity) to 14.1 (QPs via Active Set Method) |
| Lecture 21 | Section 14.2 (SQP) to 14.4 (Interior Point methods) |
| Lecture 22 | Section 14.4 (Barrier problem interpretation, SCP) to 15.5 (Simultaneous optimal control) |
| Lecture 23 | Problem reformulations and useful function approximations |
| Lecture 24 | Summary of the course |
Exercises
The exercises are based partially on pen and paper and partially on Python (see below).
Topic | Questions (pdf) | Material (code) | Solutions (pdf) | Solutions (code) |
| Introduction to CasADi, optimization problems | Exercise 1 | Template 1 | Solutions 1 | Solution code 1 |
| Convexity, duality, fitting problems | Exercise 2 | - | Solutions 2 | Solution code 2 |
| Unconstrained optimization | Exercise 3 | Template 3 | ||
| Equality constraints optimization | ||||
| Inequality constraints optimization (with an exam-type question) |
If you find typos or mistakes in the exercises, please report them to Léo Simpson.
Further Material (Books)
References on Optimization
- Jorge Nocedal and Stephen J. Wright, Numerical Optimization, Springer, 2006
- Stephen Boyd and Lieven Vandenberghe, Convex Optimization, Cambridge Univ. Press, 2004
- Amir Beck, Introduction to Nonlinear Optimization, MOS-SIAM Optimization, 2014.
References for mathematical background:
- This course's manuscript now contains an appendix with a very brief description of the required mathematical background (Appendix C).
- The Convex Optimization book mentioned above also contains a slightly more detailed appendix on mathematical foundations (Appendix A).
- For a textbook on Analysis/advanced calculus: W. Rudin. Principles of Mathematical Analysis. McGraw-Hill, 1976.
- For a textbook on linear algebra: C. D. Meyer. Matrix Analysis and Applied Linear Algebra. Society for Industrial and Applied Mathematics, 2000
- For another textbook on linear algebra: G. Strang. Linear Algebra and its Applications. Academic Press, 1980
Exam preparation
We provide two sample exams to prepare for the exam.
Software
The exercises rely heavily on the open-source tool CasADi, which provides interfaces to Python, MATLAB, and Octave. The computer exercise templates, as well as the code solutions, will be published in Python only.
Python is one of the major programming languages and, via libraries such as NumPy and SciPy, a common choice for scientific computing. Apart from CasADi, we will use the libraries NumPy, SciPy, and Matplotlib. If you are completely new to Python, you may want to check out the Anaconda distribution.
CasADi is a symbolic framework for algorithmic differentiation and numerical optimization. To install CasADi, follow the instructions here.
If you are using pip, you can simply run 'pip install casadi ' in your preferred environment. If you are using Anaconda / conda for managing your environments, we suggest to first conda install pip inside a conda environment, followed by pip install casadi.
*** This page is intended for the course as taught in the summer semester 2026 at the University of Freiburg. For a timeless version focused on self-study of the material, see here.***