Lectures: Prof. Dr. Moritz Diehl (email),
Exercises: Léo Simpson (email)
*** This page is intended for the course as taught in the summer semester 2026 at the University of Freiburg. **
The course provides an introduction to numerical methods for solving optimization problems in science and engineering. The focus is on continuous nonlinear optimization in finite dimensions, covering both convex and nonconvex problems. It is intended for a mixed audience of students from mathematics, engineering, and computer science.
The course is organized as an inverted classroom and is based on four pillars:
- a course manuscript
- lecture recordings, and
- exercises that are accompanied by solutions and solution recordings, and
- weekly alternating Q&A and exercise sessions to discuss the course contents
Contact: moritz.diehl@imtek.uni-freiburg.de, leo.simpson@imtek.uni-freiburg.de
Organization of the course
**************************************************************************************************************
Exam date: ... | Rooms: ...
**************************************************************************************************************
The course is organized as a flipped classroom. We provide recordings of the lecture and will meet once a week to discuss the course content. This course has 6 ECTS credits. It is possible to undertake a project to earn an additional 3 ECTS, bringing the total to 9 ECTS for the course and project. For more information, please get in touch with Léo Simpson.
Meetings: We will meet every Tuesday, 14:10 to 15:50 in Room HS II, Albertstraße 23b.
These meetings are alternatingly dedicated to either Q&A sessions with Prof. Diehl or exercise sessions with the teaching assistant (see calendar below), and will not be recorded.
Ilias: Even though most of the material will be published on this page, there will also be an Ilias course. Official communication will be conducted via the Ilias forum. You can also use the forum for course-related discussions and to ask questions (content- or organization-related). Please feel free to open new topics and to answer questions from your fellow students. Furthermore, a mid-term quiz will be posted on Ilias (see the section on the midterm quiz below).
Lecture recordings: The lecture recordings were already created in a past semester. There are 24 lectures, each approximately 90 minutes long. You can find the recordings in the materials section below, and a recommended schedule on the calendar.
Course manuscript: The lectures are accompanied by a detailed course manuscript. We will provide printed versions.
Exercises: The exercises are mainly computer-based. Computers with Python and CasADi installed are required to solve them (see below for details). The exercises are voluntary, but we strongly recommend that you solve them. We offer the option to hand them in to receive feedback, but please respect the deadlines listed in the calendar below. If you would like feedback on a specific part of the exercise, you can state so on your solution sheet. To hand them in, send them to Léo Simpson by email.
Q&A sessions: Every second week, there will be a virtual/(in-person?) Q&A session with Prof. Diehl, where you can ask any questions about the course content. The format is meant to be highly interactive and depends strongly on your participation. We recommend that, while watching the video lectures or reading the course script, you write down any questions that come to mind, so you have them readily available for the Q&A sessions.
Exercise sessions: We will meet every other week. They will be used to discuss the solutions and any questions related to the exercises. These can be either questions about the current exercise sheet or about the solution to the last sheet. In the Q&A sessions, this format depends heavily on your participation.
Midterm quiz: In the middle of the semester, we will publish a quiz on Ilias, with questions covering the course content so far. It is obligatory that you pass this quiz by 05.06.2026 (23:59). Note that you have an infinite number of trials to do so and will receive instant feedback by auto-grading. The quiz will be online at least one week before the deadline. Note that the questions will not necessarily be representative of an exam.
Final evaluation: The final exam is a written exam. Only pen, paper, a non-programmable calculator, and two A4 cheat-sheets (i.e., 4 pages) of handwritten content are allowed. For students from M.Sc. MSE / ESE and B.Sc. Math, this exam is graded. Students in the M.Sc. in mathematics program must pass the written exam to take the graded 11ECTS oral exam. Everyone who wants ECTS for this course needs to pass the exam.
Projects: (more detail in a section below) The optional project (3 ECTS) consists of the formulation and implementation of a self-chosen optimization problem or numerical solution method, resulting in documented computer code, a project report, and a public presentation. Project work starts in the last third of the semester. For students in the faculty of engineering, the project is graded independently of the 6 ECTS lecture. For students from the B.Sc. In Math, the grade for the lecture and project 9ECTS module is determined solely by the written exam. For students in the M.Sc. in Mathematics program, the project is again a prerequisite for the graded 11 ECTS oral exam.
Calendar
| Date | Format | Content | Watch this week | Read this week | Prepare | Deadlines | Comments |
| 21.04 | Introduction | ||||||
| 28.04 | Exercise | ||||||
| 05.05 | Q&A | ||||||
| 12.05 | Exercise | ||||||
| 19.05 | Q&A | ||||||
| 26.05 | *** Lecture break *** | ||||||
| 02.06 | Exercise | Complete mid-term quiz by Friday (23.59) | |||||
| 09.06 | Q&A | ||||||
| 16.06 | Exercise | ||||||
| 23.06 | Q&A | ||||||
| 30.06 | Q&A + project discussion | ||||||
| 07.07 | Project work | ||||||
| 14.07 | Project presentation | ||||||
| 21.07 | Exercise | ||||||
| 05.08 | (no session - just a deadline) | project report | project report deadline | ||||
Manuscript
The lectures closely follow a course manuscript draft.
Numerical optimization by Prof. Dr. Moritz Diehl
Lectures
The video recordings correspond to approximately 90 minutes each and comprise 24 lectures in total. A recommended schedule for watching can be found in the calendar above.
| Lecture | Content |
| Lecture 1 | Introduction to Section 1.3 (Mathematical formulation) |
| Lecture 2 | Section 1.4 (Definitions) to 2.7 (Mixed-Integer-Programming) |
| Lecture 3 | Section 3.1 (How to check convexity) to 3.5 (Standard form of convex opt. problems) |
| Lecture 4 | Section 3.6 (Semidefinite Programming) to Example 4.2 (Dual of LP) |
| Lecture 5 | Example 4.3 (Dual decomposition) to Chapter 6 introduction |
| Lecture 6 | Section 6.1 (Linear Least Squares) to 6.5 (L1-Estimation) |
| Lecture 7 | Section 6.6 (Gauss-Newton method) to 7.2 (Local convergence rates) |
| Lecture 8 | Section 7.3 (Newton-Type methods) to 8.1 (Local contraction) |
| Lecture 9 | Section 8.2 (Affine invariance) to 9.1 (Line search) |
| Lecture 10 | Section 9.2 (Wolfe conditions) to 9.3 (Global convergence of line search) |
| Lecture 11 | Section 9.4 (Trust-Region methods) to 9.5 (The Cauchi-Point) |
| Lecture 12 | Section 10.1 (Algorithmic Differentiation) to 10.3 (Backward AD) |
| Lecture 13 | Section 10.4 to Chapter 11 introduction |
| Lecture 14 | Section 11.1 (LICQ and linearized feasible cone) to 11.2 (SONC) |
| Lecture 15 | Section 12.1 (Optimality conditions) to 12.5 (Constrained Gauss-Newton) |
| Lecture 16 | Section 11.3 (Perturbation analysis) and 12.7 (Local convergence) |
| Lecture 17 | Section 12.6 (General constrained NT-Algorithm) to 12.9 (Careful BFGS updating) |
| Lecture 18 | Section 13.1 to 13.2 (Active constraints and LICQ) |
| Lecture 19 | Section 13.3 (Convex Problems) |
| Lecture 20 | Section 13.4 (Complementarity) to 14.1 (QPs via Active Set Method) |
| Lecture 21 | Section 14.2 (SQP) to 14.4 (Interior Point methods) |
| Lecture 22 | Section 14.4 (Barrier problem interpretation, SCP) to 15.5 (Simultaneous optimal control) |
| Lecture 23 | Problem reformulations and useful function approximations |
| Lecture 24 | Summary of the course |
Exercises
The exercises are based partially on pen and paper and partially on Python (see below).
Topic | Questions (pdf) | Material (code) | Solutions (pdf) | Solutions (code) |
| Introduction to CasADi, optimization problems | ||||
| Convexity, duality, fitting problems | ||||
| Unconstrained optimization | ||||
| Equality constraints optimization | ||||
| Inequality constaints optimization (with an exam-type question) |
If you find typos or mistakes in the exercises, please report them to Léo Simpson.
Further Material (Books)
- Jorge Nocedal and Stephen J. Wright, Numerical Optimization, Springer, 2006
- Stephen Boyd and Lieven Vandenberghe, Convex Optimization, Cambridge Univ. Press, 2004
Amir Beck, Introduction to Nonlinear Optimization, MOS-SIAM Optimization, 2014.
Sample exams
We provide two sample exam to prepare for the exam.
Project
Here you will find the project guidelines.
Software
The exercises lean heavily on the open source tool CasADi, which offers an interface to Python, Matlab, and Octave. The computer exercise templates as well as the code solutions will be published in Python only.
Python is one of the major programming languages and via libraries such as NumPy and SciPy a common choice for scientific computing. Apart from CasADi, we will use the libraries NumPy, SciPy, and Matplotlib. If you are completely new to Python, you may want to check out the Anaconda distribution.
CasADi is a symbolic framework for algorithmic differentiation and numerical optimization. In order to install CasADi, follow the instructions here.
If you are using pip, you can simply pip install casadi in your preferred run' pip install casenvironment. If you are using Anaconda / conda for managing your environments, we suggest to first conda install pip inside a conda environment, followed by pip install casadi.
*** This page is intended for the course as taught in the summer semester 2026 at the University of Freiburg. For a timeless version focused on self-study of the material, see here.***