Block Course, 06.10.2025 - 10.10.2025, 9:00-18:00
Lecturers: Prof. Dr. Joschka Boedecker (Uni Freiburg), Prof. Dr. Moritz Diehl (Uni Freiburg), Prof. Dr. Sebastien Gros (NTNU Trondheim)
Exercises: Leonard Fichtner, Andrea Ghezzi and Jasper Hoffmann
Contacts: for any questions feel free to contact us mpcrl@cs.uni-freiburg.de.
Locations:
- Lectures: Kollegiengebäude I, HS 1199, Platz der Universität 3, 79098 Freiburg, Google Maps
- Welcome reception at Waldsee: Waldseestraße 84, 79117 Freiburg im Breisgau, Google Maps
- Dinner at Dattler: Am Schlossberg 1, 79104 Freiburg im Breisgau, Google Maps
Freiburg mobility services:
Freiburg has a reliable tram services and many bike roads.
- You can buy tickets on the tram/bus or from the mobile app of VAG (local transportation company) GooglePlay, AppStore
- You can rent Frelo bikes at the Frelo stations dislocated around the city and you can ride your bike for 30 mins and bring it back to a Frelo station (this service for 6 Euro per month) Frelo website.
Event announcement: MPCRL25
Download our flyer and feel free to share it!
Preliminary timetable
JB: Joschka Boedecker, MD: Moritz Diehl, SG: Sebastien Gros, LF: Leonard Fichtner, AG: Andrea Ghezzi, JH: Jasper Hoffmann, DR: Dirk Reinhardt
| Time | Monday 6.10. | Tuesday 7.10. | Wednesday 8.10. | Thursday 9.10. | Friday 10.10. |
|---|---|---|---|---|---|
| 9:00 | Welcome (~10 min) Lecture 1 - Introduction to RL (JB) Markov Decision Process, Dynamic Programming, Bellman Equation, Policy and Value Iteration | Lecture 4 - Constrained Nonlinear Optimization (MD) Constrained Optimization, MPC as a Nonlinear Program; Sensitivity Computation | Lecture 6 - Synthesis of MPC and RL (DR, JH) Overview over Synthesis of MPC and RL | Lecture 8 - RL + MPC Why does it work? (SG) Overview of the theory that explains why doing RL over MPC works, and why MPC is a great way of building optimal policies. | Lecture 10 AI for Decision Making (SG) ~45min Extension of MPCRL to model-based decision, its consequences in AI for decision, and some recent results. |
| 10:30 | Coffee Break | Coffee Break | Coffee Break | Coffee Break | Coffee Break |
| 11:00 | Lecture 2.1 - Dynamics and Simulation (MD) Lecture 2.2 - Basics in Optimization (AG) | Exercise Acados interface + basic MPC with acados + DiffMPC layer with leap-c | Lecture 7.1 - An MPC prior for SAC (JH, LF) ~45min Lecture 7.2 - Imitation Learning from MPC (AG) ~45min | Project Work / Q&A | Project presentation |
| 12:30 | Lunch Break | Lunch Break | Lunch Break | Lunch Break | Lunch Break |
| 14:00 | Lecture 3.1 - Temporal difference methods and function approximators (JB) ~45min Lecture 3.2 - Dynamic Programming and LQR (MD) ~45min | Lecture 5 - Actor-Critic Methods (JB) ~1h | Exercise ~1h MPC prior for SAC using leap-c Project pitches & Project start-up ~30min | Lecture 9 - Safety & Stability in MPCRL (SG) Overview of some results on safe RL using MPC, stable MPC and RL, and MPCRL with belief state (new research) | Project presentation Final remarks & podium discussion |
| 15:30 | Coffee Break | Coffee Break | Coffee Break | Coffee Break | Farewell Reception |
| 16:00 - 17:30 | Project Guidelines ~10 min Exercise: Pytorch basics and DQN | Exercise ~1h Research Spotlight ~30 min | Project Start-Up / Q&A -- end at 16:45/17:00 Shakeout walk! (17:45/18:00) | Project Work / Q&A | |
| 19:00 | Welcome reception | Dinner with participants |
Lecture slides:
- Introduction ( lecture-0-intro.pdf )
- Lecture 1 - Introduction to Reinforcement Learning (JB) ( lecture-1-intro-to-rl.pdf )
- Lecture 2.1 - Dynamics and Simulation (MD) ( lecture-2-simulation.pdf )
- Lecture 2.2 - Basics in Optimization (AG, MD) ( lecture-2-part2-basics-in-optimization.pdf )
- Lecture 3.1 - Temporal difference methods and function approximators (JB) ( lecture-3.1-td-methods-and-function-approximation.pdf )
- Lecture 3.2 - Dynamic Programming and LQR (MD) ( lecture-3.2-dynamic-programming.pdf )
- Project Instructions ( project_instruction.pdf )
- Lecture 4 - Constrained Nonlinear Optimization (MD) ( lecture-4-nonlinear-optimization.pdf )
- Lecture 5 - Actor-Critic Methods (JB) ( lecture-5-actor-critic.pdf )
- Lecture 6 - Synthesis of MPC and RL (DR) ( lecture-6-synthesis-of-mpcrl.pdf )
- Lecture 7.1 - An MPC prior for SAC (JH) ( lecture-7.1-mpc-sac-prior.pdf )
- Lecture 7.2 - Imitation Learning from MPC (AG) ( lecture-7-2-imitation-learning.pdf )
- Lecture 8 - RL+MPC: Why does it work? (SG) ( lecture-8-rl+mpc-why-does-it-work.pdf )
- Lecture 9 - Safety & Stability in MPCRL (SG) ( lecture-9-safety-stability-mpcrl.pdf )
- Lecture 10 - AI for decisions (SG) ( lecture-10-ai-for-decisions.pdf )
Exercises:
- Exercise 1 - DQN ( exercise-1-dqn.zip ) Solution: exercise-1-dqn-solution.zip )
- Exercise 2 - acados + DiffMPC ( exercise-2-acados-diff-mpc.zip ) Solution: ( exercise-2-acados-diff-mpc-sol.zip )
- Exercise 3 - Policy gradient methods ( exercise-3-reinforce.zip ) Solution: ( exercise-3-reinforce-solution.zip )
- Exercise 4 - SAC-FOP ( exercise-4-sac-fop.zip ) Solution: ( exercise-4-sac-fop-solution.zip )
Software:
Please follow the installation instructions of leap-c to install the necessary resources prior to the school. In particular, please install leap-c at least using the "rendering" package configuration, i.e., when installing leap-c itself run pip install -e .[rendering]
If you encounter problems with the installation, don't worry, we will usually be able to fix the problems at the school.
If you are completely new to python or numpy, we recommend you do at least a small tutorial, e.g., the Python tutorial linked at syscop-msi-tutorial
The website also contains a short guide on how to set up Python along with the IDE VS Code. If you have more time, basic knowledge about PyTorch and CasADi is also useful.
Possible lunch locations during the school
- Mensa Rempartstraße (payment only with mensa card which can is issued at the entrance of the mensa, if you can prove to be a student you will have student prices, approx., 4 Euro per meal)
- Veggie Liebe (falafel), Blauer Fuchs (restaurant)
- Several diners (salads, burgers, kebab, backery) in Niemenstr. GMaps
Some Freiburg Recommendations by Leonard
- Freiburger Münster: Beautiful Gothic cathedral in the heart of the old town. For a small fee, it is also possible to climb the stairs of the tower.
- Schlossberg & Kanonenplatz: A forested hill right next to the old town. A small hike will lead you to panoramic views over Freiburg, with a viewpoint tower on top of the Schlossberg. An even smaller hike will take you to the Kanonenplatz, which is a classic sunset spot with a nice view over Freiburg already. There is also a funicular that will take you to restaurant Dattler.
- Schauinsland: The "city-mountain" of Freiburg. It offers a great view over Freiburg and its surroundings, on clear days you can even see the alps. Usually people would take between half a day and one day of time for a trip there, so this is recommendable if you stay over the weekend, and the weather is sufficiently nice. The foot of the mountain is well-reachable by public transport and from there you can also use a cable car to reach the top.
- Restaurants / Bars:
- Feierling: Nice microbrewery with food, also offers to sit in its Biergarten next to the main building.
- Martinsbräu: Another nice microbrewery with food.
- Kastaniengarten: Nice Biergarten (hence, preferably visit at good weather) at the foot of the Schlossberg.
- One Trick Pony: Awesome cocktail bar. The people there know their work, but it is more expensive than usual.
- O'Kellys / Oscar Wilde's: Good Irish Pubs
- Other generally good bars: Schlappen, Schwarzer Kater, Alter Simon, Uni Café.
- Markthalle: The "market hall" offers a wide variety of foods.
- Unkai: Asia restaurant, in particular offering nice all-you-can-eat sushi. There is one located in the central city, and one to the north-west of Freiburg.
- Dattler: Nice restaurant located on the Schlossberg.
- Kartoffelhaus: Nice potato-themed restaurant.
- Bar Denia: Good restaurant offering tapas.
- Goldener Anker: Good traditional german food.
For details regarding registration checkout the event page MPCRL25
Formal requirements
Relevant only for students of the university of Freiburg.
In order to receive 3 ECTS for this course, students need to pass all of the following:
- Studienleistung (SL, ungraded)
- Participation in the exercise session
- Prüfungsleistung (PL, graded)
- Project report
Every student from the University of Freiburg needs to fill out the registration form.
Please also read the project instructions from above.
On the first day (October 6th), students further need to decide whether they want to commit themselves to do the PL. The registration will take via the PL registration form here.