Modelling and System Identification

Prof. Moritz Diehl, Jakob Harzer, Katrin Baumgärtner

Modelling and System Identification (MSI) is concerned with the search for mathematical models for real-life systems. The course is based on statistics, optimization and simulation methods for differential equations. The exercises will be based on pen-and-paper exercises and computer exercises with python.

Course language is English and all course communication is via this course homepage.

If you have any questions regarding the exercises/lectures, please send an email to the tutors,

Lectures. The lectures will take place on Mondays, 8:30 - 10:00 a.m in Building 101, HS 026 and Wednesdays, 8:30-10:00 a.m. in Building 101, HS 036. If you cannot attend, you may watch the lecture recordings, see below.

Exercises. The exercise sheets include both pen-and-paper exercises as well as programming exercises using python. Exercise sheets can be handed in before the lecture on Monday, 8:30 or put into the Mailbox in front of room 00-075 in the ground floor of building 101 before. Programming exercises are handed in via Github Classrooms (more on that below). You have one week to work on the sheet and you might work in groups of at most three students.

Exercise Sessions. During the exercise session, the tutors disscuss the exercise solutions. Afterwards there is room for questions on the current exercise sheet.

  • Group 1: Wednesday, 14:15 - 15:45, SR 00-031, Building 051, Premnath
  • Group 2: Thursday, 10:15 - 11:45, SR 01-009/13, Building 101, Yizhen
  • Group 3: Thursday, 14:15 - 15:45, SR 03-026, Building 051, Adithya

Written material. The lecture closely follows the script, which can be found below:

  • Lecture notes on Modelling and System identification: PDF

If you missed the first lecture, you can pick up a printout of the script in Building 102, in the cupboard in front of room 00 075.

Please note that we do not cover Chapter 8.4. Additional material that covers some of the lecture contents:

  • A script by Johan Schoukens (Vrije Universiteit Brussel, Belgium), which can be found here.
  • The textbook Ljung, L. (1999). System Identification: Theory for the User. Prentice Hall. This book is available in the faculty library.

Summary Slides of the Course Organization

Final Evaluation and Microexams

Please make sure you register for both the MSI Exam and the MSI Studienleistung!

The final grade of the course is based solely on a final written exam at the end of the semester. The final exam is a closed book exam, only pencil, paper, and a calculator, and two handwritten double-sided A4 sheets of self-chosen formulae are allowed. It will take place on 21.03.2024, 14:00 -16:30.

Each exercise sheet gives a maximum of 10 points. Three microexams written during some of the lecture slots give a maximum of 10 exercise points each. In order to pass the exercises accompanying the course (Studienleistung), one has to obtain at least 50% of the maximum exercise points in each of the three blocks:

Summary PDF

In a microexam, you will have 45 minutes for 20 multiple-choice questions on the lecture and exercises of the respective block. 

To prepare for the written exam, check out the exams from previous semesters: 2019, 2018, 2015, 2014. (Please note that these exams contain questions on Appendix C of the MSI script, which is not covered in this year's lecture)

Solution video for the 2018 exam


Additionally, we will offer a Q&A Session on Thursday, 14.03, 10:00-12:00, Building 101, Room SR 01-016/18.

Lectures and Microexams

Monday, October 16Introduction Lecture
Wednesday, October 18Lecture
Monday, October 23Linear Algebra Tutorial 
Wednesday, October 25Lecture
Monday, October 30Statistics Tutorial 
Wednesday, November 1No Lecture
Monday, November 6Lecture
Wednesday, November 8Lecture
Monday, November 13Lecture
Wednesday, November 15Lecture
Monday, November 20Lecture
Wednesday,  November 22Lecture
Monday, November 27Lecture
Wednesday, November 29Microexam 1, 8:30 - 9:10, lecture from 9:15
Monday, December 4Lecture
Wednesday, December 6Lecture
Monday, December 11Lecture
Wednesday, December 13Lecture
Monday, December 18Lecture
Wednesday, December 20Microexam 2, 8:30 - 9:10, lecture from 9:15
Monday, January 8Lecture
Wednesday, January 10Lecture
Monday, January 15

Lecture, Application of Recursive Least Squares or KF 

in Steel Recycling, Photo1, Photo2

Wednesday, January 17Lecture (on Modelling Temperature Profiles below Icy Roads)
Monday, January 22Lecture on Quadratic Arrival Cost in MHE and KF Example. Blackboard Photos: , IMG_8532.jpg, IMG_8533.jpg, IMG_8534.jpg, IMG_8535.jpg, IMG_8536.jpg
Wednesday, January 24Talk by Wadi Abdurahim, Blackboard Photo on ML estimation of mean and variance: IMG_8537.jpg
Monday, January 29Talk by Leo Simpson, slides
Wednesday, January 31Microexam 3, 8:30 - 9:10, Talk by Moritz Berger afterwards
Monday, February 5Summary Lecture + Coffee
Wednesday, February 7Lecture: Past Exam Solution


In the first weeks, there is no mandatory exercise sheet, but if you don't feel too confident about your linear algebra and statistics skills, you might want to check out these tutorials that cover the basics needed for the MSI course.

  1.  Python Tutorial - for more information see the paragraph below
  2.  Linear Algebra Tutorial
  3.  Statistics Tutorial

In the first weeks, we will discuss these tutorial also in the lecture and the exercises.

Exercises Sheets

SheetGH Classroom LinkDeadline
Sheet 0: Intro (optional)Click to Accept Exercise 0October 23
Sheet 1: Resistance Estimation ExampleClick to Accept Exercise 1October 30
Sheet 2: Statistics + Parameter EstimationClick to Accept Exercise 2November 6
Sheet 3: Optimality Conditions and Linear Least SquaresClick to Accept Exercise 3November 20
Sheet 4: Regularized, Ill-Posed and Weighted Linear Least-SquaresClick to Accept Exercise 4November 27
Sheet 5: Maximum Likelihood and MAP EstimationClick to Accept Exercise 5December 4
Sheet 6: Recursive Least SquaresClick to Accept Exercise 6December 11
Sheet 7: Dynamic SystemsClick to Accept Exercise 7January 8
Sheet 8: Nonlinear Least SquaresClick to Accept Exercise 8January 15
Sheet 9: Kalman FilterClick to Accept Exercise 9January 29
(BONUS) Sheet 10: Extended Kalman FilterClick to Accept Exercise 10February 5
(Bonus) Bathtub Example, Data  





Github Classroom To distribute, collect and grade your coding exercises, we utilise Github Classroom. This service provides, for your exercise team, a Github repository for each exercise. If you have never worked with Git before, here we have linked a tutorial.

To use Github Classroom, every students need a github account. If you don't have one, or don't want use your private one, feel free to create a fake account just for this course. To identify your coding score with you immatriculation number, please register both your immatriculation number and your Github profile with us here. OTHERWISE YOU WILL NOT GET POINTS FOR YOUR CODING EXERCISES.

The exercise workflow looks as follows:

  1. Accept the exercise for your team above.
  2. Clone the exercises locally onto your computer.
  3. Work on the exercise, by executing the scripts
  4. Check your solutions by running


    in the console.

  5. Commit & push your solutions
  6. The same tests (with slightly different input data) are rerun on the server.

Python For the programming exercises we use Python. To work on the exercises please make sure to have Python installed on your system.

Python Installation

Here is a short guide on how to set up Python along with the IDE VS Code. If you already have Python installed on your system or want to use another IDE, feel free to skip to bullet 4.

  1. Install Python for your operating system
  2. Install VS Code
  3. Install the Python Extension for VS Code
  4. Install the required python packages:
Python Tutorial Notebooks

For people who do not know Python or want to refresh their knowledge, we provide a series of Jupyter notebooks to give you an introduction to data science programming in python. More resources, such as video tutorials for Python can be found online.

  1. Download and unzip the Tutorial Notebooks into a folder of your choice

  2. Open the folder in VS Code (File -> Open Folder)

  3. Open the first Notebook by going through the file tree (notebooks/1-basics/PY0101EN-1-1-Hello.ipynb)



Lecture Recordings

October 18 - October 20Lecture 1: Introduction + Resistance Estimation1-1.2
October 24 - October 28Lecture 2: Resistance Estimation + Statistic Basics1.2.2-2.3
October 24 - October 28Lecture 3: Random Variables + Statisitical Estimators2.3-2.4
October 31 - November 4Lecture 4: Resistance Estimation Revisited2.5-3.1
November 7 - November 11Lecture 5: Optimization Basics + Linear Least Squares3.1-4.2
November 7 - November 11Lecture 6: WLS + Ill-posed Problems4.3-4.4.1
November 14 - November 18Lecture 7: Statistical Analysis of WLS4.5-4.7
November 14 - November 18Lecture 8: Maximum Likelihood Estimation5-5.1.1
November 21 - November 25Lecture 9: MAP Estimation + Recursive LLS5.2-5.3.2
December 5 - December 9Lecture 10: Cramer Rao Bound
(the part on Section 5.4: Cramer Rao Bound starts at 38min)
(Section 5.4.1: Proof of Cramer Rao Bound. Note that the proof is not required for the exam)
December 12 - December 22Lecture 11: Practical Solution of NLS5.5.
January 9 - January 13Lecture 12: Dynamic systems (Part1, Part2, Part3) (2,5 hours in total)6.1-6.1.2
January 16 - January 20Lecture 13: Output and Equation Errors (1h)7.1-7.3
January 23 - January 27Lecture 14: State Space Models (0,5h)7.4
January 23 - February 27Lecture 15: RLS + Kalman Filter (1h)9.1-9.3
January 30 - February 3Lecture 16: Extended Kalman Filter (1h)9.5
January 30 - February 3Lecture 17: Moving Horizon Estimation (1,5h) 



Maximum Likelyhood Visualization


We sample measurements (green points on x axis) from a distribution with unknown mean. The y value of the red dots correspond to the probability of the correspondent measurement. 

Drag the border of the equation window to the right to maximize the graph window. Play around with the slider to find the value for the mean that minimizes the likelyhood function.