Modelling and System Identification

Prof. Moritz Diehl, Jakob Harzer, Katrin Baumgärtner

 

Modelling and System Identification (MSI) is concerned with the search for mathematical models for real-life systems. The course is based on statistics, optimization and simulation methods for differential equations. The exercises will be based on pen-and-paper exercises and computer exercises with python.

Course language is English and all course communication is via this course homepage.

If you have any questions regarding the exercises/lectures, please send an email to the tutors, syscop.msi@gmail.com


Lectures. The lectures will take place on Mondays, 8:10 - 10:00 a.m and Wednesdays, 9:00-10:00 a.m., in Building 101, HS 036. If you cannot attend, you may watch the lecture recordings, see below.


Exercises. The exercise sheets include both pen-and-paper exercises as well as programming exercises using python. Exercise sheets can be handed in during the lecture on Monday. Programming exercises are handed in via Ilias. You have one week to work on the sheet and you might work in groups of at most three students.

Exercise Sessions are on

  • Tuesday, 15:15-16:15, building 102, room 02-011
  • Wednesday, 10:00-11:00, building 78, room SR 00 014
  • Wednesday, 10:00-11:00, building 102, room 02-012

During the exercise session, the exercise solutions are discussed. Afterwards there is room for questions on the current exercise sheet.

Written material. The lecture closely follows the script, which can be found below:

  • Lecture notes on Modelling and System identification: PDF

If you missed the first lecture, you can pick up a printout of the script in Building 102, in the cupboard in front of room 00 075.

Please note that we do not cover Chapter 8.4. Additional material that covers some of the lecture contents:

  • A script by Johan Schoukens (Vrije Universiteit Brussel, Belgium), which can be found here.
  • The textbook Ljung, L. (1999). System Identification: Theory for the User. Prentice Hall. This book is available in the faculty library.

Final Evaluation and Microexams

Please make sure you register for both the MSI Exam and the MSI Studienleistung!

The final grade of the course is based solely on a final written exam at the end of the semester. The final exam is a closed book exam, only pencil, paper, and a calculator, and two handwritten double-sided A4 sheets of self-chosen formulae are allowed.

Each exercise sheet gives a maximum of 10 points. Three online microexams written during some of the lecture slots give a maximum of 10 exercise points each. In order to pass the exercises accompanying the course (Studienleistung), one has to obtain at least 50% of the maximum exercise points in each of the three blocks:

  • Block 1: Exercises 1 - 3 + Microexam 1 (total 40 points)
  • Block 2: Exercises 4 - 6 + Microexam 2 (total 40 points)
  • Block 3: Exercises 7 - 10 + Microexam 3 (total 40 points + 10 Bonus Points)

To prepare for the written exam, check out the exams from previous semesters: 2018, 2015, 2014. (Please note that these exams contain questions on Appendix C of the MSI script, which is not covered in this year's lecture)

Solution video for the 2018 exam


Lectures and Microexams
 

Monday, October 17 Intro session
Wednesday, October 19 Lecture
Monday, October 24 Lecture
Wednesday, October 26 Lecture
Monday, October 31 Lecture
Wednesday, November 2 Lecture
Monday, November 7 Lecture
Wednesday, November 9 Lecture
Monday, November 14 Lecture
Wednesday, November 16 Lecture
Monday, November 21 Lecture
Wednesday,  November 23 Lecture
Monday, November 28 Microexam online, 9:00-10:00
Wednesday, November 30

Talk by Andrea Ghezzi: "A real estimation problem in the steel industry" 
Talk will be streamed to the lecture hall or can be joined online.

Monday, December 5 Lecture
Wednesday, December 7 Lecture
Monday, December 12 Lecture
Wednesday, December 14 Lecture
Monday, December 19 Lecture
Wednesday, December 21 Microexam online, 9:00-10:00
CHRISTMAS BREAK  
Monday, January 9 Lecture
Wednesday, January 11 Lecture
Monday, January 16 Lecture
Wednesday, January 18 Lecture
Monday, January 23 Lecture: Covariance, Continuous Time, Numerical Integration, + EKF Drone Example (by Mohammed Hababeh)(cancelled, to be resceduled)
Wednesday, January 25 Microexam online, 9:00-10:00
Monday, January 30

Lecture: Explanation of Past Exam (real-life and recorded)  +

EKF Lighthouse CrazyFlie Drone Example by Mohammed Hababeh

Wednesday, February 1 Guest Lecture by Moritz Berger “Bayesian Sensor Calibration” (Link to Paper Bayesian Sensor Calibration)
Monday, February 6 Lecture: Explanation of Past Exam (real-life and recorded)  + Rotokite Demo (by Paul Krüger and Simon Hettich)
Wednesday, February 8 Lecture: Past Exam (cont., recorded) and Summary

Lecture Recordings

date topic chapters
October 17 - October 21 Lecture 1: Introduction + Resistance Estimation

1-1.2

October 24 - October 28 Lecture 2: Resistance Estimation + Statistic Basics

1.2.2-2.3

October 24 - October 28 Lecture 3: Random Variables + Statisitical Estimators

2.3-2.4

October 31 - November 4 Lecture 4: Resistance Estimation Revisited

2.5-3.1

November 7 - November 11 Lecture 5: Optimization Basics + Linear Least Squares

3.1-4.2

November 7 - November 11 Lecture 6: WLS + Ill-posed Problems

4.3-4.4.1

November 14 - November 18 Lecture 7: Statistical Analysis of WLS

4.5-4.7

November 14 - November 18 Lecture 8: Maximum Likelihood Estimation

5-5.1.1

November 21 - November 25 Lecture 9: MAP Estimation + Recursive LLS

5.2-5.3.2

December 5 - December 9 Lecture 10: Cramer Rao Bound
(the part on Section 5.4: Cramer Rao Bound starts at 38min)
(Section 5.4.1: Proof of Cramer Rao Bound. Note that the proof is not required for the exam)

5.3-5.4

December 12 - December 22 Lecture 11: Practical Solution of NLS

5.5.

January 9 - January 13 Lecture 12: Dynamic systems (Part1, Part2, Part3) (2,5 hours in total)

6.1-6.1.2

January 16 - January 20 Lecture 13: Output and Equation Errors (1h)

7.1-7.3

January 23 - January 27 Lecture 14: State Space Models (0,5h)

7.4

January 23 - February 27 Lecture 15: RLS + Kalman Filter (1h)

9.1-9.3

January 30 - February 3 Lecture 16: Extended Kalman Filter (1h)

9.5

January 30 - February 3 Lecture 17: Moving Horizon Estimation (1,5h)

9.6


Exercises Sheets

Sheet Material Deadline
Sheet 0: Intro   October 24
Sheet 1: Resistance Estimation Example Material Ex 1 October 31
Sheet 2: Statistics + Parameter Estimation Material Ex2 November 7
Sheet 3: Optimality Conditions and Linear Least Squares Material Ex3 (corrected) November 14
Sheet 4: Weighted Linear Least-Squares Material Ex4 November 21
Sheet 5: Ill-Posed Linear Least-Squares & Regularization Material Ex5 (corrected) December 5
Sheet 6: Maximum Likelihood and MAP Estimation Material Ex6 December 12
Sheet 7: Recursive Least Squares Material Ex7 January 9
Sheet 8: Nonlinear Least Squares Material Ex8 January 16
Sheet 9: Kalman Filter Material Ex9 January 30
(BONUS) Sheet 10: Extended Kalman Filter Material Ex10 February 6

Tutorials

In the first week, there is no mandatory exercise sheet, but if you don't feel too confident about your linear algebra and statistics skills, you might want to check out these tutorials that cover the basics needed for the MSI course.

  1.  Python Tutorial - for more information see the paragraph below
  2.  Linear Algebra Tutorial
  3.  Statistics Tutorial

In the second week, the tutors will discuss the solutions to the tutorials in the exercise sessions.


Python

For the programming exercises we use Python. To work on the exercises please make sure to have Python installed on your system.

Python Installation

Here is a short guide on how to set up Python along with the IDE VS Code. If you already have Python installed on your system or want to use another IDE, feel free to skip to bullet 4.

  1. Install Python for your operating system

  2. Install VS Code

  3. Install the Python Extension for VS Code

  4. Install the required python packages:

    • Start a terminal

    • Type and press enter:

      pip install numpy scipy matplotlib pytest
Python Tutorial Notebooks

For people who do not know Python or want to refresh their knowledge, we provide a series of Jupyter notebooks to give you an introduction to data science programming in python. More resources, such as video tutorials for Python can be found online.

  1. Download and unzip the Tutorial Notebooks into a folder of your choice

  2. Open the folder in VS Code (File -> Open Folder)

  3. Open the first Notebook by going through the file tree (notebooks/1-basics/PY0101EN-1-1-Hello.ipynb)

Blackboard Photos

First Draft of Complete Rotokite Model Equations (9.11.2023) IMG_5512.HEIC.pdf