Study Circle on Robot Learning and Control

Format of the Course

The format of the course is a study circle, meaning that the participants meet once a week to discuss a certain aspect of robot learning and control. Before each meeting, a number of key articles and book chapters will be stated on the course homepage, and one or two accompanied simulation tasks should be prepared for discussion at the meeting. Each week, one of the participants is assigned in advance to prepare a summary of the studied material and present a few slides in the beginning of the meeting.

Course Content

The content of the course includes methods for adaptive control, supervised learning, unsupervised learning, reinforcement learning, learning by human demonstration, iterative learning control and their applications to robot learning and control. In addition, state estimation for robot applications using Kalman filters and particle filters related to robot learning will be discussed. The specific course content will be adapted to the research interests of the course participants.


  • The project seminar is to be held on Friday, December 12 at 13.15-15.00 in the Seminar room M:2112B at the Dept. of Automatic Control. Each project presentation should take approximately 10-15 minutes.
  • The introduction meeting for the course is to be held on Friday, September 12, at 13.15 in the conference room on the ground floor (Lab F, M:1172). The slides from the meeting are available here.



All discussion sessions are held on Thursdays at 10.15-12.00, if not otherwise indicated in the list below.

Week Topics Responsible Literature and Tasks
38 Adaptive robot control Björn [pdf]
39 Artificial neural networks, support vector machines Fredrik [pdf]
40 Gaussian processes, Gaussian process regression (kriging) Mahdi [pdf], [data]
41 Markov chains, Hidden Markov models, maximum likelihood estimation, expectation-maximization algorithm Olof [pdf]
42 Reinforcement learning, Markov decision processes Andreas [pdf]
43 State estimation – Kalman filters and extensions (EKF, UKF), particle filters Martin K [pdf]
  The meeting is held on Wednesday, Oct 22 at 16.00-17.00.    
44 System identification, linear parameter-varying models Fredrik, Björn [pdf]
  The meeting is held on Friday, Oct 31 at 13.15-15.00.    
45 Learning by human demonstration and instruction, segmentation, mixture models Maj [pdf], [data]
  The meeting is held on Thursday, Nov 6 at 08.15-10.00.    
47 Iterative learning control, iterative feedback tuning Olof, Mahdi [pdf]
  The meeting is held on Monday, Nov 17 at 13.15-15.00.    
50 Project seminar Björn  
  The seminar is held on Friday, Dec 12 at 13.15-15.00.    



Name Project Title
Olof Sörnmo Feature-based Q-learning for Path-Planning Machining Operations
Fredrik Bagge Carlson Particle Model Predictive Control for Reinforcement Learning (PMPC-RL)
Maj Stenmark Extraction of Constraints for Guarded Motions from Demonstrations
Martin Karlsson Positioning Using IMU Measurements and Image Analysis
Andreas Stolt Implementation and Experimental Validation of a Reinforcement Learning Algorithm
Björn Olofsson Identification of LPV Systems Using Subspace-Based Methods



Active participation in the weekly meetings is required. Further, the course will end with a project, preferably related to the participant's own research, which will be presented to the rest of the group at a seminar. The course gives 7.5 hp.


Anders Robertsson ( is the examiner of the course. For questions regarding the course, please contact Björn Olofsson (