FMSF15/MASC03: Markov processes, Fall semester 2018, LP1
OBS! This page is constantly updated: last modified on: 9 November 2018
News:
 The exercise classes on 19/9 and 20/9 are replaced by lectures.
Place MH:Rieszsalen (Matematikhuset, Fbuilding, ground floor)
 Handout for Markov chain model of getting to "Heads" in a row
 lab1.pdf updated
 Stationary distribution of a RW with reflection at zero  see handout
 Extra exercise session at the time and the place of the lecture on the 17 October (for both groups!)
 Poisson point processes (Wiki)
 No more lectures in the course!
 Allowed material on the exam: same as in the sample exam below
 Exam 10/2018 with solutions here
General information:
 Course contents: Discrete Markov chains and Markov processes. Classification of states and chains/processes. Stationary distributions and convergence. Absorbing states and absorption times. Simulation and inference. The Poisson processes on the real line and more general spaces. Additional material.
Formal LTH course syllabus
 Lectures: The lectures will be given in English.
 Instructor: Stas Volkov
 All administrative inquires: Susann Nordqvist (registration for the course, exams etc.)
 Exercises: (instructors: Samuel Wiqvist and Per Niklas Waaler)
 The exercises can be downloaded here.
 Answers and hints can be found here.
 Computer assignments: The following computer laboratory sessions are compulsory part of the course. There are two groups for each computer session and you need to register in one group for each computer session at SAM
lab enroll.
In order to sign up for a group you will need your 'STIL'account so make sure you have uptodate information on your account before the registration. Computer sessions: see schedule.
 Instructions for the computer assignment: lab1.pdf.
 Matlab files needed for the first lab:
knapp.m,
monopgata.m,
pestimering.m,
seep.m,
eigv.m,
move.m,
simulering.m,
field.m,
monop.m,
pchk.m,
psimulering.m.
 Instructions for the computer assignment: lab2.pdf.
 Matlab files needed for the second lab:
porand.m,
coal.dat,
inhom_poisson_lambda.m,
inhom_poisson_simulate.m,
inhom_poisson_deriv.m,
inhom_poisson_est.m.
 Table of formulas
 The table of formulas, which may be used at the exam, is available here.
 Collection of formulas and tables for the course in probability theory and statistics: pdf.
 Exams:
Please check upcoming exams in the Centre for Mathematical Sciences or Lund University's exam schedule TimeEdit for scheduled exams.
 Past exams:
exam and solution.
Teaching plan:

Lecture 1 [03/09]. Introduction, stochastic processes. (Chap 1, Rydén och Lindgren) Repetition: Random variables, independence, conditional probability.

Lecture 2 [05/09]. Discrete Markov chains: definition, transition probabilities, ChapmanKolmogorov equation. (Chap 2.12.2)

Exercises 001, 002, 003, 004, 103, 107, 106, 201, 203
 Lecture 3 [10/09]. Limiting and stationary distributions. Classification of states and chains. (Chap 2.3)
 Lecture 4 [12/09]. More on limiting and stationary distributions. (Chap 2.4)
 Exercises 104, 302, 205, 401, 404
 Lecture 5 [19/09]. Ergodic theorem, law of large numbers. Absorbing state and time to absorption for Markov chains. (Chap 2.5)
 Lecture 6 [20/09]. Failure intensity and life time processes. Discrete Markov processes: definition, transition intensities. (Chap 3.1, 4.1)
 Lecture 7 [24/09]. ChapmanKolmogorov's forward and backward equations. Waiting times, embedded Markov chains, global balance equations. (Chap 4.14.2)
 Lecture 8 [26/09]. M/M/1 queue. Embedded Markov chains. Stationary distribution. (Chap 4.24.3)
 Exercises 402, 403, 405, 406, 410, 427, 502, 505, 506, 512, 508
 Lecture 9. Embedded Markov chains. Simulation of Markov chains/processes. Inference.
 Lecture 10. Classification of states. Ergodicity, absorption, Poisson processes. (Chap 3.2, 4.44.5)
 Exercises 509, 417, 415, 303, 304, 109, 112, 414, 420, 422, 418
 Lecture 11. Fundamental properties of Poisson processes: waiting times, conditional distributions. (Chap 3.23.3, 4.1)
 Lecture 12. Operations on Poisson processes. Nonhomogeneous Poisson processes. Spatial and general Poisson processes. Simulation. Inference. (Chap 3.43.9)
 Exercises 704, 707, 708, 709, 711, 712, 713, 719, 721
 Lecture 13. Repetition / reserve.
 Exercises Review of old exams and all exercises.
 Exercises Review of old exams and all exercises.