# FMSF15/MASC03: Markov processes, Fall semester 2018, LP1

## News:

• The exercise classes on 19/9 and 20/9 are replaced by lectures. Place MH:Rieszsalen (Matematikhuset, F-building, ground floor)
• Handout for Markov chain model of getting to "Heads" in a row
• lab1.pdf updated
• Stationary distribution of a RW with reflection at zero - see handout
• Extra exercise session at the time and the place of the lecture on the 17 October (for both groups!)
• Poisson point processes (Wiki)
• No more lectures in the course!
• Allowed material on the exam: same as in the sample exam below
• Exam 10/2018 with solutions here

## Teaching plan:

• Lecture 1 [03/09]. Introduction, stochastic processes. (Chap 1, Rydén och Lindgren) Repetition: Random variables, independence, conditional probability.
• Lecture 2 [05/09]. Discrete Markov chains: definition, transition probabilities, Chapman-Kolmogorov equation. (Chap 2.1-2.2)
• Exercises 001, 002, 003, 004, 103, 107, 106, 201, 203

• Lecture 3 [10/09]. Limiting and stationary distributions. Classification of states and chains. (Chap 2.3)
• Lecture 4 [12/09]. More on limiting and stationary distributions. (Chap 2.4)
• Exercises 104, 302, 205, 401, 404

• Lecture 5 [19/09]. Ergodic theorem, law of large numbers. Absorbing state and time to absorption for Markov chains. (Chap 2.5)
• Lecture 6 [20/09]. Failure intensity and life time processes. Discrete Markov processes: definition, transition intensities. (Chap 3.1, 4.1)

• Lecture 7 [24/09]. Chapman-Kolmogorov's forward and backward equations. Waiting times, embedded Markov chains, global balance equations. (Chap 4.1-4.2)
• Lecture 8 [26/09]. M/M/1 queue. Embedded Markov chains. Stationary distribution. (Chap 4.2-4.3)
• Exercises 402, 403, 405, 406, 410, 427, 502, 505, 506, 512, 508

• Lecture 9. Embedded Markov chains. Simulation of Markov chains/processes. Inference.
• Lecture 10. Classification of states. Ergodicity, absorption, Poisson processes. (Chap 3.2, 4.4-4.5)
• Exercises 509, 417, 415, 303, 304, 109, 112, 414, 420, 422, 418

• Lecture 11. Fundamental properties of Poisson processes: waiting times, conditional distributions. (Chap 3.2-3.3, 4.1)
• Lecture 12. Operations on Poisson processes. Non-homogeneous Poisson processes. Spatial and general Poisson processes. Simulation. Inference. (Chap 3.4-3.9)
• Exercises 704, 707, 708, 709, 711, 712, 713, 719, 721

• Lecture 13. Repetition / reserve.
• Exercises Review of old exams and all exercises.
• Exercises Review of old exams and all exercises.