FMSF15/MASC03: Markov processes, fall semester 2012

Here you will find current information for the fall semester 2012.

• (3/9) The slides of the first lecture are now available under the link below. Well met on Wednesday in K:F!
• (5/9) The slides of the 2nd lecture are now available under the link below. Well met on Monday in K:B!
• (11/9) The slides of the 3nd lecture are now available under the link below. Note that the locations of the exercise sessions have been changed: this week, these are held in MH:333 and MH:332B on Tuesday and MH:332A and MH:332B on Thursday.
• (12/9) The slides of the 4th lecture are now available under the link below. These also contains complete information concerning new locations of the exercise sessions (there have been changes in the schedule). The small Matlab routine used at the lecture for simulating and plotting Markov chain trajectories is found here. The program uses the routine randoind.m, found here.
• (17/9) The slides of the 5th lecture are now available under the link below. The proof of the last theorem will be presented next time.
• (19/9) The slides of the 6th lecture, including a now corrected expression of the Gompertz-Makeham intensity, are now available under the link below.
• (24/9) The slides of the 7th lecture are now available under the link below.
• (26/9) The slides of the 8th lecture are now available under the link below.
• (27/9) Important: the obligation to sign up in advance for the computer labs has been skipped (since there will be only a single occasion per lab). Thus, you just have to show up in MH:230 or MH:231 at the timetabled occasion.
• (28/9) A first, preliminary part of the list of examination assignments is now available here. Use the assignments to prepare for the exam. The complete and final list of assignments will be posted on this page at least one week before the exam.
• (1/10) The slides of the 9th lecture are now available under the link below.
• (3/10) The slides of the 10th lecture are now available under the link below.
• (10/10) The slides of the 11th and 12th lectures are now available under the link below. A number of old exams can be found here (see the bottom of the page). Note that a few of the problems of these exams deal with concepts that have not been treated during the current realization of the course; ask the lecturer if you are unsure. One of these old exams, that from 2006-05-31, will be discussed next week.
• (12/10) The complete list of examination assignments is now available here. Use the assignments to prepare for the exam. The assignments will be discussed next week.
• (15/10) We have now started to recapitulate the course. The slides (of the 13th lecture) are found below. The recapitulation will continue next time and the plan is to also discuss the exam from 2006-05-31. Important: the exam will take place on Monday 22, 08:00-13:00, in VIC:3A (30 places: Anje A-Karin H) and VIC:2D (the rest of the students).
• (5/11) The exam on 2012-10-22 is now marked. Results, which were very good as a whole (the lecturer is proud!), are found here. The marked exams will be shown on Wednesday 21 November, 12:00-12:30, in MH:227. Some suggestions for solutions can be found here and the exam itself here. The re-exam is on Friday 11 January, 2013, 14:00-19:00, in Sparta:C.

Course contents

Discrete Markov chains and Markov processes; classification of states and chains/processes; stationary distributions and convergence; absorbing states and absorption times; simulation and inference; Poisson processes on the real line and more general spaces; additional material.

Lectures

Course administrator and lecturer

Plan for lectures (preliminary!)

Mon 3/9	Hall M:D	Introduction, stochastic processes (Ch 1). Review: Random variables, independence, conditional probability, law of total probability, decomposition of joint probabilities.	slides
Wed 5/9	Hall K:F	Discrete Markov chains: definition, transition probabilities, Chapman-Kolmogorov equation. (Ch 2.1-2.2)	slides
Mon 10/9	Hall K:B	Stationary distributions and chains, global and local balance. (Ch 2.3)	slides
Wed 12/9	Hall K:F	Classification of states and chains, the ergodic theorem. (Ch 2.4)	slides
Mon 17/9	Hall K:B	Absorbing states and absorption times for Markov chains. (Ch. 2.5)	slides
Wed 19/9	Hall K:F	Simulation of Markov chains. The intensity concept. Discrete Markov processes: definition. (Ch 2.6-2.7,3.1,4.1)	slides
Mon 24/9	Hall K:B	Transition intensities, Kolmogorov's forward and backward equations. (Ch. 4.1-4.2)	slides
Wed 26/9	Hall K:D	Waiting times, embedded Markov chains. Stationarity, global and local balance. (Ch 4.2-4.3)	slides
Mon 1/10	Hall K:B	The cutting method.	slides
Wed 3/10	Hall K:D	Classification of states and chains for Markov processes. Absorbing states and absorption times for Markov processes. Simulation. Introduction to the Poisson process. (4.4-4.10, 3.1)	slides
Mon 8/10	Hall K:B	Fundamental properties of Poisson processes: recurrence times, conditional distributions. (Ch 3.3-3.4)	slides
Wed 10/10	Hall K:D	Non-homogeneous Poisson processes, operations on Poisson processes. Spatial and general Poisson processes. (Ch. 3.4-3.6)	slides
Mon 15/10	Hall K:B	Recapitulation.	slides
Wed 17/10	Hall K:D	Recapitulation. Review of some old exams.

Exercises

Answers and hints can be found here. These will be updated as the course progresses.

The following problems will be treated during the exercise classes. The list is still preliminary and may be updated as the course evolves. All students are encouraged to do all exercises and to be prepared to discuss the solutions in class.

Week 1	E1, E2, E3, 001, 002, 003, 004.
Week 2	103, 107, 106, 201, 203, 302, 205, E4.
Week 3	104, 302, 401, 402, 403, 404, 405, 406, 410, 427, E5.
Week 4	502, 505, 506, 512, 508.
Week 5	303, 304, 109, 112, 414, 417, 415, 418, 420, 422, 509.
Week 6	704, 707, 708, 709, 711, 712, 713, 721,
Week 7	Old exams. General discussion of the course, questions, problems, etc.

Computer labs

LAB 1 (Thursday October 4, 13:15-16:00, MH:230 and MH:231). Instructions for the lab are available in pdf format here.

Matlab files needed for the first lab: knapp.m, monopgata.m, pestimering.m, seep.m, eigv.m, move.m, simulering.m, field.m, monop.m, pchk.m, psimulering.m.

LAB 2 (Thursday October 11, 13:15-16:00, MH:230 and MH:231). Instructions for the lab are available in pdf form here.

Matlab files needed for the second lab: porand.m, coal.dat, inhom_poisson_lambda.m, inhom_poisson_simulate.m, inhom_poisson_deriv.m, inhom_poisson_est.m.

Table of formulas

1. A table of formulas that may be used at the exam is available here.
2. A collection of basic formulas and tables of probability theory and statistics is found here.

Scheduled exams

• Monday October 22, 2012, 08:00-13:00. The place will be announced later.
• Friday January 11, 2013, 14:00-19:00, in Sparta:C. Note: pre-registration for this exam is compulsory. To sign up for the exam send an e-mail to the course secratary Maria Lövgren, e-mail: marial@maths.lth.se.