FMS180/MAS204: Markov processes, autumn 2002

Here you will find current information for the autumn semester 2002. This information in entirely in English, as we expect that several exchange students will take the course.

Lectures

The lectures take place Tuesdays 15.15-17.00 in hall MA:5 and and Thursdays 15.15-17.00 in hall MA:1.

Exercise classes

The exercise classes takes place Fridays at 10.15-12.00 (room MH:362D) and 13.15-15.00 (room MH:331) (there are two groups). Our intention is that you -- the students -- will work through certain exercises in advance, and that the solutions will be discussed during these classes. This creates a high degree of student participation.

The teacher in MH:362D will be Mats Pihlsgård, and this class will be held in Swedish. The teacher in MH:331 will be Tobias Rydén, and this class will be bilingual, that is, held in English/Swedish.

Course contents

Preliminary contents: more details may be added.

Discrete Markov chains and Markov processes. Classification of states and chains/processes. Stationary distributions and convergence. Absorbing states and absorption times. Simulation and inference. The Poisson processes on the real line and more general spaces. Markov random fields: introduction, Gibbs fields, simulation, some inference. Hidden Markov models: definition, inference, the EM algorithm. Various applications.

Teaching plan

Tu 29/10 Introduction, stochastic processes. Discrete Markov chains: definition, transition probabilities (Ch 1, 2.1-2.2). Discrete Markov processes: definition, transition intensities, waiting times, embedded Markov chain (Ch 4.1, parts of 4.2). Lack of memory of the exponential distribution (Ch 3.1).

Th 31/10 Modelling with Markov chains and processes (Ch 4.1). The Chapman-Kolmogorov equations (Ch 3.3, 4.2).

Tu 5/11 Absolute probabilities (Ch 2.2, 4.2). Stationarity, classification of states and chains (Ch 2.3, 2.4).

Th 7/11 Stationarity, classification of states and chains for Markov processes (Ch 4.3, 4.4). Birth-and-death processes (Ch 4.6 4.7).

Tu 12/11 Absorbing states and absorption times (Ch 2.5, 4.5).

Th 14/11 Inference, simulation (Ch 2.7, 2.8, 4.9, 4.10).

Tu 19/11 The Poisson process: characterisations, the Poisson limit (law of small numbers), recurrence times, conditional distributions (Ch 3.2-3.3).

Th 21/11 Non-homogeneous Poisson processes, operations on Poisson processes, spatial and general Poisson processes, inference (Ch 3.4-3.8).

Tu 26/11 Markov random fields: definition, examples, local specification, Gibbs distributions (lecture notes here: PostScript or pdf format).

Th 28/11 Markov random fields: Markov chain Monte Carlo simulation, pseudo-likelihood (lecture notes as above).

Tu 3/12 Hidden Markov models: definition, examples. Filtering and smoothing, the forward-backward algorithm. Gibbs distributions (lecture notes here: PostScript or pdf format). These notes have now been updated. Pages 11-13 differ from the previous version.

Th 5/12 Hidden Markov models: parameter estimation and the EM algorithm (lecture notes as above).

Tu 10/12 Seminar on perfect simulation of Markov random fields and (hopefully!) an application of Markov chain modelling to analysis of DNA. This is not part of the course curriculum.

Th 12/12 No class.

Exercises

The exercises listed below are those that students are asked to solve before the exercise classes and that will have priority for discussion during the classes. Other exercises may be of course also be discussed on request.

The exercises themselves can be downloaded here in PostScript or pdf format, and answers and hints can be downloaded here in PostScript or pdf format.

Week 1: 101, 102, 104, 105, 201, 203.

Week 2: 301, 302, 403, 404, 405, 409, 412.

Week 3: 501, 502, 503, 601, 602.

Week 4: 701, 704, 706, 707, 708, 709, 714.

Week 5: 801, 802, 803.

Week 6: 901, 902.

Week 7: Course overview, opportunity to ask questions, solution of old exam problems, ...

OBS! Övningen fredagen den 6:e december 10-12 i MH:362D är inställd pga sjukdom!

Computer lab

The first lab will be given during study week 4 (18-22 November). There are three time slots scheduled for the lab (Tuesday morning, Tuesday night, Thursday morning), and you choose one of them by signing up at the lists that are posted on the department's billboard in the lobby in the math buildning (course code FMS180). Instructions for the first lab are available here in PostScript or pdf format.

Matlab files needed: knapp.m, monopgata.m, pestimering.m, seep.m, eigv.m, move.m, simulering.m, field.m, monop.m, pchk.m, psimulering.m.

The second lab will be given during study week 6 (2-6 December). There are three time slots scheduled for the lab (Monday afternoon, Tuesday night, Thursday morning), and you choose one of them by signing up at the lists that are posted on the department's billboard in the lobby in the math buildning (course code FMS180). Instructions for the second lab are available here in PostScript or pdf format.

Matlab files needed: coal.dat, porand.m, riodej.mat, icm.m.

Table of formulas

The table of formulas, which may be used at the exam, is available here in PostScript or pdf format (now updated!).

Exam

Written exam Wednesday 18 December, 14-19 hrs in hall MA:10A-C.
Students taking MAS204 must also do an oral examination (munta).

At the exam you may use the table of formulas (see above), a table of formulas from the basic course in mathematical statistics, and a pocket calculator without pre-stored information (such as formulas, programs,...).

Tobias Rydén

Last modified: Mon Oct 20 13:16:39 MEST 2003

Tu 29/10	Introduction, stochastic processes. Discrete Markov chains: definition, transition probabilities (Ch 1, 2.1-2.2). Discrete Markov processes: definition, transition intensities, waiting times, embedded Markov chain (Ch 4.1, parts of 4.2). Lack of memory of the exponential distribution (Ch 3.1).
Th 31/10	Modelling with Markov chains and processes (Ch 4.1). The Chapman-Kolmogorov equations (Ch 3.3, 4.2).
Tu 5/11	Absolute probabilities (Ch 2.2, 4.2). Stationarity, classification of states and chains (Ch 2.3, 2.4).
Th 7/11	Stationarity, classification of states and chains for Markov processes (Ch 4.3, 4.4). Birth-and-death processes (Ch 4.6 4.7).
Tu 12/11	Absorbing states and absorption times (Ch 2.5, 4.5).
Th 14/11	Inference, simulation (Ch 2.7, 2.8, 4.9, 4.10).
Tu 19/11	The Poisson process: characterisations, the Poisson limit (law of small numbers), recurrence times, conditional distributions (Ch 3.2-3.3).
Th 21/11	Non-homogeneous Poisson processes, operations on Poisson processes, spatial and general Poisson processes, inference (Ch 3.4-3.8).
Tu 26/11	Markov random fields: definition, examples, local specification, Gibbs distributions (lecture notes here: PostScript or pdf format).
Th 28/11	Markov random fields: Markov chain Monte Carlo simulation, pseudo-likelihood (lecture notes as above).
Tu 3/12	Hidden Markov models: definition, examples. Filtering and smoothing, the forward-backward algorithm. Gibbs distributions (lecture notes here: PostScript or pdf format). These notes have now been updated. Pages 11-13 differ from the previous version.
Th 5/12	Hidden Markov models: parameter estimation and the EM algorithm (lecture notes as above).
Tu 10/12	Seminar on perfect simulation of Markov random fields and (hopefully!) an application of Markov chain modelling to analysis of DNA. This is not part of the course curriculum.
Th 12/12	No class.

Week 1:	101, 102, 104, 105, 201, 203.
Week 2:	301, 302, 403, 404, 405, 409, 412.
Week 3:	501, 502, 503, 601, 602.
Week 4:	701, 704, 706, 707, 708, 709, 714.
Week 5:	801, 802, 803.
Week 6:	901, 902.
Week 7:	Course overview, opportunity to ask questions, solution of old exam problems, ...