Home  |  Centre for Mathematical Sciences  |  LTH  |  LU

Multi-target Tracking Using on-line Viterbi Optimisation and Stochastic Modelling

Fulltext .
Printed version Errata.
Slides with videos.


To study and compare the safety of intersection, traffic scientists today typically manually monitor the intersection during several days and count how often certain events such as evasive manoeuvres occur. This is a laboursome and costly procedure. The aim of this thesis is to provide tools that can reduce the amount of manual labour required by using automated video analytics. Two methods for creating for such tools are presented.

The first method is a probabilistic background foreground segmentation that for each block of pixels calculate the probability that this block currently views the static background or some moving foreground object. This is done by deriving the probability distribution of the normalised cross correlation in the background and the foreground case respectively. The background distribution depends on the amount of structure in the block.

The second method is a multi-target tracker that uses the probabilistic background foreground segmentation to produce the trajectories of all objects in the scene. It operates online but with a few seconds delay in order to incorporate information from both past and future frames when deciding on the current state. This means that the output is guaranteed to be consistent, i.e. no jumping between different hypothesis, and the respect constrains placed on the system such as "objects may not occupy the same space at the same time" or "objects may only appear at the border of the image".

The methods have been tested both on synthetic and numerous sets of real data by implementing applications such as people counting, loitering detection and traffic surveillance. The applications have been shown to perform very well as long as the scene studied is not too large.


A probabilistic background foreground segmentation based on normalised cross correlation features are demonstrated on a few different sequences. At the top the input image is shown. Bottom left shows the probability of foreground and bottom left the result of a MAP estimate of a binary background foreground segmentation from the probabilistic one using a Markov random field as prior. The optimal estimate were found using graph cut.
People entering and exiting an entrance are counted using multiple spatially overlapping hidden Markov models. The numbers above and below the line gives the number of people crossing the line going out and in respectively. The test sequence consists of 14 minutes video where 249 persons passes the line. 7 are missed and 2 are counted twice. This gives an error rate of 3.6%. The algorithm runs at 49 fps on a 2.4GHz P4.
A calibrated camera with the world coordinate system registered to a blueprint of the corridor. Pedestrians are tracked using a single hidden Markov model modelling the motion entries and exits of all objects.
A scene viewed from different directions by four different calibrated cameras. Pedestrians are tracked using a single hidden Markov model taking advantage of the information available in all four cameras.
The same system as above but used to do loitering detection on the PETS 2007 dataset.
The same multi-target hidden Markov model tracker as above, but extended to handle 3 types of objects (buses cars and pedestrians/bicycles). Each object is defined as a box of some dimension. The state space have here been extended to in addition to the position of each object also contain the orientation of each object
The same experiment as above, but with each of the four video sequence projected on the ground plane and overlaid with the tracing results.


Questions: webmaster
Last updated: 2009-06-08

Centre for Mathematical Sciences, Box 118, SE-22100, Lund. Phone: 046-222 00 00