Bayesian Networks - An Introduction

Henrik Bengtsson

Centre for Mathematical Sciences
Mathematical Statistics
Lund Institute of Technology
Lund University
1999:E15

Abstract:

The report covers the basic concepts and theory of Bayesian Networks, which are graphical models for reasoning under uncertainty. The graphical presentation makes them very intuitive and easy to understand, and almost any person, with only limited knowledge of Statistics, can for instance use them for decision analysis and planning. This is one of many reasons to why they are so interesting  to study and use.

A Bayesian network can be thought of as a compact and convenient way to represent a joint probability function over a finite set of variables. It contains a qualitative part, which is a directed acyclic graph where the vertices represent the variables and the edges the probabilistic relationships between the variables, and a quantitative part, which is a set of conditional probability functions.

Before receiving new information (evidence), the Bayesian network represents our {a priori} belief about the system that it models. Observing the state of one of more variables, the Bayesian network can then be updated to represent our a posteriori belief about the system. This report shows a technique how to update the variables in a Bayesian network. The technique first compiles the model into a secondary structure called a junction tree representing joint

distributions over non-disjoint sets of variables. The new evidence is inserted, and then a {message passing} technique updates the joint distributions and makes them consistent. Finally, using marginalization, the distributions for each variable can be calculated. The underlying theory for this method is also given.

All necessary algorithms for implementing a basic Bayesian network application are presented along with comments on how to represent Bayesian networks on a computer system. For validation of these algorithms a Bayesian network application in Java was implemented.

Keywords:

Bayesian networks, belief networks, junction tree algorithm, probabilistic inference, probability propagation, reasoning under uncertainty.