Welcome to discover how you can control using a video camera

Abstract

In some control situations it is difficult to use ordinary sensors. The rapid development in camera technology and frame grabbers has made it possible to use visual information in the control loop. The goal of this masters thesis was to control "the ball and beam" process using these ideas. An ordinary video camera was placed in front of the process. The camera was connected to a computer through a frame grabber. The images of the process were then analyzed in real time. The angle of the beam and the position of the ball were calculated and used as input in a traditional control algorithm. A sampling frequency of 8 Hz was obtained in the final implementation. The thesis contains a thorough description of the image analysis, the control algorithm, the hardware as well as a performance analysis of the system.

Presentation

I have finished a project in which I controlled a process by image analysis and without any electronical sensors. The process I worked with was "the ball and beam process":

Image Analysis

The most important part of the project was to find a good way to get relevant information from the images, Image Analysis. An edge detection algorithm was implemented to find edge points on the images. An edge point is a point on the image where the differences in pixel intensities in the environment are the largest. The resulted edge point do not has to be an integer. The image is a matrix of pixels that has a value according to the pixel intensity. The analysis are used in two different ways:

1. When the system is starting and the beam and the ball have to be found on the image. It is called Global Image Analysis.

2.Previous values of the beam angle and the ball position are known and one wants to find current beam angle and ball position. It is called Local Image Analysis

Global Image Analysis

With the first image nothing is known about where the beam and the ball are on the image. Two long lines are therefore drawn to get information about the angle of the beam, where an edge point should be found that should not be too far from the right positions. Around these positions smaller lines are chosen that will get a more exact position. The calculated result with this two positions as input should be a good appraise of the beam angle.

The next step is to find the edges of the beam. This is necessary in order to convert the position of the ball in pixels to a number between -1 and 1 (where -1 and 1 are the end points). It is known in advance that the edges will be close to the ends of the image. The rough position of the ball is found by searching along a long line some pixels over the beam. After that two new lines are drawn around that rough position to find the edges of the ball. The mean value of the two edge positions is used as the position of the ball. The position of the ball is calculated relative to the edges of the beam.

Local Image Analysis

After the first image analysis the last beam angle and ball position are known so the new values should not be too different. According to the control signal it is possible to calculate where the beam should be on the image as will be seen in the following chapter.

First step is to calculate the current angle by drawing two lines for identifying the beam angle. Since it should go as fast as possible the line should be as short as possible. After a number of experimentations an equation was found. More information about the equation is found in the rapport. Only two lines are analyzed since the calculations are time critical so as few lines as possible should be analyzed. Two lines is the lower limit. If more time were given more lines should be analyzed so the result could be better and more reliable.

To find the ball a line some pixels left of the previous position is drawn and just check the matrix values of the current pixels. When the first pixel is found with a value greater than a specified threshold the searching is stopped. Around that position an edge point detection is done to find the left side of the ball. Another line right of the left side position is then drawn and finds the right side of the ball. The mean value of these two points is the new position of the ball. Since the ball diameter is constant it might be enough to find the left side and then calculate where the right side should be.

With the beam angle and the ball position as input the control signals into the controller are calculated.

Combining Vision and Control

In the final solution the system includes six different parts: The ball and beam process, video camera, frame grabber, image analysis, controller and also a very simple user interface. All of them work in parallel except the image analysis, the controller and the frame grabber that work together in the same program.

The closed system includes controller, process and vision. The difference from normally systems is the vision with an image as input and the beam angle and the ball position as output. Vision is replacing the electrical sensor on the process. It includes a camera, a frame grabber and image analysis.

The system:

Vision:

A good routine for image analysis could be this: When the program starts, the beam angle and ball position is found by using global picture analysis, which takes rather long time. After that, the program is entering the local image analysis mode which updates the beam angle and the ball position. This is less time consuming since only a small part of the image has to be analyzed. It will do so until either the angle or the position are lost, which should result in re-entering the first picture analysis, or the program is stopped. This is implemented except that when the angle or the position are lost. It results in an error.

The results and many other things can be found in my report or just read the english abstract or the swedish abstract. Soon also a sequence of how well it works will be here.

Edited by Fredrik Emilsson [e93fem@efd.lth.se] and last modified 27 Nov 1997.