Automatik Registration of Large Images from Light Microscopy

Lund University Faculty of Engineering, LTH

Abstract

In the biomedical research field, an important tool in the characterisation of tissue morphology and pathology is microscopic imaging. Samples of lung tissue, provided by Medetect, are used in the registration application. The registration process is divided into four main steps: pre-processing, feature extraction, classification and image alignment. The pre-processing step involves suitable operations, which have been shown to greatly improve the registration process. Two main approaches are used to extract keypoints, SIFT and MSER. The keypoints are then matched to produce point correspondences. The alignment process uses the RANSAC algorithm to estimate a rigid transformation using point correspondences. Registration is then complete and hopefully characterisation of tissue morphology and pathology will have been facilitated considerably. Results show that registration using SIFT produces relatively good results and that it is very important to pre-process the images beforehand. However, when using both SIFT and MSER the matches can be boosted to some degree.

Introduction

The lung tissue samples are sliced into extremely thin sections and stained with different markers. Each section is scanned in an Aperio ScanScope slide digitiser, resulting in extremely large images, up to approximately 65kx65kx3 in size. This preparation process (i.e. sectioning, staining, cover glass application) can introduce a variety of deformations, including bending, stretching and tearing. However, when the images are not aligned identically this can sometimes cause difficulty at the analysis stage. Although this mismatch of consecutive sections can be corrected manually to facilitate their study, this procedure can be extremely time-consuming and an automatic process is an interesting development. The developed registration application aligns lung tissue samples in four main steps: pre-processing, feature extraction, classification and image alignment. First the alignment method is explained to facilitate understanding of the registration application. The main steps will be explained in further detail later. The results and the discussions are presented in the final section as well as further improvements.

Related work

An extensive study was made of general image registration since medical image registration is relatively new when using large images. However, there are related works that use different techniques to perform medical image alignment - commonly used methods based, for example, on PCA neural network, cross-correlation methods, Fourier transform and mutual information [SLY06].

Other related works are [Lowe04] and [MCUP02], which are used in the developed application with some modification. The SIFT and MSER techniques are widely used in image registration and have been shown to be very robust in the face of local image deformations and different illumination conditions and are invariant to scale. The authors show experimentally the use of MSER for wide-baseline matching and SIFT for object recognition in the presence of clutter and occlusion.

Image alignment

The reference and the unregistered images, see Figure (a), are first pre-processed and valuable information is extracted, referred to as feature extraction. The features are then compared and classified, resulting in point correspondences, which will be the input for the alignment process. The unregistered image and the information obtained from the alignment procedure transform the image so that it matches the reference image.

The rigid transformation (1) is the correlation between points in image A and image B, by a simple rotation and a translation, also referred to as point correspondences [Wei09].

The unknown parameters cosθ, sinθ, t_x and t_y can be estimated by rearranging (1) into (2) and solving it using Least-Squares. Note that at least two point correspondences are necessary to determine the parameters of the rigid transformation. However, it is preferable to have an over-determined equation system resulting in a better approximation.

If a point correspondence supports the model estimation it is referred to as an inlier, otherwise it is referred to as an outlier. To eliminate incorrect point correspondences, the Random Sample Consensus, also abbreviated to RANSAC, is applied [FB81]. An image alignment is shown in Figure (b), where the black lines corresponds to inliers and the red dashed lines corresponds to outliers.

Extracting and matching features

Keypoints with two properties, a location and a descriptor, are produced using two techniques, Scale Invariant Feature Transform and Maximally Stable Extremal Region [Lowe04]  [MCUP02]. The SIFT algorithm first compute the Gaussian Scale pyramid and then computes the Difference-of-Gaussian (DoG). The location is determined if its pixel value is the maximum or minimum among its 26 nearest neighbouring pixels in the DoG. The descriptor is defined by computing the orientations around each keypoint in a 4x4 region, where each region represents eight orientations. This produces a 128-dimensional descriptor, which is used to compare other keypoints. The second method produces the MSER, which are regions that do not change in size in a sequence of binary images obtained by using different thresholds. The location is determined by the centroid of the MSER. The descriptor represents the area, perimeter and the hollowness of each MSER. Note that this is a 3-dimensional descriptor. Point correspondences are produced by matching keypoint descriptors using a nearest neighbour classifier. A keypoint will be classified to its nearest neighbour if the relationship between the distances d₂ (second nearest) and d₁ (nearest) neighbour fulfills the criterion T₁ ≤  d₂ /d₁. In addition, it can be classified to its two nearest keypoints if the criterion is also fulfilled T₂ ≤  d₃ /d₁, where d₃ is the distance to its third nearest neighbour.

Results and conclusion

The keypoint descriptors are greatly affected when deformations and distortions are introduced. Consequently, some pre-processing techniques are applied to reduce these disturbances. Three main pre-processing techniques were applied: re-sizing, pre-smoothing and histogram equalisation. Disturbance and highly detailed structures might still be present after reducing the image size and smoothing is therefore applied. The MSER procedure can be improved when the foreground and the background are well distinguished. This is achieved when applying histogram equalisation. These methods have been shown to substantially improve the robustness of correct matches, see Figure (c). Note that it is very important to apply pre-processing and that SIFT, in general, performs well but can be improved by incorporating the MSER procedure, see Figure (d).

Further improvements

A simple method to boost the stability of the MSER approach is to choose better keypoint features. These features could be moments, shape descriptors, the maximum/minimum distance from the boundary to the centroid etc. Further studies could be made when choosing the classification thresholds for both methods. It is possible to increase the reliability of point correspondences by clustering for example three features together and checking if they are the same in the unregistered image as in the reference image. There is a greater chance of obtaining correct clusters than for individual feature matches. At present, the registration application chooses a reference slide and tries to match all the other slides to it. The application could be improved by using all the matching information.