Feature Informativeness, Curse-of-Dimensionality and Error Probability in
Centre for Mathematical Sciences
This thesis is based on four papers on high-dimensional discriminant analysis.
Throughout, the curse-of-dimensionality effect on the precision of the
discrimination performance is emphasized. A growing dimension asymptotic
approach is used for assessing this effect and the limiting error probability
are taken as the performance criteria.
In the first paper, the asymptotic distribution of the discriminant function
is established and the limiting expressions for the error probabilities are
then obtained. Using the latter, it is show that the performance of
discrimination is severely inflicted by the curse-of-dimensionality and a
consistent approximation of the likelihood based discriminant function is
The second paper discusses the performance of discrimination from the point
of view of feature discriminating power. A concept of informativeness is
introduced as a feature evaluation tool. By means of a weighted discri-minant
function, which distributes weights among features according to their
informativeness, the impact of the latter into the error probability is evaluated
in a high-dimensional setting. An optimal, in a sense of minimum error
probability, type of weight function is established. The weighting scheme
is illustrated by some examples which justify the appropriateness of the
developed approach and show that the discrimination performance can be improved
upon by a suitably chosen weight function.
In the third paper, the weighting technique is elaborated by using an estimation
procedure in the feature evalua-tion. The presence of high-dimensional features
is shown to lead to overestimation of their informativeness, which increases
the error probability thereby reflecting the curse-of-dimensionality effect.
The explicit form of the weight function, which provides the minimum of the
limiting error probability when weighting by estimates, is found.
In the fourth paper, a threshold based feature selection procedure is introduced
in high-dimensional discriminant analysis. This is incorporated into the
discriminant function by means of an inclusion-exclusion factor, which eliminates
the sets of features whose informativeness does not exceed a given threshold.
The relationship between the fraction of selected features and the selection
threshold is established. Combined effect of feature selection and curse-of
dimensionality on the error probability is evaluated.