Feature Informativeness, CurseofDimensionality and Error Probability in
Discriminant Analysis
Tatjana Pavlenko
Centre for Mathematical Sciences
Mathematical Statistics
Lund University,
2001
ISBN 9162847759
LUNFMS10122001

Abstract:

This thesis is based on four papers on highdimensional discriminant analysis.
Throughout, the curseofdimensionality effect on the precision of the
discrimination performance is emphasized. A growing dimension asymptotic
approach is used for assessing this effect and the limiting error probability
are taken as the performance criteria.


In the first paper, the asymptotic distribution of the discriminant function
is established and the limiting expressions for the error probabilities are
then obtained. Using the latter, it is show that the performance of
discrimination is severely inflicted by the curseofdimensionality and a
consistent approximation of the likelihood based discriminant function is
proposed.


The second paper discusses the performance of discrimination from the point
of view of feature discriminating power. A concept of informativeness is
introduced as a feature evaluation tool. By means of a weighted discriminant
function, which distributes weights among features according to their
informativeness, the impact of the latter into the error probability is evaluated
in a highdimensional setting. An optimal, in a sense of minimum error
probability, type of weight function is established. The weighting scheme
is illustrated by some examples which justify the appropriateness of the
developed approach and show that the discrimination performance can be improved
upon by a suitably chosen weight function.


In the third paper, the weighting technique is elaborated by using an estimation
procedure in the feature evaluation. The presence of highdimensional features
is shown to lead to overestimation of their informativeness, which increases
the error probability thereby reflecting the curseofdimensionality effect.
The explicit form of the weight function, which provides the minimum of the
limiting error probability when weighting by estimates, is found.


In the fourth paper, a threshold based feature selection procedure is introduced
in highdimensional discriminant analysis. This is incorporated into the
discriminant function by means of an inclusionexclusion factor, which eliminates
the sets of features whose informativeness does not exceed a given threshold.
The relationship between the fraction of selected features and the selection
threshold is established. Combined effect of feature selection and curseof
dimensionality on the error probability is evaluated.


Key words:

.