John Maindonald, Australian National University

Data Mining - Statistics Without Statistical Theory?

Abstract

The tradition of Data Mining arose from the computing community. It is
in part a response to the power of modern computational systems to
the new opportunities for collecting and collating data, to new types
of data (images, web pages, genomic data, ...) that are available
to those who have the skills to work with them, and to algorithms that
have attracted a particular following in the data mining
community. Data Mining has had also, elements of an attempt to invent
a new tradition of statistical analysis. A number of departments,
including some statistics departments, are now offering courses in data
mining. I will argue that this activity has been too often compromised
by inadequate attention to insights that have been teased out within
the statistical tradition. At the same time, new perpectives, and a
challenge to rethink some aspects of common statistical practice, can
be a useful source of invigoration. Many of the problems tackled in
the data mining literature have novel and interesting features, and
may be tackled using models with which many or most statisticians are
unfamiliar. I will o er an assessment of key issues where effective
dialogue is required between the data mining and the statistical
traditions.