Centre for Discrete and Applicable Mathematics

 CDAM Research Report, LSE-CDAM-99-07

August 1999

Cross-Validation for Binary Classification by Real-Valued Functions: Theoretical Analysis

Martin Anthony and Sean B. Holden


This paper concerns the use of real-valued functions for binary classification problems. Previous work in this area has concentrated on using as an error estimate the `resubstitution' error (that is, the empirical error of a classifier on the training sample) or its derivatives. However, in practice, cross-validation and related techniques are more popular. Here, we analyse theoretically the accuracy of the holdout and cross-validation estimators for the case where real-valued functions are used as classifiers. We then introduce two new error estimation techniques, which we call the adaptive holdout est imate and the adaptive cross-validation estimate, and we perform similar analyses for these. Finally, we show how our results can be applied to certain types of neural network.

A compressed (gzip) PostScript file (79 kB) with the full contents of this report can be downloaded by clicking here.

Alternatively, if you like a free hard copy of this report, please send the number of this report, LSE-CDAM-99-07, together with your name and postal address to:
CDAM Research Reports Series
Centre for Discrete and Applicable Mathematics
London School of Economics
Houghton Street
London WC2A 2AE, U.K.
Phone: +44(0)-171-955 7732.
Fax: +44(0)-171-955 6877.
Email: info@maths.lse.ac.uk

Introduction to the CDAM Research Report Series.
CDAM Homepage.

Copyright © London School of Economics & Political Science 2005

Last changed: Wed 9 Feb 2005
For comments go to: http://www.maths.lse.ac.uk/webmaster.html