In chapter 2 of the book "Machine Learning in Action", section 2.3 has an example of handwritten digit recognition using knn.
I have not implemented this described method, however I have some basic experience with handwritten digit recognition, particularly relating to the framework described by
Belongie et al.[PDF]. I have implemented the approach by Belongie et al. and obtained an error rate of ~8% against a small subset of the
MNIST handwritten digit database (the published error rate of the approach is 0.63%, but this is due to a significantly larger training set comprising of 60,000 digits).
Given the simplicity of the approach described in the book and the relatively small training set I was surprise to read that it was able to obtain such outstanding performance (1.2% error). What can be attributed to the performance differences of the two approaches? Is it the larger number of exemplars within the training set, differences in complexities between the data sets, or something inherent to the approach that I missed?
Thanks