Backpropagation Applied to Handwritten Zip Code Recognition

Y. LeCun, B. Boser, J.S. Denker, D. Henderson, R.E. Howard, W. Hubbard, L.D. Jackel

Comments

Yuxiang's blog

Summary

The authors describe an approach to recognizing handwritten zip codes using neural networks. Their approach uses a three layer neural network trained using backpropagation. The network takes a 16 x 16 normalized image of a single digit as input and outputs 10 units representing the 10 different digits. Feature detection on the input is done through weight sharing, which reduces the number of free parameters in the network and can express information about the geometry and topology of the task. Training and testing data for the neural network were provided by the U.S. Postal Service.

Discussion

My understanding of neural networks is limited to the small amount I remember from undergrad artificial intelligence. However, the task they have in mind with the large dataset available to them seems well suited for using neural networks. This points out one of the drawbacks of using neural networks in sketch recognition in that extensive training is needed. For handwriting recognition, where available traning data is enormous, neural network seem applicable, but perhaps not for free-form sketching.