LeNet is often cited as the earliest convolutional neural network with a similar architecture to what we see today. It is also one of the first applications of convolutional layers in neural networks. It follows work from LeCun et al. (1989), where the advantages of spatially-invariant feature maps were described.

See Yann LeCun’s page about LeNet for demos and other info.

Figure 1: The LeNet-5 Architecture.

Figure 2: (Top and bottom-left) This network showed impressive robustness to various distortions, i.e. unusual patterns, rotation, noise, etc. (Bottom-right) LeNet applied to number sequence recognition. (Source)