Classic CNN architectures
The recipe
You now own all the bricks, convolution, ReLU, pooling, dense layers, softmax. Nearly every classic image classifier arranges them in the same rhythm: scan, shrink, scan deeper, shrink again… then judge.
The rhythm implements the trade you watched in the pooling lesson: with each round, the maps get spatially smaller but the stack gets deeper, geography drains out, evidence concentrates. When the maps are tiny and rich (say 7×7×512), one final move changes the game: FLATTEN them into a single long vector. From there it is territory you have known since Fundamentals, dense layers combining evidence, softmax turning scores into probabilities.
A useful way to see it: the convolutional half is a learned EYE that turns pixels into evidence; the dense half is a learned JUDGE that turns evidence into a verdict. Both halves train together, end to end, by the same backpropagation you have been using all along.
Assemble the classic pipeline
Here are the six stages of a classic image classifier, shuffled. Click them into the order the data actually flows through.
The hall of fame: one idea per name
Four architectures carried this recipe through history, and each name stands for exactly one idea worth keeping:
- LeNet-5 (1998): the proof of concept. Yann LeCun’s digit reader processed real bank checks: conv-pool-conv-pool-dense, working in production while most of the field ignored neural networks.
- AlexNet (2012): the big bang. Same recipe scaled onto gaming GPUs, trained with ReLU and dropout on 1.2M images. It cut the ImageNet error from 26% to 15% overnight, the single result that convinced the world, ended the AI winter you read about in Fundamentals, and started the deep learning era.
- VGG (2014): radical uniformity. One rule everywhere: stack tiny 3×3 filters. Two stacked 3×3 see as far as one 5×5 with fewer weights and an extra ReLU of nonlinearity between them. Nineteen layers of monastic sameness, elegant, and famously heavy (~140M parameters).
- ResNet (2015): depth unlocked. Skip connections (your previous module!) made 152 layers trainable and won ImageNet 2015. From then on, "deeper" stopped being a gamble.
Notice the pattern in the pattern: each leap came from removing an obstacle you have already studied, compute (GPUs), dying gradients (ReLU), overfitting (dropout), untrainable depth (skips). Architecture history is just this protocol’s lessons, deployed in order.
Name that architecture
Six claims to fame. Attribute each one to its network.
Won ImageNet 2012 by training on gaming GPUs with ReLU and dropout, and started the deep learning era.
Its entire philosophy: stack identical tiny 3×3 filters, two of them see as far as one 5×5, cheaper and with more nonlinearity.
Made 152-layer networks trainable and won ImageNet 2015.
Read handwritten digits on real bank checks, in 1998.
Elegant, uniform, and famously heavy: around 140 million parameters.
Its core block computes F(x) + x, each layer learns only the correction.
Check your understanding
Why did AlexNet (2012) matter so much beyond its benchmark score?
What to remember
- ✓The classic recipe: [conv+ReLU → pool] repeated, then flatten → dense → softmax. A learned eye feeding a learned judge, trained end to end.
- ✓LeNet proved it (1998). AlexNet scaled it and started the era (2012). VGG showed small-filters-stacked-deep (2014). ResNet unlocked real depth with skips (2015).
- ✓Each leap removed one obstacle: compute, dying gradients, overfitting, untrainable depth, the exact topics of this protocol so far.
- ✓Read any CNN diagram as variations on the recipe: where it scans, where it shrinks, where it judges.
- ✓One question left for this module: do you actually have to TRAIN all this yourself? Almost never: next lesson.