Transfer learning
You will (almost) never train one of these from scratch
Everything this module built: AlexNet, VGG, ResNet, trained on 1.2 million labeled images for GPU-weeks. Your actual project has 800 photos and a laptop. Honest question: is everything you just learned useless to you?
The answer hides in a fact you discovered two lessons ago: first-layer filters converge to edge and texture detectors on almost ANY visual task. Edges are edges: for cats, for tumors, for defective circuit boards. The deeper you go the more task-specific features get, but the bulk of a trained network is general-purpose visual machinery. Machinery someone else already paid to build.
So don't rebuild it: take it. Download a ResNet pretrained on ImageNet, remove its final layer (the only part that answers the OLD question, "which of 1000 ImageNet classes?"), bolt on a fresh small head that answers YOURS, and train just that on your 800 photos. The frozen backbone turns each photo into rich evidence; your head only learns to judge that evidence. This is transfer learning, and it is how most real-world computer vision actually ships.
Ship a classifier this week
The standard transfer learning workflow, shuffled. Put the five steps in working order.
How much should you retrain?
The dial goes from "freeze everything" to "train from scratch", and where you set it depends on how much data you have and how far your domain is from the pretraining data. Five projects: set the dial.
800 phone photos of two mushroom species, ordinary-looking photos, tiny dataset.
80,000 chest X-rays: plenty of data, but grayscale medical imagery looks nothing like ImageNet photos.
20 million labeled product images and a GPU cluster booked for a month.
Hackathon demo due tomorrow: 300 images, a laptop, coffee.
60,000 satellite images, decent size, and texture statistics unlike anything in ImageNet.
The default, not the trick
Calibrate how normal this is: starting from pretrained weights is the DEFAULT in professional computer vision, the overwhelming majority of shipped systems begin from someone else's training run. "From scratch" is the exotic exception that needs justifying, not the other way around.
And the idea is bigger than vision. Pretrain a huge model once on general data, then adapt it cheaply to specific tasks, replace "ResNet on ImageNet" with "language model on the internet" and you have described GPT, Claude, and the entire modern era. The Building with AI protocol has a whole module on fine-tuning; when you get there, remember it is this lesson wearing a bigger coat.
Module complete. You can now read the machinery that lets networks see, convolution, feature maps, pooling, the classic architectures, and you know the professional move: stand on a giant, train a judge. Next module the data stops sitting still: sequences.
Check your understanding
Why can a network pretrained on cats and cars help classify X-rays it has never seen?
What to remember
- ✓Pretrained backbones are the professional default: borrow the eye, train the judge. Training from scratch is the exception that needs justifying.
- ✓It works because features transfer: early layers are universal, task-specificity concentrates at the top, exactly the hierarchy this protocol keeps meeting.
- ✓The workflow: pick pretrained → replace head → freeze → train head → optionally fine-tune the top, always with a tiny learning rate.
- ✓The dial: small data + similar domain = freeze; big data + domain gap = fine-tune deeper; ImageNet-scale data = maybe scratch.
- ✓Pretrain once, adapt cheaply, the same economics that power modern language models. You will meet this idea again, much bigger, in Building with AI.