What are generative models?
Learning where the data lives
Every model in this course so far shares one posture: given an input, ANSWER A QUESTION about it. Cat or dog, spam or not, hostile or benign. These are discriminative models: their whole job is learning the boundary that separates classes. They can judge a cat photo; they could not produce one to save their weights.
Generative models flip the posture. Instead of learning the boundary BETWEEN kinds of data, they learn the shape OF the data: which images are plausible photos, which sequences are plausible sentences, where, in the space of all possibilities, real data actually lives. Learn that shape well enough and something new becomes possible: SAMPLE from it. Produce a brand-new point from the same region, a face nobody owns, a sentence nobody wrote.
In probability language, one line of notation: discriminative models learn P(y|x), "given this input, which label?", while generative models learn P(x) itself, "how plausible is this data point at all?". A model that knows P(x) can do more than create: it can also raise an eyebrow at data that shouldn't exist. Hold that thought for the exercise.
Sample from the model
Gray dots: training data, it lives on a ring. A generative model trained on it has learned that shape. Draw samples from the model, then draw random points, and watch the difference.
Generative or discriminative?
Six tasks. Decide which kind of model each one fundamentally needs.
Given an email, decide: spam or legitimate.
Write a realistic phishing email for security-awareness training.
Given a network session, classify: hostile or benign.
Produce synthetic patient records so researchers can work without touching real, sensitive ones.
Flag a transaction because the model considers it wildly improbable under normal behavior.
Given a photo of a dog, output its breed.
Check your understanding
The essential difference between discriminative and generative models is…
What to remember
- ✓Discriminative: learn the boundary between kinds, P(y|x). Everything in this course until today.
- ✓Generative: learn where the data LIVES, P(x), and gain the power to sample new points from that region.
- ✓Generated samples are new, but not from nowhere: they come from the learned shape of the training world.
- ✓Knowing P(x) cuts both ways: creation (faces, emails, records) and detection (anomalies too improbable to be normal).
- ✓Three great architectures learned this shape in three wildly different ways, GANs, VAEs, diffusion. First up: the one that learned by fighting itself.