BERT vs GPT
Same parts, opposite bets
In 2018, two teams took the transformer and bet on opposite halves. OpenAI kept the decoder and the causal mask: GPT, built to continue text. Google kept the encoder, no mask at all: BERT, built to understand text. Same attention, same FFNs, same residual glue; one architectural decision apart.
BERT's training exam had to be different, since "predict the next token" is trivial when you can see it. The replacement: masked language modeling. Take a sentence, hide 15% of the tokens at random, and ask the model to reconstruct each hole using BOTH sides of it. A fill-in-the-blank exam instead of a continue-the-story exam.
For a few years, BERT was the undisputed champion of understanding tasks: search ranking, classification, entity extraction. Google shipped it into production search in 2019, which at the time was the largest transformer deployment on the planet.
Sit both exams yourself
Same sentence, same blank, two rulebooks. First fill the hole with both sides visible (BERT's exam), then with the right side confiscated (GPT's view).
BERT's exam: both sides visible
GPT's view: the future is confiscated
Staff the project
Five real tasks. Assign each to the architecture whose exam actually prepared it for the job.
Autocomplete the next sentence in a writing app.
Classify 50,000 support tickets by urgency.
Power a chatbot that answers in free-form text.
Highlight which contract clause answers a lawyer's question.
Turn bullet points into a polished product description.
How the rivalry actually ended
The spotlight went to GPT, for a structural reason: a strong enough generator can IMITATE understanding tasks ("read this ticket, answer with one word: high/medium/low"), while no understander can imitate generation. One skill subsumes the other at scale, so the general-purpose bet won the platform war.
But look inside production systems and BERT's bloodline is everywhere. The embedding models that power semantic search and RAG retrieval (that map from module 2, industrialized) are encoder-family models. When you build a RAG system in the AI Engineer protocol, an encoder will fetch the documents and a decoder will write the answer: the two 2018 bets, working the same shift.
File this pattern away: in AI, "who won" usually means "who got the interface". The loser often keeps the infrastructure.
What to remember
- ✓BERT = encoder, no mask, fill-in-the-blank exam, built to understand. GPT = decoder, causal mask, continue-the-text exam, built to generate.
- ✓Bidirectional context is strictly better evidence for understanding, and exactly what generation cannot have.
- ✓Generation subsumes understanding at scale; the reverse is impossible. That decided the platform war.
- ✓Encoder descendants still run search and RAG retrieval: the interface went to GPT, the infrastructure kept BERT.
- ✓Choosing between them is a task question, not a fashion question.