NLP & Transformers
The complete pipeline that turns raw text into ChatGPT, with nothing skipped. You start where the model starts, with text it cannot read, and follow the data through tokenization, embeddings, and attention, up to the full GPT architecture and the training recipe (pretraining, instruction tuning, RLHF) that turns an autocomplete engine into an assistant.
The interactives run the real algorithms in your browser: you will perform BPE merges yourself, compute attention weights through an honest softmax, and reshape a sampling distribution with the same temperature dial the APIs give you.
What you'll be able to do
- Explain how text becomes tokens, and why tokenizers cause the model's strangest failures: counting letters, doing arithmetic, charging some languages triple
- Read the attention mechanism end to end: scores, softmax, weighted mixing, causal mask, multiple heads
- Say what GPT is in one precise sentence (a decoder-only transformer) and explain why scaling it kept working
- Describe how an assistant is made, stage by stage: pretraining, instruction tuning, RLHF
- Set temperature, top-p and context length in an API knowing exactly what each dial does to the distribution
From Text to Tokens
Models can't read words: how text becomes numbers.
0/5 lessons completed · ~33 min
Embeddings
From token IDs to meaning: vectors, semantic space, context.
0/5 lessons completed · ~30 min
The Transformer
The architecture that changed everything: attention is all you need.
0/5 lessons completed · ~35 min
From Base Model to Assistant
Pretraining, fine-tuning, RLHF: how ChatGPT is actually made.
0/5 lessons completed · ~36 min