NLP & Transformers
Module 5 · NLP & Transformers · ~36 min

From Base Model to Assistant

Pretraining, fine-tuning, RLHF: how ChatGPT is actually made.

  1. Pretraining: learning from the internet7 minUp next
  2. Instruction tuning6 min
  3. RLHF: learning from feedback9 min
  4. Sampling: temperature and top-p7 min
  5. The context window7 min