Skip to content
World 3 · Guessing the Next Word · 11 min · PRO

Learning by guessing

Pretraining as next-token prediction

PRO · How AI Works

World 3: Guessing the Next Word is part of PRO

The first world of How AI Works is free. PRO opens the other 24 lessons in 6 worlds — simulations, quizzes and all.

  • From scores to odds
  • Temperature
  • Top-k, top-p and sampling
  • What goes into the training data
  • Teaching it to follow instructions
PRO includes
  • All 60 labs — every world, every lesson
  • Practice exams and certificates

$8.33/mo, billed yearly ($99.99) · or $39.99 monthly

Unlock How AI WorksTry the free world
My notes
Only saved in this browserAll my notes →
← Top-k, top-p and samplingWhat goes into the training data →

Report a mistake

What is wrong?
0 / 2000 · at least 10 characters
Sent along with your report
  • Page
  • About Learning by guessing — How AI Works · llm/pretraining
Plus your browser type. Reports go to our support system; see the privacy notice.