A model cannot take raw letters or whole words directly. Text is first cut into tokens: pieces taken from a fixed list called the vocabulary, which is built before training. Each entry has an integer ID. What the model receives is a list of integers.

A token can be a common whole word, part of a longer word, a punctuation mark, or a digit group. The space before a word is usually stuck to the word, so “cat” and “ cat” (with a space) are normally different tokens with different IDs. Capital letters matter too: “Cat” is another token again.