Token IDs are only labels. ID 4821 is not “close in meaning” to ID 4822. To compute with meaning-like patterns, the model needs a representation in which similarity is a number you can calculate.

That representation is an embedding: for every token, a list of numbers called a vector. The vectors sit in a big table. Their values start random and are adjusted during training. Real models use hundreds to thousands of numbers per token.