Tokens and the Context Window
What the model actually sees; and why it seems to forget things mid-conversation.
Models don't read words the way you do. They read tokens, and they can only hold so many at once. That one fact explains a surprising amount of weird AI behavior.
What a token is
A token is a chunk of text, often a whole word, sometimes a piece of one. "cat" is a single token; a long word like "unbelievable" might be three. The model never sees letters or words as you do; it sees a stream of these chunks, each turned into a number. Everything it reads and everything it writes is tokens.
The context window
The context window is how many tokens the model can hold in mind at once. It counts everything together: your prompt, the conversation so far, any documents you pasted, and the answer it is in the middle of writing. Every model has a limit. Go past it and the oldest material falls out of view.
Why it seems to forget
When a long chat runs past the window, the earliest messages drop off the back. The model is not being forgetful; those tokens are simply no longer in front of it. That is why a chatbot can lose the thread in a very long conversation, and why pasting a huge document can crowd out your actual question.
Why it matters
Two reasons. Cost and speed both scale with tokens, so more in the window means more to pay for and longer to wait (the math is in the true cost of one LLM call). And quality depends on what is in the window: put the relevant material in, leave the noise out. A lot of working well with AI is just managing what is in the window at the right moment.
A model is only ever reasoning over what fits in its context window right now.