What is a context window in simple terms?

It is the limit on how much text a language model can look at in one request, counted in tokens. The instructions, the conversation so far, your new message and the model's reply all have to fit inside it together.

Why does ChatGPT seem to forget what I said earlier?

Any chat app has to fit the conversation into a fixed window. Depending on the app, older turns are dropped or summarized, and details that do stay in can still be used less reliably when they sit in a very long input. How a particular app handles it is that app's design choice.

Is a token limit the same as a context window?

They are related but not identical. The context window is the total a single request can hold, input and output together. Some APIs also let you cap the length of the reply alone, which is a smaller limit set inside the window.

Does AI remember me between chats?

Not by default. A new chat starts with an empty window. Apps that have a memory feature save short notes and load them into new conversations, and the app's settings usually let you switch that off.

Does a bigger context window fix forgetting?

It helps with fitting more in, but it does not guarantee the model uses everything equally well. Research on long inputs found that information in the middle was often used worse than information at the start or end, and providers describe accuracy degrading as the token count grows.

LLM · LIMITS

What Is a Context Window? Why AI Seems to Forget

A context window is the maximum amount of text, counted in tokens, that a language model can take in for one request. Your messages, the earlier conversation and the model's own reply all share that space. Whatever does not fit is never seen by the model, which is why a long chat can seem to forget.

One-pager titled "What is a context window?" with the subtitle "The working memory of a model: a fixed amount of text". Two amber frames of the same size. The left, "Early in the chat", holds short wavy-line rows labeled System, History, You and a dashed amber Reply, with empty space marked "Room left". The right, "Much later, same box", holds the same rows plus a dashed green "Summary" row where earlier history was compacted, a longer History and a Reply down to one short line. A REMEMBER band notes: each model has its own size (boxes labeled 8K, 200K and 1M), when full it gets compacted (several lines of text with an arrow to one short dashed green line), and more memory, more power (a "Model" box linked to a 200K box with one lightning bolt, then an arrow to a larger 1M box with three bolts, so the same model uses more power with a bigger window). — One-pager: the same fixed-size box early and late in a chat. Wavy lines stand in for text, so the history grows, the reply room shrinks, and the oldest text is dropped.

One request, one fixed budget

Every time you send a message, the app builds one request and hands it to the model. The context window is the largest number of tokens that request can hold. OpenAI's documentation puts it as the maximum number of tokens that can be used in a single request. Anthropic describes it as all the text the model can reference when generating a response, including the response itself.

That last part surprises people. The limit covers more than your message. It holds the instructions the app adds, the conversation so far, any documents you attached, your new message, and the reply being written. A long answer needs room left over.

Tokens are pieces of words, not words. How many fit in a given piece of text depends on the tokenizer and the language, so read the window as a token count, not as a number of pages. The tokenization page explains the split, and the token counter on this site shows it for your own text.

The model does not remember, the app re-sends

A language model keeps nothing between requests. OpenAI's guide calls each text generation request independent and stateless. The chat feels continuous because the app sends the earlier turns again with every new message.

Anthropic's documentation shows the pattern. Each turn's input holds all previous conversation history plus the current user message, and the reply the model generates becomes part of the input for the next turn. The model reads the whole thing from the top, every time.

That is why "forgetting" is a loose word here. Nothing is erased from a memory the way a person loses a name. A detail is either in the request being read right now, or it is not.

What happens when it fills up

A conversation grows by one turn at a time. The window does not. At some point the full history no longer fits, and what happens next depends on the app.

Through an API, the options are blunt. Anthropic's documentation says that if the input alone exceeds the window, the request fails with an error. OpenAI notes that tokens generated beyond the limit may be truncated, so a reply can be cut off. Some chat apps manage it for you: Anthropic notes that chat interfaces can manage the window on a rolling first-in, first-out basis, so the oldest turns go first. Another approach is compaction, where earlier parts of the conversation are summarized and the summary takes their place.

In every case, a turn that is not in the request is not available to the model, and a summary keeps the gist but loses detail. That is one reason an instruction from the start of a very long chat can stop shaping the answers.

A bigger window is not a better memory

Fitting something in the window does not mean the model uses it well. Liu and colleagues tested models on questions where the relevant passage sat at different positions in a long input. Performance was often highest when the relevant information was at the beginning or end, and dropped when it sat in the middle, even for models built for long contexts. Those tests used models available in 2023, so treat the result as a reason for care, not a fixed ranking.

Anthropic's documentation makes the same point from the provider side: more context is not automatically better, and accuracy and recall degrade as the token count grows, an effect it calls context rot. Its engineering team describes context as a finite resource with diminishing marginal returns.

Length also costs work. Attention compares tokens with each other in pairs, so the work grows with the square of the input length. The attention page walks through that cost.

Memory features are a separate thing

Chat apps also offer memory, and it is easy to confuse with the window. A memory feature saves short notes about you, such as preferences or ongoing projects, in storage outside the model and carries them into later conversations. Anthropic's support page describes Claude saving memory as individual topics as you chat. For agents, Anthropic's engineering guide describes notes persisted outside the context window that get pulled back into it later.

So memory does not make the window bigger. Saved notes have to be loaded into the request to be read, and they take up room there like any other text. Whether a detail is available to the model comes down to one question: is it in the request right now.

FAQ

What is a context window in simple terms?: It is the limit on how much text a language model can look at in one request, counted in tokens. The instructions, the conversation so far, your new message and the model's reply all have to fit inside it together.
Why does ChatGPT seem to forget what I said earlier?: Any chat app has to fit the conversation into a fixed window. Depending on the app, older turns are dropped or summarized, and details that do stay in can still be used less reliably when they sit in a very long input. How a particular app handles it is that app's design choice.
Is a token limit the same as a context window?: They are related but not identical. The context window is the total a single request can hold, input and output together. Some APIs also let you cap the length of the reply alone, which is a smaller limit set inside the window.
Does AI remember me between chats?: Not by default. A new chat starts with an empty window. Apps that have a memory feature save short notes and load them into new conversations, and the app's settings usually let you switch that off.
Does a bigger context window fix forgetting?: It helps with fitting more in, but it does not guarantee the model uses everything equally well. Research on long inputs found that information in the middle was often used worse than information at the start or end, and providers describe accuracy degrading as the token count grows.

Sources

Last updated 2026-10-11