You paste a long report into ChatGPT, ask questions about it for twenty minutes, and then it answers something that flatly contradicts what you told it at the start. Most people assume the AI got confused. Usually it is simpler than that. The chat ran out of room, and that room has a name: the context window.

It takes five minutes to understand, and once you do, the strange behaviour stops looking random and starts looking predictable. That is the whole point, because predictable problems can be worked around.

What is a context window, in plain English

A context window is the total amount of text an AI model can look at while writing its answer. Google’s own developer documentation uses the clearest analogy: think of it as short term memory. Anthropic’s documentation calls it “working memory” and defines it as all the text the model can reference when generating a response, including the response itself.

Here is the part that surprises people. The model does not remember your conversation the way you do. Every time you send a message, the tool quietly re-sends the whole chat so far, and the model reads all of it again from scratch before replying. Nothing is stored in its head between messages. The context window is simply how much of that pile it can hold at once.

The pile is measured in tokens rather than words. A token is a chunk of text, usually a short word or part of a longer one. Our guide on what a token is in AI explains it with examples.

Everything counts, including things you forgot about

People assume only their own typing fills the window. According to Anthropic’s documentation, everything in the request counts: the hidden system instructions the company wrote, every message in the conversation, any files or images you attached, and the answer the model is currently writing.

So a chat where you uploaded three PDFs and got five long answers is far fuller than it looks. The scroll bar shows your side of it. The window is holding both sides plus the attachments.

Why the AI forgets what you told it

When a conversation gets close to the limit, something has to give. Anthropic notes that chat interfaces can manage the window on a rolling first in, first out basis, which is exactly what it sounds like: the oldest part of the conversation drops off the front to make space at the back.

That is your answer. The instruction you gave in message two, the one about writing in British English or never using bullet points, was not ignored. By message forty it was no longer in the window at all. The model was not being careless. It could not see it.

Some tools handle this more gracefully by summarising the earlier part of the chat and carrying the summary forward instead of the full text. Anthropic calls this compaction. It helps, but a summary is still lossy, so small details from early on can quietly disappear.

A bigger context window is not automatically better

This is the part most beginner articles skip, and it matters more than the headline numbers. Anthropic’s documentation says it plainly: more context is not automatically better, because as the token count grows, accuracy and recall degrade. They call the effect context rot, and their engineering team writes that while some models degrade more gently than others, the pattern shows up across all of them.

Google says something similar in its long context documentation. Finding one specific fact buried in a huge amount of text works well, often around 99 percent of the time. Finding several specific facts at once does not work as reliably, and performance varies a lot depending on the material.

So when a company advertises a million token context window, read it as “this much will fit” rather than “this much will be read carefully”. Both companies say so themselves, in their own developer docs.

How big are context windows now?

They have grown quickly. Google’s documentation traces the path: earlier models handled around 8,000 tokens, then 32,000, then 128,000, and Gemini was the first to accept a million. Google makes a million tokens concrete like this.

• All the text messages you have sent in the last five years
• Eight average length English novels
• Transcripts of more than 200 podcast episodes

These numbers change every few months, and free tiers often have a smaller window than paid ones, so treat any figure you read online as a snapshot. The shape of the problem does not change: there is a limit, everything counts toward it, and getting close to it costs you accuracy.

Context window and memory are two different things

These get mixed up constantly. The context window is the limit inside one conversation. Memory is a separate feature that saves facts about you and carries them across different conversations. OpenAI’s Memory FAQ describes it as ChatGPT automatically remembering useful context from your chats, files and connected apps, and says you can turn it off in Settings at any time.

The practical difference: memory is why a brand new chat already knows you are a nurse in Dublin. The context window is why that same chat forgets a formatting rule you gave it half an hour ago. We covered the memory side in how AI memory works, including how to see and delete what has been saved about you.

Five habits that fix most “the AI forgot” problems

1. Put your question at the end. Google’s own documentation recommends this: when the context is long, put your question after all the background material rather than before it. Paste the document first, then ask.

2. Start a new chat instead of arguing. If a long conversation starts going in circles, that is a room problem, not a reasoning problem. Open a fresh chat and paste a short summary of what matters.

3. Give each chat one job. One chat for the cover letter, one for the spreadsheet, one for the holiday plan. Long mixed conversations fill the window with material that is irrelevant to whatever you are asking right now.

4. Repeat the rule that matters. If a formatting or tone instruction is important, restate it in the message where you need it. One line, no guesswork.

5. Upload the ten relevant pages, not the whole book. Trimming the input is the most reliable improvement, because you are spending the model’s attention on the part you care about.

From my own work on client websites, this changed how I use these tools. I used to paste an entire plugin’s documentation into a chat and then wonder why the answers drifted by the fifth question. Now I paste the one page I need and start a new chat when the topic changes. The answers got noticeably better without changing tools or paying for anything.

Important tip: if a long chat starts contradicting itself, do not try to correct it in place. Open a new chat and paste a five line summary of what has been decided so far. You will get a better answer in less time than the argument would have taken.

What this means for the way you write prompts

Once you picture the window, good prompting stops being a list of tricks. You are deciding what deserves the space. Anthropic’s engineering team frames it as finding the smallest set of high value information that gets the outcome you want, and that scales all the way down to writing one email. For the practical version, see our guide to writing better AI prompts. For the wider picture, start with what a large language model is, and keep our AI glossary open if the jargon slows you down.

Common Questions

Does a longer chat make the AI slower?

Usually yes, a little. Google’s documentation notes that longer inputs generally mean higher latency before the first word appears, because there is simply more to read.

Can I check how full my context window is?

In most consumer chat apps, no. Developers can count tokens before sending, but the everyday apps do not show a meter. Watch for the warning signs instead: repeated questions, forgotten instructions, and answers that contradict earlier ones.

What happens if I go over the limit?

In a chat app, the oldest messages usually drop out or get summarised, and the conversation carries on without telling you. If a single input is too big on its own, for example one very large file, you will normally get an error instead.

Does a bigger context window mean fewer wrong answers?

No. A bigger window reduces forgetting, but it does not stop the model inventing things, and both Google and Anthropic report that recall gets less reliable as the window fills. Check important facts either way.

Final takeaway

The context window is the AI’s desk, not its brain. It only holds so much, everything you put on it competes for space, and the pile at the back gets pushed off the edge when new work arrives. Keep the desk tidy and the work gets better.

You do not need a bigger plan or a smarter model to see the difference. Trim what you paste, put your question last, give each chat one job, and start fresh when things drift. Those four habits are free, and they fix most of the moments where AI feels unreliable.