Context Window — AI Glossary

Reference details

Related terms

• Token
• Inference
• RAG (Retrieval-Augmented Generation)
• Attention Mechanism

Example: Input and output share the budget

With a 128K window, a 120K-token prompt leaves roughly 8K for the response — and if you asked for a long report you will get a truncation instead. The window covers input plus output together, which is why long-document tasks fail at the end rather than the beginning.

Commonly confused with

The context window is not memory. Nothing persists between calls: each request re-sends whatever the model should know. A chatbot that appears to remember is being handed its own history every turn, and paying for it every turn.

When to use it

Reach for it when:

• Sizing whether a document fits before building around it
• Budgeting prompt against expected response length
• Deciding between long context and retrieval

Reach for something else when:

• As a proxy for capability — a bigger window says nothing about reasoning quality
• Assuming attention is uniform across it; middle-of-context recall is measurably weaker

Referenced by

This term is used by the following ModelRefs references:

• short term memory

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Context Window — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path