Context Length Exceeded: Reduce the Request Without Losing the Answer

Fix context length exceeded by measuring the whole request, not by randomly deleting half the prompt. Identify the selected model and the size of each component, remove mechanical duplicates, irrelevant history, and oversized tool results, then reserve enough output space. After retrying, verify both the absence of the error and the presence of required facts.

What consumes the context window

system instructions
+ conversation history
+ current message
+ images and documents
+ tool definitions
+ tool results
+ output and thinking budget
= total context usage

Anthropic's context-window guide includes the system prompt, all messages, images, documents, tool definitions, tool results, and generated output. Prompt caching can change the price of reused tokens, but cached tokens still occupy the window.

Do not hard-code one universal context size. The window and overflow behavior depend on the selected model and API, so check the current model documentation.

Measure each component

Component Measure Safe reduction
System instructions Duplicate rules and large examples Merge duplicates; preserve hard constraints
History Tokens per message and obsolete branches Remove irrelevant branches or create a verifiable summary
Files and retrieval Each document, duplicate chunks, low-relevance results Deduplicate, reduce top_k, pass only required sections
Tool definitions Unused tools and long descriptions Send only tools required for the current step
Tool results Full JSON, logs, HTML, base64, repeated responses Keep required fields, links, and identifiers; store large data outside the prompt
Output budget max_tokens and thinking allocation Reserve realistic space or split the result into stages

Claude provides a Token Counting API that can count messages and tools before a request. For another provider, use its own counter when available. A local tokenizer is useful for early warnings, but its estimate is not a guaranteed server count for a different model.

For requests through BetterToken, the Dashboard can show input, output, and cache tokens after the test. The current API documentation helps verify the request contract, but the Dashboard does not expose the full prompt and does not replace preflight token counting.

Reduce context in the right order

1. Remove exact duplicates

Find repeated system rules, duplicated files, repeated retrieval chunks, copied schemas, and full logs already represented elsewhere. This is the lowest-risk reduction because it does not change the task.

2. Remove irrelevant history

Separate durable state from conversational process. Preserve the goal, accepted decisions, hard constraints, verified facts, and open questions. Remove abandoned options and completed tool traces, or replace them with a structured summary that records exact state.

3. Compact files, retrieval, and tool results

Pass relevant sections instead of full documents. Keep only the tool-result fields required by the next step. Do not delete sources or safety requirements merely to make a request pass; split the task into verifiable stages instead.

4. Reserve output space

Input and output share the context budget. If input nearly fills the window, the model may not have enough room to finish. Reduce optional input, select a realistic output budget, or produce the result in stages.

5. Change model only after measuring

A larger context model may be correct for a document that cannot be safely divided. But switching models without removing duplication only postpones the next failure and can reduce information density.

Preflight logic

components = count_by_section(request)
estimated_input = sum(components)
reserved_output = requested_output_budget

if estimated_input + reserved_output approaches current_model_window:
    remove exact duplicates
    drop irrelevant history
    compact tool results and retrieved chunks
    count again

send only after required facts and constraints remain present

Approaches is intentionally not a fixed percentage. The safety margin depends on the counter, model, thinking mode, and API behavior.

Verify the fix

  1. The API no longer returns context length exceeded or prompt is too long.
  2. The response finishes normally instead of exhausting its output budget.
  3. Every required fact, constraint, and output-format rule is present.
  4. Citations still match the supplied sources.
  5. Tool calls retain the correct arguments and required results.
  6. New input usage is lower than the original request.

If the error disappears but the answer loses a critical constraint, the fix failed. Restore that block and free space from less relevant history or tool output. If the answer is truncated, inspect the output budget separately.

The reliable sequence is: identify the model, count components, remove duplicates, discard irrelevant history, compact files and tool results, reserve output space, retry, and validate quality.

Sources

Related articles