Errors & Reliability

Context Length Exceeded: Reduce the Request and Verify the Result

Measure every context component, remove safe duplicates and irrelevant history, reserve output space, and verify that no required facts were lost.

BetterToken

Editorial Team

Aug 8, 20265 min read

Fix Context length exceeded by accounting for the total budget, not by randomly deleting half the prompt. Identify the model and measure each request component first. Then remove mechanical duplicates, irrelevant history, and heavy tool results. After resending, verify both that the error is gone and that required facts and answer completeness remain intact.

What fills the context window

Context is the working memory of one request, not just the latest user prompt. In simplified form:


system instructions
+ conversation history
+ current message
+ images and documents
+ tool definitions
+ tool results
+ output / thinking budget
= total context usage

Anthropic's context-window documentation explicitly includes the system prompt, all messages, images, documents, tool definitions, tool results, and generated response. Prompt caching changes the cost of reused tokens, but it does not remove them from the window.

Do not hard-code a “universal limit”: the context window and overflow behavior depend on the selected model and API. Check the current model card on the day you configure it.

Find the heaviest component

Component	What to measure	Safe reduction
System instructions	Repeated rules and long examples	Merge duplicates and retain mandatory constraints
History	Tokens by message and old branches	Remove irrelevant branches or replace them with a verifiable summary
Files and RAG fragments	Each document's size, duplicates, and low-relevance chunks	Lower `top_k`, deduplicate, and pass only the required sections
Tool definitions	Unused tools and long descriptions	Pass only the tools needed for the current step
Tool results	Complete JSON, logs, HTML, base64, and repeated responses	Keep required fields, links, and identifiers; store large data outside the prompt
Output budget	`max_tokens` and thinking budget	Reserve a realistic amount or split a large result into stages

Claude has a separate Token Counting API that accounts for messages and tools before a request is sent. For another provider, use its own counter if available. A local tokenizer is useful for early warnings, but its estimate is not a guaranteed server-side count for a different model.

For a request sent through BetterToken, the Dashboard lets you compare input, output, and cache tokens after a test. The current API documentation helps verify the contract, but the Dashboard does not show the full prompt or replace preflight token counting; it is a post-request budget check.

Reduce size without losing meaning

1. Remove mechanical duplicates

Look for repeated system rules, the same file in several messages, duplicate RAG chunks, repeated schemas, and complete logs pasted more than once. This is the safest stage because it reduces size without changing the task.

2. Remove irrelevant history

Separate long-lived facts from the temporary flow of the conversation. Preserve goals, accepted decisions, mandatory constraints, and open questions. Remove old reasoning, rejected alternatives, and tool results that have already been processed, or compact them into a structured summary.

A bad summary says “we discussed the integration.” A useful one records the chosen endpoint, schema version, accepted constraints, confirmed facts, and next step.

3. Compact files, RAG, and tool results

Pass relevant sections instead of an entire document. In a tool result, keep the fields required by the next step instead of the full HTTP response or log. Do not discard sources or mandatory data merely to make the request fit; split the work into several verifiable stages instead.

4. Leave room for the answer

Input and output share the total budget. If the request nearly fills the window, the model may not have enough room for a complete answer. Reduce optional input, set a realistic output budget, or split the result into parts. Remove mechanical duplicates before critical facts.

5. Change models only after measuring

A larger-context model may be the right choice for a document that cannot be split safely. Moving to a larger window without removing duplicates only postpones the next failure and may reduce information density.

Minimal preflight check


components = count_by_section(request)
estimated_input = sum(components)
reserved_output = requested_output_budget

if estimated_input + reserved_output approaches current_model_window:
    remove exact duplicates
    drop irrelevant history
    compact tool results and retrieved chunks
    count again

send only after required facts and constraints remain present

approaches is intentionally not replaced by a fixed percentage. The necessary headroom depends on counter accuracy, the model, thinking, and the specific API's behavior.

How to verify the fix

Compare the new request against this checklist:

The API no longer returns context length exceeded or prompt is too long.
The response completes normally instead of being cut off by the output budget.
The response includes all mandatory facts, constraints, and the required format.
Quotes or links still correspond to the supplied sources.
Tool calls use the right arguments, and important results were not lost during compaction.
New input usage is genuinely lower than before.

If the error disappears but the model forgets a key constraint, the fix failed. Restore the mandatory block and free space by removing less relevant history or a heavy tool result. If the answer is cut off, inspect the output budget separately; it is another part of the same total window.

Short conclusion

The working sequence is: identify the model → count components → remove exact duplicates → extract irrelevant history → compact files and tool results → leave room for the answer → resend → verify quality. This addresses the cause without turning context into an arbitrarily truncated set of facts.

Sources

Ready to optimize your LLM workflow?

Join thousands of developers building faster, smarter, and more cost-effective AI applications with BetterToken.