GPT-6.2 Responses API: Keep the Prefix Unchanged or Reasoning Drops
Starting with GPT-6.2 Astra and Sol, editing earlier Responses API history drops subsequent reasoning items. Keep the prefix unchanged with append-only turns, configuration_update, additional_tools, and developer-first instructions, then verify with input_transformations and Prompt Cache Diagnostics.
Contents

Starting with gpt-6.2-astra and gpt-6.2-sol, the OpenAI Responses API drops reasoning items after a mid-conversation prefix edit, a model downgrade, or an organization or end-user change. This affects every developer and agent team whose code rewrites earlier developer messages, trims past tool results, or changes top-level parameters such as reasoning.effort or tools between turns. Move your integration to an append-only input history, use configuration_update and additional_tools for mid-thread changes, and check input_transformations on every response to confirm no reasoning was removed.
Choose a route for GPT-6.2 Responses API calls
GPT-6.2 Astra and Sol support Chat Completions for text, but tool calling requires the Responses API, and prefix-preservation controls (configuration_update, additional_tools, and input_transformations) are Responses API features. Pick an API route that exposes POST /v1/responses and fits your access and billing setup.
| Route | Requirements | Billing unit | Responses API notes | When to choose |
|---|---|---|---|---|
| BetterToken API gateway | BetterToken account, console API Key, pay-as-you-go balance (paid balance does not reset monthly) | By the model’s current billing unit (input, output, and cache tokens can have different prices/rules); see current prices | https://www.bettertoken.ai/v1 (POST /v1/responses) lists gpt-6.2-astra and gpt-6.2-sol; verify stateless store: false replay, configuration_update, and input_transformations passthrough on your key first | You want one pay-as-you-go API Key across available models or cannot use direct OpenAI billing |
| OpenAI API direct | OpenAI account in a supported country or territory and API billing | Per 1M input, cached input, cache-write, and output tokens | Documents native support for configuration_update, input_transformations, prompt_cache_diagnostics, previous_response_id, background completion windows, and organization-level Zero Data Retention (ZDR) | You have direct OpenAI access and rely on OpenAI-hosted state, ZDR, or documented beta features |
Verify append-only replay and prefix detection through BetterToken
BetterToken provides an OpenAI-compatible API route at https://www.bettertoken.ai/v1 (POST /v1/responses) authenticated with your API Key from the BetterToken console. As of 2026-11-21, both gpt-6.2-sol and gpt-6.2-astra are listed for the Responses endpoint. BetterToken is a multi-model platform and API access layer, not an OpenAI official product. Its Responses API reference does not verify passthrough for stateless store: false encrypted reasoning replay, configuration_update, input_transformations, or prompt_cache_diagnostics. Run this controlled two-turn stateless check on your key before deploying.
Export your BetterToken key in Bash without echoing it:
read -rs BETTERTOKEN_API_KEY && export BETTERTOKEN_API_KEY
Send Turn 1 with store: false, confirm it returns at least one reasoning item with encrypted_content, and compare an append-only stateless replay against an edited-prefix control where the earlier developer message edit is the only difference:
import os
from openai import OpenAI
client = OpenAI(
api_key=os.environ["BETTERTOKEN_API_KEY"],
base_url="https://www.bettertoken.ai/v1",
timeout=120.0,
max_retries=0,
)
model = "gpt-6.2-sol" # Or "gpt-6.2-astra"
history = [
{"role": "developer", "content": "You are a backend reviewer. Give concise engineering answers."},
{"role": "user", "content": "Draft a two-step plan to add a non-blocking index on PostgreSQL."},
]
first = client.responses.create(
model=model,
store=False,
reasoning={"effort": "low"},
input=history,
)
reasoning_items = [item for item in first.output if item.type == "reasoning"]
if not reasoning_items:
raise RuntimeError("Turn 1 returned no reasoning items; use a harder prompt or higher effort.")
if not all(getattr(item, "encrypted_content", None) for item in reasoning_items):
raise RuntimeError("Stateless replay unverified: Turn 1 reasoning items lack encrypted_content.")
reasoning_ids = [item.id for item in reasoning_items]
first_output = [item.to_dict() for item in first.output]
follow_up = {"role": "user", "content": "List the failure modes and rollback steps for step 2."}
append_only = client.responses.create(
model=model,
store=False,
reasoning={"effort": "low"},
input=[*history, *first_output, follow_up],
)
edited_history = [{**history[0], "content": history[0]["content"] + " Be brief."}, history[1]]
edited_control = client.responses.create(
model=model,
store=False,
reasoning={"effort": "low"},
input=[*edited_history, *first_output, follow_up],
)
print("reasoning_ids:", reasoning_ids)
print("append_only:", append_only.status, getattr(append_only, "input_transformations", None))
print("edited_control:", edited_control.status, getattr(edited_control, "input_transformations", None))
print("append_only_cached_tokens:", append_only.usage.input_tokens_details.cached_tokens)
Read four signals before shipping:
status == "completed"confirms only that the request was accepted and finished. It does not prove that the gateway preserved reasoning across turns.- Confirm Turn 1 with
store: falsereturns non-emptyreasoning_idsand that every reasoning item includesencrypted_content. If the gateway omitsencrypted_content, treat stateless replay as unverified because Turn 2 cannot carry encrypted reasoning tokens across upstreams. - Compare
input_transformationsacross both Turn 2 calls. A verified passthrough reports aprefix_mismatchentry matching an ID inreasoning_idsonedited_controland none onappend_only. Ifinput_transformationsisNoneor empty in both runs, treat the check as inconclusive because the gateway may not forward that field. - Test
configuration_updatepassthrough separately from the prefix check. You can verify atext.formatupdate by sending a separate Turn 2 request that switchestext.formatto a strict JSON schema and checking that the output matches that schema. OpenAI does not document a response field that confirms a mid-threadreasoning.effortupdate (the response’sreasoning.effortstill reports the request-level setting), so treatconfiguration_updateforreasoning.effortas unverified through BetterToken.
For prompt caching, OpenAI sets a minimum cacheable prefix of 1,024 visible input tokens on GPT-5.6 and later models as an eligibility condition. Use append_only.usage.input_tokens_details.cached_tokens to observe actual reuse once your reusable prefix meets that threshold. Check current prices for current model rates and cache rules before scaling.
If you call OpenAI directly instead, omit base_url, use OPENAI_API_KEY, and follow OpenAI’s Keep the prefix unchanged guide.
What counts as a prefix change in GPT-6.2
A reasoning item’s prefix is the conversation history rendered before it. On GPT-6.2, modifying that earlier history causes the API to drop reasoning items that follow the edit.
OpenAI documents two categories of prefix changes:
- Editing earlier
inputitems: shortening a tool result, rewriting adevelopermessage, downsizing past images or files, or replacing older turns with a summary. - Mutating request-level parameters between turns: updating
tools(adding, removing, reordering, or editing schemas and descriptions), changingtext.format, changing top-levelreasoning.effort, or alteringinstructionson pre-GPT-6.2 models.
In OpenAI’s Reasoning guide, prefix_mismatch triggers on GPT-6.2 and later when you insert or edit content before a reasoning item. Deleting whole items without editing remaining items, or appending items after a reasoning item, does not by itself trigger prefix_mismatch. However, removing old turns can still reduce prompt cache reuse, and Prompt Cache Diagnostics may report input_changed when that change prevents reuse.
Prefix edits are one of five removal reasons reported in input_transformations:
Removal reason | When the API removes the reasoning item |
|---|---|
prefix_mismatch | On GPT-6.2 and later, content before the reasoning item was inserted or edited |
end_user_mismatch | On GPT-6.2 and later, safety_identifier differs between the creating and reusing requests, including when omitted on only one request |
model_mismatch | You switched model families or downgraded within the same family (upgrading within the same family preserves compatible reasoning if other checks pass) |
organization_mismatch | You reused an item from another organization or a different workspace under a different user |
turn_boundary | The item comes from an earlier turn when the model uses current-turn reasoning only (reasoning.context controls eligibility on supported models) |
What happens when reasoning items are dropped
A Responses API request can complete even when reasoning items are withheld, so reasoning loss stays silent unless you inspect input_transformations on the response. Mid-conversation prefix edits create two separate production penalties:
- Impact of dropped reasoning: when reasoning items are removed, the model may have to redo earlier reasoning, which can increase token usage, cost, and latency. If the model cannot recover the lost reasoning, output quality may also drop.
- Impact of changing the cacheable prefix: because prompt caching matches requests from the beginning, editing the prefix can cut cache reuse from that point onward, raising input costs and slowing responses.
OpenAI’s published Standard rates as of 2026-11-21 (for 272,000 input tokens or fewer) show why cache misses add up quickly:
- GPT-6.2 Astra (
gpt-6.2-astra): $0.40/1M cached input vs. $8/1M uncached input ($10/1M for 30-minute cache writes, $20/1M for 12-hour cache writes) and $40/1M output. - GPT-6.2 Sol (
gpt-6.2-sol): $0.10/1M cached input vs. $2/1M uncached input ($2.50/1M for 30-minute cache writes, $5/1M for 12-hour cache writes) and $10/1M output.
Above 272,000 input tokens, OpenAI doubles input and cache rates and charges 1.5x the short-context output rate for the full request. To decide when 12-hour writes pay off on stable prefixes, see our guide on OpenAI 12-hour prompt caching.
Update parameters and context without breaking the prefix
Replace in-place mutations with append-only items so reasoning and cached prefixes survive across turns.
Change reasoning.effort and text.format with configuration_update
Changing top-level reasoning.effort can rewrite hidden system instructions. On supported GPT-6 and later models in standard single-agent mode, keep request-level reasoning.effort and text.format unchanged, and append a configuration_update item to input before the next user message:
{
"type": "configuration_update",
"reasoning": {
"effort": "high"
},
"text": {
"format": {
"type": "json_schema",
"name": "migration_checks",
"strict": true,
"schema": {
"type": "object",
"properties": {
"safe": { "type": "boolean" },
"rollback_sql": { "type": "string" }
},
"required": ["safe", "rollback_sql"],
"additionalProperties": false
}
}
}
}
Follow five rules when using configuration_update:
- A
configuration_updatestays in effect across turns until another update overrides it. Updatingreasoning.effortalone preserves an earliertext.formatupdate, and vice versa. - The response object’s
reasoning.effortstill reports the top-level request value, not the updated effort. - Carry updates forward with
previous_response_id, or replay eachconfiguration_updatein its original position when managinginputmanually. - Automatic truncation and the standalone
/responses/compactendpoint reject histories containingconfiguration_updateitems. Usecontext_managementorcompaction_triggerinstead (see compaction below). - Both
gpt-6.2-astraandgpt-6.2-solrejectreasoning.effort: "none". OpenAI’s Using GPT-6.2 guide recommends starting withmedium, or usinglowinstead ofnoneorminimal.
Add or restrict tools without mutating tools
Keep the request-level tools array, schemas, and order unchanged across turns. Adjust callable tools with three append-safe patterns:
- Set
tool_choiceto"none"to disable tools for one turn without removingtools. - Use
tool_choicewithallowed_toolsto restrict callable tools for one turn:
{
"tool_choice": {
"type": "allowed_tools",
"mode": "auto",
"tools": [
{ "type": "function", "name": "get_weather" },
{ "type": "function", "name": "search_docs" }
]
}
}
- Append a developer-role
additional_toolsitem to add tools mid-conversation, or use tool search withdefer_loading: trueso discovered tools append at the end of context. Tools inadditional_toolsbecome callable only after that item appears, so replay it in its original position:
{
"type": "additional_tools",
"role": "developer",
"tools": [
{
"type": "function",
"name": "get_customer",
"description": "Look up a customer by ID.",
"parameters": {
"type": "object",
"properties": {
"customer_id": { "type": "string" }
},
"required": ["customer_id"],
"additionalProperties": false
}
}
]
}
Split reusable developer messages from top-level instructions
Starting with GPT-6.2, OpenAI appends top-level instructions after input, beyond eligible cache breakpoints. OpenAI bills those tokens at the uncached input rate on every request even if unchanged, and top-level instructions do not carry over with previous_response_id.
Place instructions by scope:
- Put reusable system prompts and shared policies in a
developermessage as the first item ininputso prompt caching can match them. - Append a new
developermessage at the end ofinputwhen updating reusable instructions mid-thread instead of editing the first one. - Use top-level
instructionsonly for guidance specific to the current request.
Handle compaction and long tool outputs
Shorten long tool outputs and downsize images or files before sending them to the model for the first time. Do not truncate past items retroactively.
Compaction changes the prefix, so cache misses are hard to avoid. For long sessions, OpenAI’s Compaction guide documents two paths:
- Automatic compaction (
context_managementwithcompact_threshold): OpenAI retains subsequent reasoning items despite the prefix change. If earlier turns containconfiguration_updateitems, the API emits one freshconfiguration_updateright after thecompactionitem and storesadditional_toolsdeclarations inside the encrypted compaction item. Replay returned output items in order. - Explicit or custom compaction: standalone
/responses/compactrejects histories containingconfiguration_updateitems, and when calling/responses/compact, omitinstructionsand pass the returned window in its original order. If your history usesconfiguration_update, include acompaction_triggeritem in/responsesinstead, then append a freshconfiguration_updaterestating each changed field before the next user message. For custom compaction, place the compacted summary at the end of context instead of replacing older turns in place, or accept dropped reasoning items.
Verify prefix preservation with input_transformations and Prompt Cache Diagnostics
Pair two response checks: one reports withheld reasoning items, and the other identifies what broke prompt cache reuse.
1. Inspect input_transformations on every response
In the Responses API create reference, input_transformations is an optional array of {id, action, reason, type} objects listing withheld reasoning items in input order (action is "removed" and type is "reasoning"). When reason is "prefix_mismatch", match id to a reasoning item in an earlier response’s output to locate the edit before it.
2. Run Prompt Cache Diagnostics with comparison_response_id
To explain lower-than-expected usage.input_tokens_details.cached_tokens, choose a recent completed response from the same organization, pass its id in prompt_cache_options.comparison_response_id, and inspect prompt_cache_diagnostics:
{
"prompt_cache_diagnostics": {
"type": "cache_miss",
"reason": "tools_changed",
"comparison_reusable_tokens": 5629,
"cache_missed_tokens": 5629
}
}
OpenAI’s Prompt cache diagnostics guide returns prompt_cache_diagnostics.type as cache_hit, cache_miss, comparison_response_not_found, or unavailable. For cache_miss, it returns reason and cache_missed_tokens (estimated tokens not reused), and may include comparison_reusable_tokens (raw token count of the baseline’s reusable prefix). Misses are classified by reason:
input_changed: earlier input changed, such as a timestamp or request ID added to adevelopermessage, or old turns edited, reordered, or removed.tools_changed: tools were added, removed, reordered, or edited.reasoning_effort_changedortext_format_changed: top-levelreasoning.effortortext.formatchanged instead of usingconfiguration_update.model_changed,service_tier_changed,verbosity_changed,prompt_cache_key_changed, orcontext_compacted: model selection, service tier, verbosity, cache key, or compaction changed.
Keep five diagnostic limits in mind: diagnostics require GPT-5.6 or later and a reusable prefix of at least 1,024 tokens; they report only the first classified reason on a best-effort basis; they add no extra fee and work with Zero Data Retention; when stream=True, read prompt_cache_diagnostics from event.response in the response.completed event; and diagnostics cannot verify whether the model reused reasoning.
Watch these details in custom API and gateway setups
Teams routing GPT-6.2 through a custom API base URL or multi-upstream gateway should verify four details:
- Call
POST /v1/responses, not Chat Completions: GPT-6.2 Astra and Sol support Chat Completions for text, but tool calling,configuration_update,additional_tools, andinput_transformationsrequire Responses. For a base migration checklist, see our GPT-6 Astra Responses API migration guide. - Replay full append-only
inputarrays instead of relying only onprevious_response_id: BetterToken’s smart routing sends requests to currently available upstream channels, and a model with several configured upstreams can fail over on an applicable error or timeout (not every model has several upstreams, not every error triggers failover, and success is not guaranteed). Stateless replay avoids server-side state tied to one upstream. On OpenAI’s API, Zero Data Retention (ZDR) is an organization-level setting for approved organizations on eligible endpoints and features, distinct from settingstoretofalseon a request. In either stateless mode (store: falseor an organization with ZDR enabled, where OpenAI treatsstoreasfalse), reasoning items inoutputincludeencrypted_contentby default. OpenAI still accepts legacyinclude: ["reasoning.encrypted_content"]for compatibility, but does not require it. Replay all returnedoutputitems in order, keepsafety_identifieridentical across turns (or omit it on every turn) to avoidend_user_mismatch, and monitorinput_transformationsfororganization_mismatch. - Omit unsupported sampling parameters: for Responses calls to
gpt-6.2-astraandgpt-6.2-sol, omit the top-leveltemperature,top_p, andtop_logprobsparameters, and remove onlymessage.output_text.logprobsfrominclude. - Scope background completion windows to direct OpenAI calls: on OpenAI’s Responses API, GPT-6.2 Astra and Sol do not support
service_tier: "flex". For long-running tasks called directly on OpenAI’s API, use background mode’s 2-hour or 8-hour completion windows (public beta) as detailed in our Responses API completion_window guide. Background completion windows are unverified in BetterToken’s public Responses API documentation, so verify or route directly to OpenAI if your workload depends on them.