GPT-6.2 Responses API: Keep the Prefix Unchanged or Reasoning Drops

Starting with GPT-6.2 Astra and Sol, editing earlier Responses API history drops subsequent reasoning items. Keep the prefix unchanged with append-only turns, configuration_update, additional_tools, and developer-first instructions, then verify with input_transformations and Prompt Cache Diagnostics.

Contents
GPT-6.2 Responses API: Keep the Prefix Unchanged or Reasoning Drops

Starting with gpt-6.2-astra and gpt-6.2-sol, the OpenAI Responses API drops reasoning items after a mid-conversation prefix edit, a model downgrade, or an organization or end-user change. This affects every developer and agent team whose code rewrites earlier developer messages, trims past tool results, or changes top-level parameters such as reasoning.effort or tools between turns. Move your integration to an append-only input history, use configuration_update and additional_tools for mid-thread changes, and check input_transformations on every response to confirm no reasoning was removed.

Choose a route for GPT-6.2 Responses API calls

GPT-6.2 Astra and Sol support Chat Completions for text, but tool calling requires the Responses API, and prefix-preservation controls (configuration_update, additional_tools, and input_transformations) are Responses API features. Pick an API route that exposes POST /v1/responses and fits your access and billing setup.

RouteRequirementsBilling unitResponses API notesWhen to choose
BetterToken API gatewayBetterToken account, console API Key, pay-as-you-go balance (paid balance does not reset monthly)By the model’s current billing unit (input, output, and cache tokens can have different prices/rules); see current priceshttps://www.bettertoken.ai/v1 (POST /v1/responses) lists gpt-6.2-astra and gpt-6.2-sol; verify stateless store: false replay, configuration_update, and input_transformations passthrough on your key firstYou want one pay-as-you-go API Key across available models or cannot use direct OpenAI billing
OpenAI API directOpenAI account in a supported country or territory and API billingPer 1M input, cached input, cache-write, and output tokensDocuments native support for configuration_update, input_transformations, prompt_cache_diagnostics, previous_response_id, background completion windows, and organization-level Zero Data Retention (ZDR)You have direct OpenAI access and rely on OpenAI-hosted state, ZDR, or documented beta features

Verify append-only replay and prefix detection through BetterToken

BetterToken provides an OpenAI-compatible API route at https://www.bettertoken.ai/v1 (POST /v1/responses) authenticated with your API Key from the BetterToken console. As of 2026-11-21, both gpt-6.2-sol and gpt-6.2-astra are listed for the Responses endpoint. BetterToken is a multi-model platform and API access layer, not an OpenAI official product. Its Responses API reference does not verify passthrough for stateless store: false encrypted reasoning replay, configuration_update, input_transformations, or prompt_cache_diagnostics. Run this controlled two-turn stateless check on your key before deploying.

Export your BetterToken key in Bash without echoing it:

read -rs BETTERTOKEN_API_KEY && export BETTERTOKEN_API_KEY

Send Turn 1 with store: false, confirm it returns at least one reasoning item with encrypted_content, and compare an append-only stateless replay against an edited-prefix control where the earlier developer message edit is the only difference:

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["BETTERTOKEN_API_KEY"],
    base_url="https://www.bettertoken.ai/v1",
    timeout=120.0,
    max_retries=0,
)

model = "gpt-6.2-sol"  # Or "gpt-6.2-astra"
history = [
    {"role": "developer", "content": "You are a backend reviewer. Give concise engineering answers."},
    {"role": "user", "content": "Draft a two-step plan to add a non-blocking index on PostgreSQL."},
]

first = client.responses.create(
    model=model,
    store=False,
    reasoning={"effort": "low"},
    input=history,
)
reasoning_items = [item for item in first.output if item.type == "reasoning"]
if not reasoning_items:
    raise RuntimeError("Turn 1 returned no reasoning items; use a harder prompt or higher effort.")
if not all(getattr(item, "encrypted_content", None) for item in reasoning_items):
    raise RuntimeError("Stateless replay unverified: Turn 1 reasoning items lack encrypted_content.")

reasoning_ids = [item.id for item in reasoning_items]
first_output = [item.to_dict() for item in first.output]
follow_up = {"role": "user", "content": "List the failure modes and rollback steps for step 2."}

append_only = client.responses.create(
    model=model,
    store=False,
    reasoning={"effort": "low"},
    input=[*history, *first_output, follow_up],
)

edited_history = [{**history[0], "content": history[0]["content"] + " Be brief."}, history[1]]
edited_control = client.responses.create(
    model=model,
    store=False,
    reasoning={"effort": "low"},
    input=[*edited_history, *first_output, follow_up],
)

print("reasoning_ids:", reasoning_ids)
print("append_only:", append_only.status, getattr(append_only, "input_transformations", None))
print("edited_control:", edited_control.status, getattr(edited_control, "input_transformations", None))
print("append_only_cached_tokens:", append_only.usage.input_tokens_details.cached_tokens)

Read four signals before shipping:

  • status == "completed" confirms only that the request was accepted and finished. It does not prove that the gateway preserved reasoning across turns.
  • Confirm Turn 1 with store: false returns non-empty reasoning_ids and that every reasoning item includes encrypted_content. If the gateway omits encrypted_content, treat stateless replay as unverified because Turn 2 cannot carry encrypted reasoning tokens across upstreams.
  • Compare input_transformations across both Turn 2 calls. A verified passthrough reports a prefix_mismatch entry matching an ID in reasoning_ids on edited_control and none on append_only. If input_transformations is None or empty in both runs, treat the check as inconclusive because the gateway may not forward that field.
  • Test configuration_update passthrough separately from the prefix check. You can verify a text.format update by sending a separate Turn 2 request that switches text.format to a strict JSON schema and checking that the output matches that schema. OpenAI does not document a response field that confirms a mid-thread reasoning.effort update (the response’s reasoning.effort still reports the request-level setting), so treat configuration_update for reasoning.effort as unverified through BetterToken.

For prompt caching, OpenAI sets a minimum cacheable prefix of 1,024 visible input tokens on GPT-5.6 and later models as an eligibility condition. Use append_only.usage.input_tokens_details.cached_tokens to observe actual reuse once your reusable prefix meets that threshold. Check current prices for current model rates and cache rules before scaling.

If you call OpenAI directly instead, omit base_url, use OPENAI_API_KEY, and follow OpenAI’s Keep the prefix unchanged guide.

Create a BetterToken account

What counts as a prefix change in GPT-6.2

A reasoning item’s prefix is the conversation history rendered before it. On GPT-6.2, modifying that earlier history causes the API to drop reasoning items that follow the edit.

OpenAI documents two categories of prefix changes:

  • Editing earlier input items: shortening a tool result, rewriting a developer message, downsizing past images or files, or replacing older turns with a summary.
  • Mutating request-level parameters between turns: updating tools (adding, removing, reordering, or editing schemas and descriptions), changing text.format, changing top-level reasoning.effort, or altering instructions on pre-GPT-6.2 models.

In OpenAI’s Reasoning guide, prefix_mismatch triggers on GPT-6.2 and later when you insert or edit content before a reasoning item. Deleting whole items without editing remaining items, or appending items after a reasoning item, does not by itself trigger prefix_mismatch. However, removing old turns can still reduce prompt cache reuse, and Prompt Cache Diagnostics may report input_changed when that change prevents reuse.

Prefix edits are one of five removal reasons reported in input_transformations:

Removal reasonWhen the API removes the reasoning item
prefix_mismatchOn GPT-6.2 and later, content before the reasoning item was inserted or edited
end_user_mismatchOn GPT-6.2 and later, safety_identifier differs between the creating and reusing requests, including when omitted on only one request
model_mismatchYou switched model families or downgraded within the same family (upgrading within the same family preserves compatible reasoning if other checks pass)
organization_mismatchYou reused an item from another organization or a different workspace under a different user
turn_boundaryThe item comes from an earlier turn when the model uses current-turn reasoning only (reasoning.context controls eligibility on supported models)

What happens when reasoning items are dropped

A Responses API request can complete even when reasoning items are withheld, so reasoning loss stays silent unless you inspect input_transformations on the response. Mid-conversation prefix edits create two separate production penalties:

  1. Impact of dropped reasoning: when reasoning items are removed, the model may have to redo earlier reasoning, which can increase token usage, cost, and latency. If the model cannot recover the lost reasoning, output quality may also drop.
  2. Impact of changing the cacheable prefix: because prompt caching matches requests from the beginning, editing the prefix can cut cache reuse from that point onward, raising input costs and slowing responses.

OpenAI’s published Standard rates as of 2026-11-21 (for 272,000 input tokens or fewer) show why cache misses add up quickly:

  • GPT-6.2 Astra (gpt-6.2-astra): $0.40/1M cached input vs. $8/1M uncached input ($10/1M for 30-minute cache writes, $20/1M for 12-hour cache writes) and $40/1M output.
  • GPT-6.2 Sol (gpt-6.2-sol): $0.10/1M cached input vs. $2/1M uncached input ($2.50/1M for 30-minute cache writes, $5/1M for 12-hour cache writes) and $10/1M output.

Above 272,000 input tokens, OpenAI doubles input and cache rates and charges 1.5x the short-context output rate for the full request. To decide when 12-hour writes pay off on stable prefixes, see our guide on OpenAI 12-hour prompt caching.

Update parameters and context without breaking the prefix

Replace in-place mutations with append-only items so reasoning and cached prefixes survive across turns.

Change reasoning.effort and text.format with configuration_update

Changing top-level reasoning.effort can rewrite hidden system instructions. On supported GPT-6 and later models in standard single-agent mode, keep request-level reasoning.effort and text.format unchanged, and append a configuration_update item to input before the next user message:

{
  "type": "configuration_update",
  "reasoning": {
    "effort": "high"
  },
  "text": {
    "format": {
      "type": "json_schema",
      "name": "migration_checks",
      "strict": true,
      "schema": {
        "type": "object",
        "properties": {
          "safe": { "type": "boolean" },
          "rollback_sql": { "type": "string" }
        },
        "required": ["safe", "rollback_sql"],
        "additionalProperties": false
      }
    }
  }
}

Follow five rules when using configuration_update:

  • A configuration_update stays in effect across turns until another update overrides it. Updating reasoning.effort alone preserves an earlier text.format update, and vice versa.
  • The response object’s reasoning.effort still reports the top-level request value, not the updated effort.
  • Carry updates forward with previous_response_id, or replay each configuration_update in its original position when managing input manually.
  • Automatic truncation and the standalone /responses/compact endpoint reject histories containing configuration_update items. Use context_management or compaction_trigger instead (see compaction below).
  • Both gpt-6.2-astra and gpt-6.2-sol reject reasoning.effort: "none". OpenAI’s Using GPT-6.2 guide recommends starting with medium, or using low instead of none or minimal.

Add or restrict tools without mutating tools

Keep the request-level tools array, schemas, and order unchanged across turns. Adjust callable tools with three append-safe patterns:

  • Set tool_choice to "none" to disable tools for one turn without removing tools.
  • Use tool_choice with allowed_tools to restrict callable tools for one turn:
{
  "tool_choice": {
    "type": "allowed_tools",
    "mode": "auto",
    "tools": [
      { "type": "function", "name": "get_weather" },
      { "type": "function", "name": "search_docs" }
    ]
  }
}
  • Append a developer-role additional_tools item to add tools mid-conversation, or use tool search with defer_loading: true so discovered tools append at the end of context. Tools in additional_tools become callable only after that item appears, so replay it in its original position:
{
  "type": "additional_tools",
  "role": "developer",
  "tools": [
    {
      "type": "function",
      "name": "get_customer",
      "description": "Look up a customer by ID.",
      "parameters": {
        "type": "object",
        "properties": {
          "customer_id": { "type": "string" }
        },
        "required": ["customer_id"],
        "additionalProperties": false
      }
    }
  ]
}

Split reusable developer messages from top-level instructions

Starting with GPT-6.2, OpenAI appends top-level instructions after input, beyond eligible cache breakpoints. OpenAI bills those tokens at the uncached input rate on every request even if unchanged, and top-level instructions do not carry over with previous_response_id.

Place instructions by scope:

  • Put reusable system prompts and shared policies in a developer message as the first item in input so prompt caching can match them.
  • Append a new developer message at the end of input when updating reusable instructions mid-thread instead of editing the first one.
  • Use top-level instructions only for guidance specific to the current request.

Handle compaction and long tool outputs

Shorten long tool outputs and downsize images or files before sending them to the model for the first time. Do not truncate past items retroactively.

Compaction changes the prefix, so cache misses are hard to avoid. For long sessions, OpenAI’s Compaction guide documents two paths:

  • Automatic compaction (context_management with compact_threshold): OpenAI retains subsequent reasoning items despite the prefix change. If earlier turns contain configuration_update items, the API emits one fresh configuration_update right after the compaction item and stores additional_tools declarations inside the encrypted compaction item. Replay returned output items in order.
  • Explicit or custom compaction: standalone /responses/compact rejects histories containing configuration_update items, and when calling /responses/compact, omit instructions and pass the returned window in its original order. If your history uses configuration_update, include a compaction_trigger item in /responses instead, then append a fresh configuration_update restating each changed field before the next user message. For custom compaction, place the compacted summary at the end of context instead of replacing older turns in place, or accept dropped reasoning items.

Verify prefix preservation with input_transformations and Prompt Cache Diagnostics

Pair two response checks: one reports withheld reasoning items, and the other identifies what broke prompt cache reuse.

1. Inspect input_transformations on every response

In the Responses API create reference, input_transformations is an optional array of {id, action, reason, type} objects listing withheld reasoning items in input order (action is "removed" and type is "reasoning"). When reason is "prefix_mismatch", match id to a reasoning item in an earlier response’s output to locate the edit before it.

2. Run Prompt Cache Diagnostics with comparison_response_id

To explain lower-than-expected usage.input_tokens_details.cached_tokens, choose a recent completed response from the same organization, pass its id in prompt_cache_options.comparison_response_id, and inspect prompt_cache_diagnostics:

{
  "prompt_cache_diagnostics": {
    "type": "cache_miss",
    "reason": "tools_changed",
    "comparison_reusable_tokens": 5629,
    "cache_missed_tokens": 5629
  }
}

OpenAI’s Prompt cache diagnostics guide returns prompt_cache_diagnostics.type as cache_hit, cache_miss, comparison_response_not_found, or unavailable. For cache_miss, it returns reason and cache_missed_tokens (estimated tokens not reused), and may include comparison_reusable_tokens (raw token count of the baseline’s reusable prefix). Misses are classified by reason:

  • input_changed: earlier input changed, such as a timestamp or request ID added to a developer message, or old turns edited, reordered, or removed.
  • tools_changed: tools were added, removed, reordered, or edited.
  • reasoning_effort_changed or text_format_changed: top-level reasoning.effort or text.format changed instead of using configuration_update.
  • model_changed, service_tier_changed, verbosity_changed, prompt_cache_key_changed, or context_compacted: model selection, service tier, verbosity, cache key, or compaction changed.

Keep five diagnostic limits in mind: diagnostics require GPT-5.6 or later and a reusable prefix of at least 1,024 tokens; they report only the first classified reason on a best-effort basis; they add no extra fee and work with Zero Data Retention; when stream=True, read prompt_cache_diagnostics from event.response in the response.completed event; and diagnostics cannot verify whether the model reused reasoning.

Watch these details in custom API and gateway setups

Teams routing GPT-6.2 through a custom API base URL or multi-upstream gateway should verify four details:

  1. Call POST /v1/responses, not Chat Completions: GPT-6.2 Astra and Sol support Chat Completions for text, but tool calling, configuration_update, additional_tools, and input_transformations require Responses. For a base migration checklist, see our GPT-6 Astra Responses API migration guide.
  2. Replay full append-only input arrays instead of relying only on previous_response_id: BetterToken’s smart routing sends requests to currently available upstream channels, and a model with several configured upstreams can fail over on an applicable error or timeout (not every model has several upstreams, not every error triggers failover, and success is not guaranteed). Stateless replay avoids server-side state tied to one upstream. On OpenAI’s API, Zero Data Retention (ZDR) is an organization-level setting for approved organizations on eligible endpoints and features, distinct from setting store to false on a request. In either stateless mode (store: false or an organization with ZDR enabled, where OpenAI treats store as false), reasoning items in output include encrypted_content by default. OpenAI still accepts legacy include: ["reasoning.encrypted_content"] for compatibility, but does not require it. Replay all returned output items in order, keep safety_identifier identical across turns (or omit it on every turn) to avoid end_user_mismatch, and monitor input_transformations for organization_mismatch.
  3. Omit unsupported sampling parameters: for Responses calls to gpt-6.2-astra and gpt-6.2-sol, omit the top-level temperature, top_p, and top_logprobs parameters, and remove only message.output_text.logprobs from include.
  4. Scope background completion windows to direct OpenAI calls: on OpenAI’s Responses API, GPT-6.2 Astra and Sol do not support service_tier: "flex". For long-running tasks called directly on OpenAI’s API, use background mode’s 2-hour or 8-hour completion windows (public beta) as detailed in our Responses API completion_window guide. Background completion windows are unverified in BetterToken’s public Responses API documentation, so verify or route directly to OpenAI if your workload depends on them.

Ready to optimize your LLM workflow?

Join thousands of developers building faster, smarter, and more cost-effective AI applications with BetterToken.

Get Started for Free