A streaming API request is complete only after the terminal event defined by its protocol. A closed socket, client timeout, or final text fragment does not prove completion. On disconnect, preserve received events, the request ID, and operation state. Retry only after checking side effects: a stream cannot be universally resumed from the last token, and a blind retry may execute a tool twice.

Normal Completion or a Real Disconnect

Server-Sent Events carry a sequence of events over a long-lived HTTP connection. A client reads them until one of these outcomes:

created -> receiving -> completed
                   \-> failed
                   \-> disconnected

completed is the protocol's terminal success event. failed is an error delivered inside the stream. disconnected means the transport ended without a confirmed terminal state. The last case requires diagnosis.

OpenAI Responses emits typed events for response creation, output fragments, and terminal states such as completion, failure, or incompleteness. Anthropic Messages uses message_start, content-block events, message_delta, and message_stop. Do not mix those names in one parser.

Want to reproduce a Streaming disconnect with a controlled request? You can create your own BetterToken account and API key, open the API reference, and start with a short stream without tool calls. Match the time, model, and status with the Dashboard record; add a bounded retry only after checking the terminal event and saved partial output.

What to Preserve When a Stream Breaks

A minimal log helps separate a client failure from a server failure:

{
  "started_at": "2026-08-03T12:00:00Z",
  "protocol": "openai-responses",
  "request_id": "req_placeholder",
  "http_status": 200,
  "last_event_type": "response.output_text.delta",
  "events_received": 42,
  "bytes_received": 8192,
  "terminal_event_received": false,
  "client_error": "socket closed"
}

Do not log the API Key, full prompt, tool arguments, or sensitive output. Store partial text only where application policy allows it. In production, an operation hash, event count, and last safe sequence marker are often more useful than raw content.

A request ID may arrive in HTTP headers or an event. Save it as early as possible rather than waiting for stream completion.

Test the Parser Before Blaming the Network

The client should handle:

multiple data: lines in one event;
blank lines between events;
UTF-8 characters split across network chunks;
unknown event types without crashing;
an error event after a successful HTTP status;
a terminal event without one last text delta;
tool arguments split across several fragments.

A TCP chunk is not an SSE event. One event may arrive in several chunks, while several events may arrive in one chunk. Assemble a complete SSE frame before parsing JSON.

The handler can follow this state machine:

state = receiving
for event in parse_sse(response_body):
    log_safe_metadata(event)
    apply_event_to_partial_result(event)

    if is_terminal_success(event):
        state = completed
        break
    if is_terminal_failure(event):
        state = failed
        break

if connection_closed and state == receiving:
    state = disconnected

Implement is_terminal_success and is_terminal_failure separately for Responses, Chat Completions, and Messages.

Check Every Timeout Layer

A long-lived connection can cross several timers:

SDK or HTTP client timeout;
application idle or read timeout;
reverse proxy;
load balancer or ingress;
corporate proxy;
mobile or home network;
server-side generation limit.

An overall request timeout and an idle timeout are different settings. If events keep arriving, an idle timer should not fire. If long gaps are valid for the workload, configure the value from measured behavior.

Also check proxy buffering. A buffering proxy may hide chunks until it releases a large block or reaches a timeout. Measure the first-event time and intervals between events locally, behind the reverse proxy, and in production.

Mark Partial Output Explicitly

Partial text can be useful in a UI, but it needs an honest state:

streaming — the text is still changing;
complete — a terminal success event arrived;
partial — the connection ended after some events;
failed — the protocol delivered an error;
cancelled — the user or application stopped the request.

For partial, preserve the old text separately from a retry. Automatic concatenation is unsafe because a new generation may repeat content, change wording, or call tools in another order.

Decide Whether a Retry Is Safe

Plain text without external actions

A short text request can usually be retried with a bounded policy. Keep the old output marked as partial and show the new generation separately, or replace it only after explicit confirmation.

Tool calls and transactions

Before retrying, determine whether the tool already ran. If the stream ended after a command was sent, a second request may create another issue, email, record, or payment action. Use an idempotency key at the tool layer, your own operation ID, and a ledger of completed actions.

A long agent task

Generic continuation from the last token is rarely part of the API contract. Recover from saved application state: confirmed messages, completed tool results, and the last durable step. Raw text fragments are not consistent agent state.

Use Bounded Backoff

Retry policy needs a hard attempt limit:

attempts = 0
while attempts < MAX_ATTEMPTS:
    result = run_request(operation_id)
    if result.completed:
        return result
    if not result.retryable:
        raise result.error
    wait(base_delay * 2**attempts + random_jitter)
    attempts += 1

Determine retryable from the protocol error, HTTP status, terminal-event state, and side effects. 401, an invalid Model ID, or invalid JSON does not improve after a delay. A 429, temporary 5xx, or transport fault may permit a retry, but only with limits and the provider's retry headers.

Minimal Test Plan

Send a short streaming request without tools.
Record event types through the terminal event.
Deliberately stop the client after several events.
Confirm that the result is marked partial.
Test one bounded retry.
Repeat the test behind the production proxy.
Match both requests by time, model, and status in the Dashboard.

Take exact event types from the official documentation for OpenAI streaming Responses and Anthropic Messages streaming.

FAQ

Can I resume a stream from the last token?

There is no universal mechanism. Preserve the partial output and application state, then follow the specific API's capabilities. A new request may repeat or change text.

Why did the stream fail after HTTP `200`?

Headers arrive before generation finishes. A protocol error or transport disconnect can happen later, so the initial status is insufficient.

Should every disconnect trigger a retry?

No. Check the terminal event, error, request ID, and side effects first. A tool call requires idempotency before retry.

Where should I look if it works locally?

Inspect the reverse proxy, load balancer, idle timeout, buffering, and corporate network. Compare event intervals before and after each layer.

What should I check in BetterToken?

Match time, model, status, and usage in the Dashboard. Verify current request fields in the API reference, and never send a complete API Key or sensitive prompt to support.

Streaming and SSE Disconnects: Recover Without Duplicates