Streaming and SSE Disconnects: Recovery Without Duplicates

Normal termination or real break

Server-Sent Events send a sequence of events over a long HTTP connection. The client reads them until one of the outcomes:

created -> receiving -> completed
                   \-> failed
                   \-> disconnected

completed — terminal protocol event. failed - error passed inside stream. disconnected means that the transport ended without a confirmed ending. The last case requires diagnosis.

In the OpenAI Responses API, events have types like response.created, output fragments, and response.completed; failure or incomplete states are also possible. Anthropic Messages uses message_start, content block events, message_delta and message_stop. Names cannot be mixed in one parser.

Do you want to reproduce a Streaming break on a controlled request? You can create your own BetterToken and API Key account, open API reference and start with a short stream without tool calls. Then match the time, model and status with the Dashboard entry; only after checking the terminal event and partial output, add a limited retry.

What data to save in case of a break

Minimal logging helps distinguish a client problem from a server problem:

{
  "started_at": "2026-08-03T12:00:00Z",
  "protocol": "openai-responses",
  "request_id": "req_placeholder",
  "http_status": 200,
  "last_event_type": "response.output_text.delta",
  "events_received": 42,
  "bytes_received": 8192,
  "terminal_event_received": false,
  "client_error": "socket closed"
}

Do not save the API Key, full prompt, tool arguments, or sensitive response. Store partial text only where application policy allows it. For production, it is more useful to record hash operations, the number of events and the last safe sequence marker.

Request ID can come in HTTP headers or events. Save it as early as possible, not after the stream has finished.

Check parser before network

The client must correctly process:

• several lines data: in one event;
• empty lines between events;
• UTF-8 fragments at the border of network chunks;
• unknown types of events without process crash;
• error event after successful HTTP status;
• terminal event without the required last text delta;
• tool arguments, divided into several fragments.

TCP chunk is not equal to SSE event. One event can come in parts, and several events can come in one read. First build a full SSE frame, then parse the JSON.

Handler pseudocode:

state = receiving
for event in parse_sse(response_body):
    log_safe_metadata(event)
    apply_event_to_partial_result(event)

    if is_terminal_success(event):
        state = completed
        break
    if is_terminal_failure(event):
        state = failed
        break

if connection_closed and state == receiving:
    state = disconnected

The functions is_terminal_success and is_terminal_failure should be separate for Responses, Chat Completions and Messages.

Check timeout on each layer

A long connection goes through several timers:

1. timeout SDK or HTTP client;
2. idle/read application timeout;
3. reverse proxy;
4. load balancer or ingress;
5. corporate proxy;
6. mobile or home network;
7. server-side generation limit.

General request timeout and idle timeout are different parameters. If the model regularly sends events, a short idle timeout should not fire. If long pauses between events are acceptable, the value must be adjusted to the expected load.

Check buffering proxy. When a proxy accumulates SSE chunks, the user does not see the text for a long time, and then receives a large block or timeout. Test the time of the first event and the intervals between events in each environment: locally, behind a reverse proxy and in production.

Partial output: show, save or discard

Partial text is useful for the interface, but its status must be explicit. Don't show a dangling answer as complete.

Convenient state model:

• streaming - the text is still changing;
• complete — terminal success event received;
• partial - the connection was lost after several events;
• failed — the protocol sent an error;
• cancelled - the request was stopped by the user or application.

For partial, save the already received text separately from the new retry. It is dangerous to automatically glue two generations together: the model may repeat part of the answer, change the wording, or call tools in a different order.

When the request can be repeated

The security of a retry depends on the action.

Text without external actions

A short text request can usually be repeated with a limit of attempts. The application shows the old result as partial and the new one as a separate generation, or replaces it after explicit confirmation.

Tool calls and transactions

Before retrying, check if tool has already been executed. If the stream terminates after sending the command, a second request can re-create an issue, letter or payment transaction. Use an idempotency key at the tool level, your own operation ID and a log of completed actions.

Long agent task

The automatic "continue from last token" is rarely confirmed by the protocol. It is better to restore the task from the saved application state: confirmed messages, tools results and the last completed step. Do not pass off raw text fragments as a consistent agent state.

Limited retry with backoff

A retry policy must have a finite number of attempts:

attempts = 0
while attempts < MAX_ATTEMPTS:
    result = run_request(operation_id)
    if result.completed:
        return result
    if not result.retryable:
        raise result.error
    wait(base_delay * 2**attempts + random_jitter)
    attempts += 1

Retryable is determined by protocol error, HTTP status, presence of terminal event and side effects. 401, incorrect Model ID and invalid JSON will not be corrected by a pause. 429, temporary 5xx or transport failure sometimes allows retry, but only with a limit and taking into account the provider headers.

Minimal test

1. Send a short streaming request without tools.
2. Write down all event types and wait for the terminal event.
3. Artificially terminate the client after several events.
4. Make sure the result is labeled partial.
5. Check one limited retry.
6. Repeat for production proxy.
7. Match both queries by time, model and status in Dashboard.

Take the exact event types from the official documentation: OpenAI streaming Responses and Anthropic Messages streaming.

FAQ

Is it possible to continue stream from the last token?

There is no universal mechanism for this. Save the partial result and application state, then follow the capabilities of the specific API. A new request may repeat or change the text.

Why did HTTP status 200 and stream end with an error?

Headers arrive before full generation. The error may appear later as a protocol event or transport break. Therefore, status alone is not enough.

Do I need to repeat the request after any disconnect?

No. First check terminal event, error, request ID and side effects. Retry for tool call requires idempotency.

Where to look for the reason if everything works locally?

Check reverse proxy, load balancer, idle timeout, buffering and corporate network. Compare the event intervals before and after each layer.

What to check in BetterToken?

Open Dashboard and match time, model, status and usage. Check the current request parameters with API reference; Do not send the full API Key or sensitive prompt to support.