API 429 Too Many Requests: Limits, Retry-After, and Safe Retries

An API 429 response means the server rejected a request because of a rate or usage constraint. The correct response is not an infinite retry loop. First inspect the error body and headers, determine whether the condition is temporary or requires billing or quota action, and then perform one bounded retry. Before retrying a mutating operation, verify whether the first request already succeeded.

What 429 actually means

RFC 6585 defines 429 as too many requests in a given period. A response may include Retry-After, but the standard does not prescribe one counting method. A provider may limit a resource, API key, project, organization, or cluster.

That is why the status alone does not tell you how long to wait. A 429 can represent a short RPM or TPM window, exhausted credits, a monthly quota, or a configured spend limit. The current OpenAI error guide, for example, documents separate machine-readable causes for these cases. They must not share one retry path.

Diagnostic table

Symptom Layer to inspect Evidence Safe action
429 after a concurrency spike Burst or short-window limit Retry-After, error body, rate-limit headers Reduce concurrency, wait, then send one probe
429 on large prompts Tokens-per-window constraint Previous usage, input size, max_tokens Reduce or distribute the workload
429 on every request Credit, quota, or spend limit Machine-readable code and billing message Stop retries and fix the account condition
429 for one project or key Limit scope Project/org identity and current tier Check the correct project and its limits
More 429s after retry Retry storm Attempt count, queue, concurrent workers Stop the wave; add jitter and a total deadline
The response is actually 401 or 403 Authentication or permissions HTTP status and error type Fix the key or permissions; do not retry as 429

For requests made through BetterToken, the current API documentation and Dashboard provide a second observation point: request time, model, status, input/output/cache tokens, and cost. That helps reconstruct your own load profile, but it cannot prove why a different provider returned 429.

Respect Retry-After, then bound the fallback

Save the HTTP status, error body, and headers without logging the API key. If a valid Retry-After is present, treat it as the minimum delay and add a small random jitter so workers do not resume simultaneously.

If the header is absent or invalid, fall back to exponential backoff with jitter. Bound both the number of attempts and total retry time. OpenAI's rate-limit guidance warns that failed requests can still consume a per-minute limit, so repeated immediate retries can extend the incident.

function send_with_retry(request, max_attempts, total_deadline):
    attempt = 0
    while attempt < max_attempts and now() < total_deadline:
        response = send(request)
        if response.status != 429:
            return response

        error = parse_error(response.body)
        if error requires billing, quota, or manual action:
            stop without retry

        wait = valid_retry_after(response)
            ? parse_retry_after(response)
            : exponential_delay(attempt)
        sleep(wait + random_jitter())
        attempt += 1

    raise RetryBudgetExhausted

Check whether the SDK already retries eligible failures. Otherwise one application attempt can trigger multiple hidden HTTP requests.

Idempotent reads and dangerous writes

Retrying a status read is usually safer because it should not create a new object. A POST that sends an email, charges a card, starts a job, or creates a record may have completed even when the client lost the response.

Use one of these protections before an automatic retry:

  1. A server-supported idempotency key created before the first attempt and reused unchanged.
  2. A unique business identifier plus a result lookup.
  3. No automatic retry until an operator verifies the actual state.

Do not add an idempotency header by assumption. Verify that the endpoint supports it and understand its retention rules.

Verify recovery without a duplicate

  1. Send one minimal controlled request after the limit window.
  2. Confirm the expected status and response schema.
  3. Check usage and application logs for hidden retries.
  4. For a mutating operation, confirm the side effect occurred exactly once.
  5. Restore traffic gradually while watching the 429 ratio and retry count.

If the probe still reports a quota or credit condition, waiting will not fix it. If 429 returns only as concurrency rises, control the queue and worker count instead of adding more retries.

Final checklist

The safe sequence is: capture the response, classify the 429, take the required account action or honor Retry-After, use bounded backoff with jitter, probe once, and verify that no side effect was duplicated. Infinite retry and a shared handler for 401, 403, 429, and timeout both hide the cause and create more load.

Sources

Related articles