Retries and rate-limit backoff

Written by a person. Last read by a person on 2026-09-04, 4 days ago. Its facts were checked by the eval suite on 2026-09-07.

Your calls fail intermittently and you are wondering whether to retry, you are seeing 429s that never clear no matter how long you wait, or you have a retry loop already and want to know whether it can run up a bill.

The SDK already retries for you. Most of this page is about the cases where that is not enough, and one case where retrying is actively the wrong thing to do.

What you get without writing anything

The official SDKs retry transient failures with exponential backoff, 2 times by default, honoring the retry-after header when the response carries one. Connection errors, rate limits, and server errors are all covered.

client = anthropic.Anthropic(max_retries=8)          # raise it
client = anthropic.Anthropic(max_retries=0)   # or own the loop yourself

Two consequences people miss.

Timeouts are retried too. Wall-clock time for one call can reach the timeout multiplied by attempts plus one. With the default 10-minute timeout and default retries, a single call can occupy your process for half an hour before raising.

Retries are invisible. By the time an exception reaches you, several attempts have already happened. If you then wrap that call in your own retry loop, the counts multiply rather than add: five outer attempts around two inner retries is fifteen requests, not seven.

Which failures are worth retrying

Retrying a request that cannot succeed is not free: it costs latency, it can cost money, and under a rate limit it makes the condition worse.

Catch these as a chain from most specific to least, not as one broad class. The SDK defines a distinct exception per status precisely so you can tell retryable from terminal:

try:
    with client.messages.stream(...) as stream:
        message = stream.get_final_message()
except anthropic.NotFoundError:
    raise                                    # a model ID typo: fail loudly, now
except anthropic.RateLimitError as e:
    wait = e.response.headers.get("retry-after")
    ...
except anthropic.APIStatusError as e:
    if e.status_code >= 500:
        ...                                  # retry with backoff
    else:
        raise                                # other 4xx: terminal
except anthropic.APIConnectionError:
    ...                                      # never reached the API; retry

A single except Exception here is the bug that produces a service which retries a malformed request forty times and reports "the API is flaky".

The 429 that never succeeds

This is the case that justifies the whole page.

Two very different conditions return HTTP 429 with error type rate_limit_error:

An ordinary rate limit. You are sending faster than your limit allows. The response carries a retry-after header. Waiting works. This resolves in seconds.

Your organization's monthly spend cap. API usage is paused until 00:00 UTC on the first of next month. The response carries no retry-after header, and per Anthropic's documentation, retrying — including the SDK's automatic retries — fails until access resumes.

Tell them apart by error.details.error_code, which is enforced_spend_limit_reached for the spend cap:

except anthropic.RateLimitError as e:
    body = e.response.json().get("error", {})
    code = body.get("details", {}).get("error_code")
    if code == "enforced_spend_limit_reached":
        # Retrying will not help before the month rolls over. Page a human.
        raise SpendCapReached(body.get("message"))
    retry_after = e.response.headers.get("retry-after")
    ...

There is a third variant worth knowing: a spend limit you set on your own organization or workspace returns HTTP 400 invalid_request_error, not 429. A retry loop keyed on status code will treat it as a malformed request and give up, which is the right outcome reached by the wrong reasoning — and it means the message never surfaces as a billing problem in your logs.

Reading the headers instead of guessing

Every response carries headers reporting the limit, what is left, and when it replenishes. They are prefixed anthropic-ratelimit-, and there are separate families for requests, input tokens and output tokens.

raw = client.messages.with_raw_response.create(...)
remaining = raw.headers.get("anthropic-ratelimit-requests-remaining")
resets_at = raw.headers.get("anthropic-ratelimit-requests-reset")

The reset values are RFC 3339 timestamps rather than durations. Limits replenish continuously under a token-bucket algorithm rather than resetting on a fixed boundary, so the useful question is usually "how much headroom do I have now", not "when does the window roll".

Worth pinning, because secondary sources get it wrong: the prefix is anthropic-ratelimit-. A header name from memory that no response ever carries reads as zero remaining quota and throttles a service that was never rate-limited.

Backoff that is not hostile

If you write your own loop, three properties matter.

Respect the header. When retry-after is present it is not a suggestion; earlier retries fail.

Exponential, with jitter. Without jitter, everything you throttled retries simultaneously and re-trips the limit. Jitter exists to break that synchronization, not to be polite.

A ceiling on attempts. Which brings us to the point of the section below.

import random, time

def backoff(attempt, retry_after=None, base=1.0, cap=60.0):
    """Seconds to wait before the next attempt."""
    if retry_after is not None:
        return float(retry_after)
    return min(base * (2 ** attempt) + random.uniform(0, 1), cap)

Writing it, three times over

Cost containment is a feature, not a nicety

A retry loop with no ceiling is an unbounded spend authorization written in code.

This is not hypothetical arithmetic. Each retry of a request that has already produced output is billed. A loop that retries until success, against a condition that will not resolve — the spend cap above, a permanently malformed request, a model ID that does not exist — spends money at whatever rate your backoff allows, for as long as the process lives.

The anti-pattern, stated plainly:

while True:                     # no
    try:
        return call()
    except Exception:
        time.sleep(5)

That loop cannot terminate on a terminal error, cannot distinguish a spend cap from a rate limit, and has no upper bound on cost. It is also, in the author's experience, the most common retry implementation in production code, because it looks correct at a glance.

The ceiling this site runs. This documentation site caps retries at 5 attempts, backing off from 1 second and multiplying by 2.

Those are not numbers written into this paragraph. They are read from the same key that the site's own live demo reads, so the discipline described here and the discipline practised here cannot drift apart. The eval suite fails the build if the documented ceiling and the enforced one disagree.

Which is the whole argument: a document that tells you to cap your retries, while the system serving it does not, is advice. A document that reads its number from the thing it is describing is a demonstration.