APIsReliability

HTTP 429: what it means and how to retry

Vishnujan Narayanan July 2026

One status code, three meanings — wait, stop, or out of quota. How to tell them apart in code.

Two messages from the same API, six hours apart, both arriving as HTTP 429:

429 Rate limit reached ... on tokens per minute (TPM):
    Limit 12000, Used 10986. Please try again in 3.035s.

429 Your project has exceeded its monthly spending cap.

The first clears in three seconds. The second holds until somebody raises a limit or the month rolls over. A retry loop treating them the same handles one of them badly, and which one depends on which way you guessed.

The status code cannot tell you which you have. RFC 6585 defines 429 in about a paragraph and is deliberately silent on which limit was hit or for how long. Everything that distinguishes those two messages is in prose written for a human. This is what I do about that now, after getting it wrong twice in ways that cost whole runs.

Exponential backoff and its assumption

The usual recipe is exponential backoff with jitter, and it is correct as far as it goes. Wait, double the wait each failure, add a random offset so that everything which failed together does not retry together. tenacity gives you this in a decorator; urllib3's Retry gives it to you inside requests with a backoff_jitter parameter, and either beats writing it by hand.

except Exception as exc:
    if attempt < attempts:
        time.sleep(delay)
        delay = min(delay * 2, max_delay)

Written that way the loop encodes a belief: that every failure is worth waiting for. Mine ran batches of 25 items, five attempts each, capped at 90 seconds. When a provider retired a model and every call started returning 404 no longer available, that belief turned a permanent failure into roughly four minutes of waiting per item across all 25. The run did not crash. It produced identical errors for a quarter of an hour and then reported that it was done.

The fix people reach for is a two-way split: transient things like 503s, timeouts and 429s get retried, permanent things like a 404 or a bad key fail immediately. I wrote exactly that, and left a comment noting that 429 was deliberately absent from the permanent list because rate limiting is transient by definition. The spend-cap message arrived a few hours later.

The third class: exhaustion and scope

A spend cap is not recoverable within the run, but that is not what makes it different. What makes it different is that it tells you something about every subsequent call. A 404 on item 3 says nothing about item 4. A spend cap on item 3 tells you precisely what items 4 through 25 will do.

So the question that assigns a class is not "will this work if I try again" but "does this failure predict the next call's failure". Transient failures are scoped to the call and you retry them. Permanent failures are scoped to the call and you skip that item and continue. Exhaustion is scoped to the account, and the correct response is to stop the batch or move to a different provider, because everything after this will fail the same way.

Some providers spare you the guesswork. Where RateLimit headers are present you get the limit, what remains and when the window resets as structured fields, and the better move is to read remaining before you send and throttle yourself rather than reacting to a refusal at all. It is still a draft, adoption is patchy, and the older X-RateLimit-* spellings are more common in the wild, so check what your provider actually sends before writing a parser for prose.

Classifying on message text

Two things surprised me when I got to this. The first is that you cannot classify by exception type: between an SDK, a structured-output wrapper and the transport, one failure arrives as three nested classes wrapping each other, and the status text is the only thing that survives every layer intact.

The second is that error messages are written for humans, and sometimes for marketing. My list of exhaustion markers included the substring billing, which seemed safe, because spend-cap messages mention billing. Then I read a full rate-limit message from the provider I was using most:

429 Rate limit reached for model `llama-3.3-70b-versatile` on tokens per minute
(TPM): Limit 12000, Used 10986, Requested 1621. Please try again in 3.035s.
Need more tokens? Upgrade to Dev Tier today at https://console.groq.com/settings/billing

Routine throttling that would clear in three seconds, carrying an advertisement containing the word I was treating as proof the account was finished. The classifier promoted it to exhaustion and moved to a fallback that was itself spend-capped. Two runs ended early that way, at 5 of 21 jobs and 4 of 13.

The taxonomy was fine. The evidence I was using to detect one of its classes was contaminated by text that had nothing to do with the error. What fixed it was ordering the checks so a narrow signal beats a broad one.

def _is_budget_exhausted(exc) -> bool:
    message = str(exc).casefold()
    if any(m in message for m in _DAILY_QUOTA_MARKERS):     # "tokens per day"
        return True
    if any(m in message for m in _TRANSIENT_RATE_MARKERS):  # "tokens per minute"
        return False
    return any(m in message for m in _BUDGET_ERROR_MARKERS)

Removing billing opened a gap in the other direction, because one provider signals a genuine paywall with 402 payment_required and that then read as transient. Adding it explicitly closed that. The only reason I caught it before it ran was that I had saved four real error strings and pushed them through the classifier, printing what each came out as. Four strings and four expected answers is the whole test, and it is the highest-value thing in this post.

There is a subtlety in the ordering that took me a second pass to see. The same provider words its per-minute and per-day limits almost identically:

... rate limit reached ... on tokens per minute (TPM): Limit 12000 ...
... rate limit reached ... on tokens per day   (TPD): Limit 100000, Used 98752 ...

Per-minute reopens inside the run and per-day does not, so a daily quota is exhaustion for this run's purposes even though it shares its opening clause with the case that clears in seconds. That is why the daily markers are checked first: it is the narrowest signal available, and it has to outrank wording it shares with the transient case.

Retry-After: honouring the stated wait

Separately from classification, I was wasting attempts by ignoring what the provider told me. Where a limit is transient, the message usually says how long it will last.

Please try again in 11.344999999s

My loop backed off on its own schedule instead — two seconds, four, eight — arriving before the window reopened every time, being refused every time, and burning all five attempts on a limit that would have cleared by itself in eleven seconds. Then it escalated to a fallback it did not need.

hinted = retry_after_seconds(exc)      # parsed out of the message
wait = hinted if hinted is not None else delay
time.sleep(wait)
if hinted is None:
    delay = min(delay * 2, max_delay)   # exponential only when guessing

Two details from using this. Add a small margin, half a second or so, because retrying exactly on the boundary is routinely a hair early. And cap what you are willing to honour: a stated wait of an hour is a daily quota wearing a different hat, so above about 60 seconds I stop waiting and classify it as exhaustion instead of blocking the run.

Where the provider sends a header rather than a sentence this is the same idea done properly. Retry-After is standard, takes either seconds or an HTTP date, and the server knows when its own window reopens while you are guessing.

Multi-provider abort conditions

This one only appears once you have more than one provider. My first rule was that if any provider reported exhaustion, abandon the batch. A briefly-throttled primary plus a spend-capped fallback satisfies that, and a run stopped at 4 of 13 jobs while the primary's window was seconds from reopening.

The question the rule is trying to answer is still whether the next item will fail the same way, and it will only if every provider failed for a reason that persists.

any_transient = any(getattr(e, "transient", False) for _cfg, e in failures)
if budget_failures and not any_transient:
    raise LLMBudgetError(summary)   # nothing will serve; stop the batch
raise LLMError(summary)             # this item failed; the next may not

Requiring all failures to be budget errors is wrong in the other direction: a retired model on the primary plus a capped fallback would then keep going item after item with nothing able to answer. Both mistakes are a single word apart, which is the argument for writing the condition as one statement about the list of failures rather than as flags accumulated inside a loop.

A day-one checklist for 429 handling

Three classes rather than two, decided by scope. Read the provider's stated wait before applying your own, cap it, and treat an implausibly long one as exhaustion. Match on specific phrases in order, narrowest first, and never on a word that could turn up in marketing copy — tokens per day is a signal, billing is a coincidence.

Keep the real error strings as you collect them. A handful of saved messages from live failures is worth more than any amount of reasoning about what providers ought to send, and it makes the classifier testable in four lines.

Once more than one job shares an account, the per-batch rules above stop being enough, and what you want is a circuit breaker: after N consecutive failures it opens, everything fails fast for a cool-down, then one probe decides whether to close it again. It is the standard answer to several callers collectively hammering a service that is already struggling, and it is what I would add next.

And log which class you assigned, every time, with the wait and where the number came from. Every mistake above was only visible in hindsight because that line existed.

One limitation: this is calibrated against three providers, and the marker strings are specific to how those three word things. The three classes generalise. The strings do not, and I would expect to add to them after the first surprise on a new provider.

References and further reading

Google's SRE book chapter on handling overload covers the other side of this: why a server sends 429 at all, and why clients retrying aggressively make a struggling service worse rather than better. Its companion chapter on cascading failures is the one to read if your retries are pointed at a service you own.

AWS's timeouts, retries and backoff with jitter is the clearest short explanation of why random jitter belongs in the pause — without it, everything that failed together retries together.

The specifications

RFC 6585 §4 — where 429 is defined. Worth reading for how little it says: too many requests, and explicitly not which limit was hit or for how long.
RFC 9110 — Retry-After — the header that removes the guesswork, in both its seconds and its date form.
RateLimit header fields (IETF draft) — limit, remaining and reset as structured fields. Check whether your provider already sends them before you write a parser for their prose.
Stripe — idempotency keys — the other half of retrying safely: how to retry a request that may already have succeeded without doing it twice.

Implementations you can copy instead of writing

tenacity — retry policies as decorators, including jitter and per-exception rules, which maps cleanly onto the three classes above.
urllib3 Retry — status-aware retries under requests, and it honours Retry-After if you set respect_retry_after_header.
Circuit breaker — the pattern for stopping calls entirely after repeated failures, which is what "exhausted" wants once more than one job shares an account.
Stripe on rate limiters — written from the server side, and the clearest explanation of why different limits exist and therefore why one status code has to cover all of them.
Groq's rate-limit documentation — a concrete example of per-minute and per-day quotas sitting behind the same status, which is the ambiguity the whole post is about.

Local LLM vs API — what a daily cap costs you in practice, and how a fallback chain between providers is put together.
Why your scraper gets blocked — the refusals that look like rate limiting but are not, and what actually causes them.
How to make an ETL pipeline safe to rerun — what a run that was partly refused has to leave behind so the next one can finish the job.