ModelRefs / Retry with Exponential Backoff — AI Glossary
Retry with Exponential Backoff — AI Glossary
A resilience pattern retrying failed API calls with exponentially increasing delays and random jitter to avoid thundering herd. Retry is not a circuit breaker.
Overview
Exponential backoff: first retry after 1s, second after 2s, fourth after 4s, capped at 60–120s. Jitter (random ±50% of delay) prevents synchronized retry storms from multiple clients. OpenAI and Anthropic SDKs implement automatic retry (max_retries parameter) with exponential backoff for 429 and 5xx errors.
Reference details
| Topic | ecosystem |
|---|---|
| Also known as | exponential backoff, backoff retry |
| Last reviewed | 2026-06-24 |
Related terms
Example: Budget the whole ladder, not the formula
Base 1s and doubling gives waits of 1, 2, 4, 8 and 16 seconds — 31 seconds of sleeping across five retries. That is the part everyone plans for. The part that surprises: if each attempt itself has a 60-second timeout and all five fail slowly, the caller waits 5 × 60 + 31 ≈ 331 seconds, over five minutes, on one request a user is still watching. Jitter of ±50% shifts each wait but not the shape. The fix is a total deadline on the operation that cancels the ladder, not a smaller number of retries.
Commonly confused with
Retry is not a circuit breaker. Retrying rides out a transient failure; a circuit breaker stops calling a dependency that is consistently failing, so retries do not become the load keeping it down. Systems under sustained failure need both, and the breaker takes precedence.
When to use it
Reach for it when:
- Rate limits and 5xx responses, where the same request will plausibly succeed later
- Any provider call in a batch job, where latency matters less than completion
- Alongside jitter whenever many clients could retry on the same schedule
Reach for something else when:
- Non-idempotent actions with no idempotency key — a retried payment or email may land twice
- 4xx errors other than 429: a malformed request stays malformed
- Interactive paths without a hard deadline, where the ladder outlives the user's patience
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Retry with Exponential Backoff — AI Glossary.
Frequently asked questions
What is Retry with Exponential Backoff?
A resilience pattern retrying failed API calls with exponentially increasing delays and random jitter to avoid thundering herd.
Is Retry with Exponential Backoff the same as exponential backoff?
Yes — exponential backoff, backoff retry are common aliases for Retry with Exponential Backoff.
What concepts are related to Retry with Exponential Backoff?
Closely related concepts include rate limit error, circuit breaker, retry logic.