Bound retry delays
Calls to a model or a tool sometimes fail for a moment: the service is busy, or the network drops. Trying again often works. But trying again at once, and for ever, makes a busy service busier. Exponential backoff is the polite way to retry: wait a little, then twice as long, then twice as long again. Two limits keep it sane. A cap stops the wait from growing without end, and a retry budget says when to give up.
About this lesson
Words to know
- retry
- Trying a failed call again, in the hope that the failure was brief.
- exponential backoff
- Waiting longer before each retry, usually twice as long as the time before.
- retry budget
- The most retries allowed before the caller gives up and reports the failure.
Why it matters
Every call to a model service needs this, and client libraries often do it for you. Real retry code adds a small random change to each wait, called jitter, and takes care not to repeat a write that succeeded.
With a year
How it works, a worked example, the exercise with its tests, three hints and the explained solution are in the app, with a year of Algo150 for $19. The first chapter, Model foundations, is free for everyone: start with Vector dot product.