← ELI5 · Nestor G Pestelos Jr

Software Engineering

What is exponential backoff?

It's a rule for retrying something that failed: wait longer after every failure, instead of trying again right away or waiting the same fixed amount every time.

Wait longer, every time

Four retry attempts, each waiting roughly twice as long as the one before: 1 second, 2 seconds, 4 seconds, 8 seconds 1s try 1 fails 2s try 2 fails 4s try 3 fails 8s try 4 works

Each failed try earns a wait roughly double the last one. A little patience after the first failure; a lot of patience if the failures keep coming.

Why doubling alone isn't enough

Three clients all failed at the same moment, so their doubling wait times line up exactly, and all three retry at the exact same instant, overloading the server again A B C A, B, and C all retry at the exact same instant SERVER hit by A, B, and C at once, again OVERLOADED

If everyone who failed together computes the same wait, everyone retries together — and the pile-up that caused the failure just happens again, a little later.

Why this is called the "thundering herd"

It usually shows up right after an outage: a service goes down, thousands of clients fail at the same instant, and if they all back off on the same schedule, they all come back at the same instant too — knocking the recovering service straight back over.

The fix: jitter

The same three clients now add a random amount to their wait, so their retries land spread out over time instead of all at once, and the server handles them one at a time A B C SERVER hears from A, B, C one at a time STEADY, NOT SPIKY

Add a random wobble to each wait, still growing on average, and the exact same three retries land at three different moments instead of one pile-up.

It doesn't wait forever

Wait times double for the first few attempts, then stop growing once they hit a ceiling, so later retries all wait the same capped amount instead of climbing forever CAP try 1 try 2 try 3 try 4 try 5 try 6

The wait doubles until it hits a ceiling — then every later retry waits that same capped amount, so a run of bad luck never turns into an hours-long silence.

Sources: Marc Brooker, "Exponential Backoff and Jitter," AWS Architecture Blog, 2015. "Retry strategy," Google Cloud Storage documentation. Longer, more technical version: Reference: Exponential Backoff.

Back to top