Explainer · rate limiting

Why does an API answer 429?

Almost every rate limit is a token bucket inside. We built one as a machine: a coin magazine, a refill clock and a request belt. Play with it, watch the video, and in five minutes the rule b + r·T will feel obvious.

What does 429 Too Many Requests mean?
How many requests can I send at once?
Why does the average settle at exactly r per second?
What should a client do after a 429?

01The machine

A token bucket you can touch

A magazine of 5 coins is the bucket (capacity b). The hopper clock drops a coin every 2 seconds — the refill (rate r). Every request on the belt takes one coin; no coin, and the kicker pushes it into the 429 tray.

refill

5 / 5tokens in the bucket

0passed · 200

0rejected · 429

The counters come from the same model that moves the parts: a coin drops once per period, an extra one rolls into the overflow cup, a request without a coin gets 429. Try a burst with a full bucket, then with an empty one.

02The video · 2:27

Vera explains, Max asks

A dialogue over the same machine: the bucket, refill, a request, a burst, the average and the two knobs of a limit. There is a vertical cut for Reels/Shorts.

format

Live player ↗

03The gist

Four facts about a token bucket

b · capacity

A burst is at most b

A full bucket lets b requests through in a row, however many arrive at once.

r · rate

The average is at most r

Once the reserve is gone, exactly as many get through as drip in: r requests per second.

overflow

No hoarding

The bucket never grows: extra tokens are lost. A long silence doesn’t earn a giant burst.

429

A rejection is a hint

Got a 429? Wait Retry-After seconds: by then a new token will have dropped.

in any T seconds, at most b + r·T requests get throughIn our machine b = 5, r = 0.5/s: in 10 seconds at most 5 + 0.5·10 = 10 requests.

04Code

The same algorithm in twelve lines

Real servers don’t drop tokens one by one: they recompute the count on every request from the elapsed time. The result is the same as the machine’s.

class TokenBucket {
  constructor(b, r) { this.b = b; this.r = r; this.tokens = b; this.last = Date.now() / 1000; }
  take() {
    const now = Date.now() / 1000;
    // refill: r tokens per second, but never above capacity b
    this.tokens = Math.min(this.b, this.tokens + (now - this.last) * this.r);
    this.last = now;
    if (this.tokens >= 1) { this.tokens -= 1; return { ok: true }; }             // 200
    return { ok: false, retryAfter: Math.ceil((1 - this.tokens) / this.r) };  // 429
  }
}

Built with anatomy (the isometric figure) and video-kit (the narrated video). The machine is figure.js, the model is model.js, the script is video/SCENARIO.md.

Explainer · github.com/nemilya/explainer