Fibonacci lockouts: rate limiting a scraper that keeps coming back

Talent search at WhiteCrow ran over a 4 TB JSON corpus in Elasticsearch. A single query could fan out across a lot of that, which makes the interesting property of the platform not throughput but blast radius: one caller issuing expensive searches in a loop degrades search for every other client on the cluster at the same time.

So the endpoint needed a limiter. The part worth writing about is not the counter — everyone has written the token bucket post — it is what happens on the second offence, and the third.

The threat model is not malice

The instinct is to picture a scraper. Most of the time it is an integration.

A client wires our API into their own tooling, the retry logic is a while loop with no backoff, someone leaves it running over a weekend, and by Monday they have issued more searches than the rest of the platform combined. They are not attacking anything. They have a bug, and we are the ones who feel it.

The actual scraper — somebody paying for one seat and pulling the corpus through it — looks identical for the first thirty seconds. Same shape, same burst, same endpoint. What separates them is not the first violation. It is what they do after being told no.

That is the whole design. The limiter's job is not to classify the caller up front. It is to make the two cases diverge over time, cheaply, without a human reading logs on a Saturday.

The budget is per client, not per IP

An IP is the wrong unit of account here.

Clients are companies. Their traffic arrives from an office range, a VPN, a CI runner, and four laptops in three countries, and all of it is legitimately one account. Meanwhile a determined scraper rotates addresses for the price of a coffee. Limiting by IP punishes the organisation that has a proxy and barely inconveniences the one you actually care about.

So the budget attaches to the authenticated client, and it is set per client rather than globally — what they are entitled to is a commercial question, and different contracts get different numbers. A limiter that cannot express "this customer paid for more" gets overridden by hand within a month, and a limiter that gets overridden by hand is not a limiter.

Why not exponential

Once you are escalating, the obvious reach is for doubling: 1, 2, 4, 8, 16, 32. It is the default because backoff is the famous use of it, and it fits there for a reason — an overloaded server wants callers gone fast, and if a client punishes itself too hard the only victim is that client.

Lockout is the mirror image of that situation, and doubling is wrong in both directions.

Early, it is too harsh. The buggy integration hits the limit four or five times in quick succession before anyone notices, because that is what a retry loop does. Under doubling that costs them most of an hour, on a mistake that deserves minutes. Now you have taken a paying customer's Tuesday away for a missing sleep.

Late, it arrives too slowly and then overshoots. Doubling is patient for the first few offences — exactly when the genuine scraper is learning the shape of your limits — and then jumps to a wall so tall it is indistinguishable from a ban. The honest user who trips it on day two gets locked out for half a day, support gets an angry email, and someone lifts the block manually. Which resets the attacker too.

Linear escalation has the opposite failure: it is forgiving forever. Add a minute per offence and a scraper simply schedules around it and keeps pulling.

The curve that sits between them

Fibonacci grows at roughly φ each step — about 1.6× rather than 2× — and the early terms are close together:

// Offence n inside the window earns the nth lockout in the sequence.
const LOCKOUT_MINUTES = [1, 2, 3, 5, 8, 13, 21, 34, 55];

function lockoutFor(offence: number): number {
  const i = Math.min(offence, LOCKOUT_MINUTES.length) - 1;
  return LOCKOUT_MINUTES[i];
}

Read that sequence as a conversation rather than a punishment schedule.

The first three offences cost one, two, and three minutes. That is a limiter clearing its throat — enough that a well-behaved client's retry logic notices and backs off, cheap enough that the cost of being wrong about them is nearly zero. Nobody files a ticket over three minutes.

By the sixth offence you are at thirteen minutes, and by the ninth at fifty-five. That is genuinely expensive for anybody trying to enumerate a corpus — it drags a scrape that would have taken hours out into days, which is usually enough for the economics to stop working — but it got there gradually, and every step along the way was a chance for a legitimate caller to correct itself without losing its afternoon.

The gap between the two curves is small in the abstract and decisive in practice, because almost all of your real traffic lives in the first four offences. That is precisely where Fibonacci is gentle and doubling is not.

What the 24-hour window is actually for

The escalation only means anything paired with a decay. Offences count against you within a rolling day; fall quiet and the counter comes back down.

This is the part that does the classification, and it does it without any heuristics. A client that fixes its retry loop stops offending, ages out of the sequence, and never sees a long lockout again — the system forgives, on its own, without a support ticket. A client that is systematically pulling the corpus cannot age out, because to stay under the decay it would have to slow down to roughly the rate you wanted in the first place.

Either outcome is a win, and neither one required anybody to decide which sort of caller this was. That is the property I would keep if I rebuilt it.

What I would change now

Three things.

The lockout should say when it ends. A Retry-After header and a plain error body turn a mystery outage into something a client's engineer can act on in ten seconds. Silence here generates far more support load than the limiter itself ever saves.

Weight the budget by cost, not by request count. Searches are not interchangeable — a narrow filtered query and a broad one across the corpus are different amounts of cluster, and counting both as "one" lets the expensive pattern hide inside a modest-looking request rate.

Tell the account manager before the customer notices. By the time a client reaches the fifth lockout, somebody on our side should already know. Nearly every case that escalated that far was a relationship problem wearing a technical costume, and it was always cheaper to solve with a phone call than with a sequence.