Rate limiting
Say no cheaply, before the expensive part of your system finds out.
What problem does rate limiting solve?
Any public endpoint will be hit harder than you planned. It might be a buggy client stuck in a retry loop, a scraper, or someone attacking you. With no limit, one caller can use up everything and everyone else is down.
A rate limiter rejects the extra requests at the edge, where rejecting is cheap, instead of letting them reach a database, where they are expensive.
What rate limiting costs you
Every concept in here states a cost. A technique with no price is a technique you have not understood yet — and naming the cost out loud is most of what separates a senior answer from a junior one.
When to reach for rate limiting, and when not to
Use it when
- Every public API. This is a default.
- Expensive endpoints: search, exports, anything calling a paid third party.
- Login and password reset, where the limit is a security control, not a capacity one.
- Protecting something downstream that cannot grow as fast as you can.
Do not, when
- Calls between your own services, where a circuit breaker and a timeout fit better.
- When the real fix is more capacity. A limiter should stop abuse, not paper over a system that cannot serve its real users.
How rate limiting actually works
Token bucket is the default, and it is worth being able to describe in one sentence. Tokens are added at a fixed rate up to a maximum. Each request takes one. A request that finds no token is rejected. The bucket size is how big a burst you allow. The refill rate is the steady limit. That burst allowance is why it suits real traffic, which arrives in clumps.
Fixed window means counting per minute and resetting on the minute. It is the easiest to build and it has an obvious flaw. A caller can spend the whole allowance at 10:59:59 and the whole allowance again at 11:00:00, so a 100-per-minute limit allows 200 in one second. A sliding window log keeps every request timestamp and is exact, at the cost of memory that grows with traffic. A sliding window counter blends the current and previous windows in proportion. It is nearly as accurate for a fraction of the memory, which makes it a common production choice.
Where to put it: at the edge or the gateway, so rejected traffic never reaches your services. Per-server counters are simple, and they let a spread-out caller get N times the limit. A shared counter in Redis is accurate, and adds a hop plus a dependency to every request. That means you need a decision: if Redis is down, do you reject everything or allow everything? Usually allow, because a limiter outage should not become a site outage. Say that out loud. It is a deliberate choice, and it has a security cost.
What you limit by matters more than the algorithm. By API key or user id is right for logged-in traffic. By IP is the fallback for anonymous traffic, and it punishes shared networks. Layered limits work well: one per user, one per IP, and a global ceiling that protects the system whatever happens.
Be a good citizen when you reject. Return 429, include a Retry-After header, and document the limits. A client that knows when to come back stops hammering you. One that gets a bare error retries immediately and makes it worse.
- Limiting by IP only, then blocking an entire office.
- A shared counter with no answer for what happens when it is down.
- Returning a bare 429 with no Retry-After, which teaches clients to retry immediately.
The rate limiting follow-up an interviewer will ask
"A legitimate customer hits the limit during their busiest hour. What do you do?"
Short term, raise their limit. Which means limits have to be per-key configuration, not a constant in the code, and that is a decision to make on day one. Then work out whether it was abuse, or whether my limit is simply wrong for how the product is used. If several customers hit it during a normal peak, the limit is the bug. Longer term I would rather shape traffic than reject it: queue the extra and process it slightly slower, or return a partial result, because a slow answer beats an error for most APIs. I would also split the limits by cost. A cheap read endpoint and an expensive export should not share a budget, or one burst of exports locks the customer out of everything. And I would make the 429 useful: Retry-After, plus headers showing how much quota is left, so a well-written client manages itself instead of finding out by failing.
Your turn
Your limit is 100 requests per minute using a fixed window. How can a caller legitimately send 200 requests in one second, and which algorithm fixes it?
Learn it from the source
This page gets you interview-ready. These take you to where the idea is actually specified or measured — read them when you want the real depth.
- RFC 6585 — Additional HTTP Status Codes ↗Where 429 Too Many Requests is defined, along with Retry-After.