Rate Limits & Quotas

Why limit at all

A shared API has finite capacity: database connections, worker threads, a write path that one process owns. Without limits, the first caller to write a tight loop takes whatever it wants, and every other caller waits behind it. A limit does four jobs:

• Protects capacity. The limit is a ceiling you chose, rather than the ceiling the database picked when it fell over.
• Shares it fairly. One tenant's bug does not become every tenant's outage.
• Caps cost. Each request costs CPU, storage and sometimes a paid upstream call. A limit puts a bound on the bill.
• Blunts abuse and runaway clients. A scraper, a credential-stuffing run, or an importer stuck in a retry loop all hit the same wall.

Snip's two planes need different treatment. The redirect (snip.ly/abc123) is the data plane: it is read-heavy, cached at the edge, and built to absorb spikes, so the per-key limit described here does not apply to it the same way. The API at https://api.snip.ly/v1 is the control plane: it creates links, edits them, and reads analytics, and each call touches the primary database. That is what we limit.

Rate versus quota

The two words get mixed up, and they answer different questions.

• A rate limit bounds requests per short unit of time: 100 per minute. It protects the system now. It resets quickly, so a client that is told to wait can usually be back in seconds.
• A quota is a budget over a long window: 50,000 requests per month, or 10,000 links created per day. It protects the business: cost, plan boundaries, fair use. It resets on a calendar boundary, so a client that hits it may be locked out for days.

Snip today has a per-minute rate by plan. A monthly quota is a natural addition for the free tier, and the two stack: a request must pass both. Return them differently. A rate-limit rejection says "retry in 30 seconds". A quota rejection says "upgrade, or wait until the first of the month", and a client should not retry it on a timer. Use a distinct problem type for each so a client can tell them apart without parsing prose (the body is covered in Errors & Validation).

Four algorithms

All four answer one question: given the requests this caller has made so far, may this one proceed? They differ in what state they keep and how they treat bursts.

Fixed window

Count requests per calendar window, for example per clock minute, and reject once the count passes 100. State is one counter and an expiry. It is the cheapest to build and the easiest to explain. Its flaw is the boundary: a client sends 100 requests at 12:00:59 and 100 more at 12:01:00. Each window is within its limit, yet the API took 200 requests in two seconds, twice the rate you meant to allow.

Sliding window

Count requests in the last 60 seconds from now, not from the top of the minute. Two ways to do it. A log keeps a timestamp per request and counts those younger than 60 seconds: exact, but memory grows with the limit. A weighted counter keeps two fixed-window counts and blends them by how far into the current window you are: for example, 30% of the way through, count 70% of the previous window plus all of the current one. It uses two numbers per caller and is approximate, but the boundary burst mostly disappears.

Token bucket

A bucket holds up to N tokens and refills at a steady rate. Each request spends one token; with none left, the request is rejected. The next section covers it in full, because it is the one to pick.

Leaky bucket

Requests join a queue and drain at a constant rate, so the output is smooth whatever the input looks like. This is traffic shaping, not rejection: it fits when you must feed a downstream that cannot take bursts, and it is a poor fit for an interactive API, where a queued request is a slow request.

The token bucket

Use a token bucket unless you have a reason not to. Real clients are bursty: a dashboard loads and fires eight calls at once, then goes quiet. A strict "one request every 600 ms" rule punishes that harmless pattern. A bucket accepts it, and still enforces the average.

Two parameters set the behaviour:

• Capacity is the most tokens the bucket holds. It sets the largest burst a caller can send after a quiet spell.
• Refill rate is how fast tokens return. It sets the long-run average, whatever the burst.

You do not need a timer to refill the bucket. Store the token count and the time of the last update, and compute the refill when a request arrives:

on request(key):
    bucket = load(key)                      # tokens, updated_at
    elapsed = now() - bucket.updated_at
    tokens  = min(capacity, bucket.tokens + elapsed * refill_per_second)

    if tokens >= 1:
        save(key, tokens - 1, now())
        allow                               # RateLimit-Remaining = floor(tokens - 1)
    else:
        save(key, tokens, now())
        reject                              # Retry-After = (1 - tokens) / refill_per_second

For Snip's free tier, set capacity to 100 and the refill to 100 per 60 seconds, about 1.67 tokens a second. A new key starts with a full bucket. A caller that sends 100 calls in one second gets all 100, then is held to one every 600 ms. A caller that sends one call a second never notices the limit. Pro is the same shape with capacity and refill at 1,000 per minute.

One caution with bucket-based headers. RateLimit-Remaining is the whole tokens left, and RateLimit-Reset is the seconds until the bucket would be full again, or until a token returns if it is empty. Pick one meaning, document it, and keep to it on every response.

Who you limit

Limit by identity, not by address. Behind one office NAT or one mobile carrier gateway sit thousands of unrelated users, and a limit keyed on the IP punishes all of them for one. In the other direction, a hostile caller rotates through addresses at will. The identity you can trust is the one you authenticated in Auth & Scopes:

1. API key (sk_live_…) for server-to-server callers. One key, one bucket.
2. OAuth client for an integration used by many people. Give the client its own bucket so one popular integration cannot starve the rest, and consider a smaller per-user bucket inside it.
3. IP address only for unauthenticated routes, such as sign-up and token exchange, where no better identity exists. Keep that limit loose and treat it as a backstop.

Set limits by plan. Snip's free tier gets 100 requests a minute and pro gets 1,000. The plan is a property of the key, so the limiter reads one record and picks the bucket parameters. Keep the tier table as configuration, not code, so support can raise one customer's limit in minutes without a deploy. Give write endpoints a tighter bucket than reads when the cost differs: creating a link writes to the primary, reading one usually hits a cache.

The response contract

A limit that clients cannot see is a trap. Three pieces make it a contract.

Status. Over the limit, answer 429 Too Many Requests (RFC 6585). Do not use 403 or 503: a 403 says "you may not", and a 503 says "we are in trouble". A 429 says "you, specifically, are going too fast", and clients and proxies know what to do with it.

Retry-After. Tell the client when to come back. The header carries either a number of seconds or an HTTP date (RFC 9110). Seconds are simpler and not subject to clock skew, so prefer them.

The RateLimit headers, on every response. Send them on successful responses too, not only on the 429, so a client can slow down before it is refused:

A normal response from a free-tier key that is running hot:

HTTP/1.1 200 OK
RateLimit-Limit: 100
RateLimit-Remaining: 3
RateLimit-Reset: 22
Content-Type: application/json

And the rejection, a few calls later, with a problem body (the shape is defined in Errors & Validation):

HTTP/1.1 429 Too Many Requests
Retry-After: 30
RateLimit-Limit: 100
RateLimit-Remaining: 0
RateLimit-Reset: 30
Content-Type: application/problem+json

{
  "type": "https://api.snip.ly/errors/rate-limited",
  "title": "Rate limit exceeded",
  "status": 429,
  "detail": "The free plan allows 100 requests per minute. Retry in 30 seconds.",
  "instance": "/v1/links"
}

The body is for the human reading a log. The headers are for the program. Never make a client parse detail to learn how long to wait.

What a good client does

The headers only help if the client reads them. A good Snip client does three things, in this order.

1. Stay under the limit. Read RateLimit-Remaining on every response. When it nears zero, wait out RateLimit-Reset before the next call. Do not wait for the 429.
2. On a 429, obey Retry-After. It is the server's own number, and it beats any guess.
3. If there is no Retry-After, or a 5xx arrives, back off exponentially with jitter. Wait a random time up to min(cap, base * 2^attempt), then try again, and give up after a bounded number of attempts.

Jitter is not optional. If 500 clients all hit the limit at the same second and all sleep exactly 30 seconds, all 500 return at the same second, and the API is flattened again. That is a thundering herd, and it is the retry storm that turns a short overload into a long outage. Randomising the wait spreads the return over time.

Take the batch importer. A customer loads 40,000 links from a CSV through POST /v1/links on a free key. A naive loop fires as fast as it can, burns the 100 tokens in a second, collects 429s, and retries immediately, which just consumes the tokens as they refill. A well-behaved importer paces itself:

#!/usr/bin/env bash
# Import links; sleep for RateLimit-Reset when the bucket is nearly empty.
while IFS=, read -r url slug; do
  headers=$(curl -s -D - -o /dev/null -X POST https://api.snip.ly/v1/links \
    -H "Authorization: Bearer $SNIP_KEY" \
    -H "Idempotency-Key: $(uuidgen)" \
    -H "Content-Type: application/json" \
    -d "{\"url\":\"$url\",\"slug\":\"$slug\"}")
  remaining=$(echo "$headers" | awk -F': ' 'tolower($1)=="ratelimit-remaining"{print $2+0}')
  reset=$(echo "$headers"     | awk -F': ' 'tolower($1)=="ratelimit-reset"{print $2+0}')
  if [ "${remaining:-1}" -le 2 ]; then sleep "${reset:-1}"; fi
done < links.csv

At 100 a minute, 40,000 links take about 400 minutes. That is the honest arithmetic, and it is a good moment to point the customer at pro (1,000 a minute) or at a bulk endpoint, which spends one token for many links. A retry must not create a link twice: every write above carries an Idempotency-Key, covered with retries in Caching, Concurrency & Idempotency.

Where to enforce

Enforce as early as you can, before the request does any work. A limiter inside the handler has already spent the CPU, the connection and the auth lookup on a request it was going to refuse. Put it at the edge or the API gateway, just after authentication has identified the caller, and reject there. A rejected request then costs a counter lookup, not a database query.

The catch is that you run many gateway instances, and the caller's requests are spread across them. A counter held in each process's memory lets a caller send 100 requests to each of 10 instances. The counters need a shared home, and the usual choice is a fast in-memory store such as Redis. Each check is one round trip: read the bucket, compute, write it back. Do that as a single atomic script on the store, or two requests that race will both read "one token left" and both pass.

Shared state brings the distributed-counter problem. An exact global count needs every check to coordinate, and the coordinator becomes a bottleneck and a new single point of failure. Most APIs should accept approximate counters instead:

• Let each instance keep a local bucket and sync with the store every few hundred milliseconds. The limit may be exceeded by a small, bounded amount.
• Shard keys across the store so one hot customer does not pin one node.
• Decide what happens when the store is down. For most APIs, fail open (allow the request, log it, alert), because refusing all traffic over a limiter fault is worse than briefly having no limit. For a sensitive endpoint such as login, fail closed.

An error of 2% on a limit of 100 a minute hurts nobody. A limiter that adds 40 ms to every call, or takes the API down with it, hurts everybody. The trade is the same one as for any shared cache, discussed in Caching in the scale guide: you buy speed with a small, bounded amount of staleness.

Limits versus load shedding

These look alike, since both refuse requests, but they answer different questions.

• A rate limit is policy about one caller. It fires when that caller exceeds its share, however healthy the system is. The response is 429, and the caller can fix it by slowing down.
• Load shedding is a reliability mechanism about the system. It fires when the whole service is near its limit, whoever is calling: queues are deep, latency is climbing, CPU is saturated. The response is 503, with Retry-After, and no single caller is at fault.

You need both. Per-caller limits sum to more than your capacity, by design, because not every caller is at its limit at once. If 300 free keys and 40 pro keys all run flat out, the sum is over 70,000 requests a minute, whatever the server can take. Limits keep any one caller from starving the others, but they do not promise that the total fits. Shedding is the backstop for that case: drop the least important work first (analytics reads before link creation), and keep the system answering. The mechanics, and how to choose what to drop, are in Reliability & Degradation.

Keep the two signals distinct on the wire. A client that gets a 429 should slow itself down. A client that gets a 503 should back off and may try another region. If you answer both with the same status, you lose the information that tells a client which one to do.

✓ Check yourself

Work through these before the next chapter. Answer in your own words first, then open the answer.

1. A fixed-window limiter allows 100 requests per clock minute. Describe a request pattern that gets 200 requests through in about two seconds, and name a limiter that would stop it.

   Show answer

   Send 100 requests in the last second of one minute and 100 in the first second of the next. Each window counts 100, so both pass. A sliding window (or a token bucket with capacity 100) sees the 200 requests inside one rolling minute and refuses the second batch. See Four algorithms.

2. You set a token bucket to capacity 100, refill 100 per minute. A key has been idle for an hour, then sends 150 requests in one second. How many succeed, and what does the caller see for the rest?

   Show answer

   About 100 succeed, as the bucket is full after an hour, plus one or two for tokens that refill during the second. The rest get 429 with a Retry-After of the time until one token returns, about one second here. See The token bucket.

3. Why key the limit on the API key or OAuth client rather than the IP address? Name one case where the IP is still the right key.

   Show answer

   Many users share an IP (NAT, carrier gateways), so IP limits punish the innocent, and an attacker can rotate IPs, so they miss the guilty. A key is authenticated and tied to a plan. The IP is still right on unauthenticated routes such as sign-up or token exchange, where no identity exists yet. See Who you limit.

4. Your 429 response has a clear body but no headers. What does the client lose, and which three headers would you add to every response?

   Show answer

   The client cannot learn how long to wait without parsing prose, and it cannot slow down before it is refused. Add Retry-After to the 429, and RateLimit-Limit, RateLimit-Remaining and RateLimit-Reset to every response. See The response contract.

5. 500 clients hit a 429 at the same moment and each sleeps exactly Retry-After seconds. What happens when they wake, and what change fixes it?

   Show answer

   They all return at the same instant and exhaust the limit again, which repeats on each cycle. Add random jitter to the wait, and use exponential backoff with a cap and a maximum attempt count when no Retry-After is given. See What a good client does.

6. Every key is within its limit, yet the API is slow and queues are growing. Which mechanism applies, what status does it return, and why did the limits not prevent this?

   Show answer

   Load shedding, returning 503 with Retry-After. Per-caller limits sum to more than total capacity, so many callers each within their share can still overload the system. See Limits versus load shedding.

Takeaway

• Limit to protect capacity, share it fairly, cap cost and contain runaway clients. A rate bounds requests per minute; a quota bounds a budget per month, and they fail differently.
• Default to a token bucket: capacity sets the burst, refill sets the average. Fixed windows allow a 2x spike at the boundary.
• Key limits on the authenticated API key or OAuth client and set them by plan. Use the IP only where no identity exists.
• Return RateLimit-Limit, RateLimit-Remaining and RateLimit-Reset on every response, and 429 with Retry-After and a problem body when over.
• Clients read the headers, obey Retry-After, and back off with jitter. Without jitter, a retry herd undoes the limit.
• Enforce at the edge with a shared counter, accept approximate counts, and add load shedding (503) for when the system, not a caller, is the problem.

Next, the other half of speed and safety: keeping reads cheap and making retries harmless. Caching, Concurrency & Idempotency →

Further reading

• RateLimit header fields for HTTP (IETF draft) — the standardisation effort behind the RateLimit-* headers, including the newer RateLimit-Policy form.
• RFC 6585: Additional HTTP Status Codes — defines 429 Too Many Requests.
• RFC 9110, Retry-After — the header's two forms, seconds and HTTP date.
• RFC 9457: Problem Details for HTTP APIs — the body shape for the 429 response.
• Stripe: Scaling your API with rate limiters — a production account of request limiters and load shedders working together.
• GitHub REST API rate limits — a public example of per-identity limits, headers and retry guidance.
• AWS Builders' Library: Timeouts, retries and backoff with jitter — why jitter matters and how to apply it.