API rate limiting: algorithms, headers and code

API rate limiting caps how many requests each client can make in a given time and answers the excess with 429 Too Many Requests plus a Retry-After header. For an API you charge for, limit each customer by the API key they authenticate with, and start with a token bucket: it allows short bursts, needs two numbers per customer, and its two settings, burst size and refill rate, read naturally in a plan.

Why rate limit, and per what

An API rate limit protects you from any single client: a retry loop with no backoff, a cron job that fires 10,000 requests at midnight, someone guessing keys. For a paid API it also keeps one heavy customer from slowing down the rest, and it adds a second dimension to your plans, as in "100,000 requests a month, at most 10 per second".

Rate limits and quotas are different tools. A rate limit works in seconds or minutes and recovers on its own. A quota covers a billing period, and going past it is a billing decision, covered in usage-based billing.

What you count against matters as much as the algorithm:

Per-IP limits belong at the edge (CDN, WAF or load balancer), where they stop floods before your code runs. Per-customer limits belong in your app after authentication, the first point that knows who owns a key. Behind a proxy, configure Express's trust proxy with care: with true, Express trusts the left-most X-Forwarded-For entry, which clients can forge unless your last proxy overwrites the header.

Rate limiting algorithms

A fixed window counts requests per calendar window, say 12:00:00 to 12:00:59, with one counter per client (INCR plus EXPIRE in Redis). With a limit of 100 per minute, a client can send 100 requests at 12:00:59 and 100 more at 12:01:00: 200 within two seconds.

A sliding window log stores a timestamp per request and counts those from the last 60 seconds. It's exact, but memory grows with the limit, so keep it for small limits like 5 login attempts per 15 minutes.

A sliding window counter keeps this window's count and the previous one's, and weights the previous count by how much of it still overlaps. With 80 requests last minute and 30 so far, 15 seconds into this minute, the estimate is 30 + 80 × 45/60 = 90. It assumes last minute's traffic was evenly spread, which is close enough for most per-minute limits.

A token bucket gives each client a bucket of tokens that refills at a fixed rate, and each request takes one. The bucket size sets the largest burst; the refill rate sets the sustained rate.

A leaky bucket queues requests and releases them at a constant rate, for backends that can't take bursts. nginx's limit_req works this way: excess requests are delayed until they exceed the burst size, then rejected with a 503 unless you set limit_req_status 429.

Token bucket math, worked through

You don't need a timer. Refill lazily when a request arrives:

tokens = Math.min(capacity, tokens + secondsSinceLastRequest * refillPerSecond);

If tokens is at least 1, allow the request and subtract 1. Otherwise reject it and tell the client to wait ceil((1 - tokens) / refillPerSecond) seconds. Round up, because a client that retries early just gets another 429.

Take a plan of 1 request per second with bursts of 10, so capacity is 10 and refillPerSecond is 1:

1. The customer has been idle, so the bucket is full. They send 10 requests at once and all succeed. The bucket is empty.
2. 0.4 seconds later the bucket holds 0.4 tokens. Request 11 gets a 429 with Retry-After: 1: the missing 0.6 tokens take 0.6 seconds, rounded up.
3. At 1.0 seconds there's a whole token, and the next request goes through.
4. After that the customer gets 1 request per second, or 60 a minute.

Note what a full bucket does to a one-minute window: 10 at once plus 60 refilled is up to about 70 requests in the first minute after a quiet spell. A bucket of 60 on the same plan allows up to about 120. If you sell "60 per minute", size the bucket for the burst you can absorb, not for the whole minute.

What to send with a 429

RFC 6585 defines 429 as "the user has sent too many requests in a given amount of time", says the response may include Retry-After, and forbids caches from storing it.

Retry-After (RFC 9110) takes an HTTP date or a number of seconds. Send seconds; they don't depend on the client's clock.

RateLimit-Policy and RateLimit come from an IETF working group draft, draft-ietf-httpapi-ratelimit-headers. Revision -11, published May 23, 2026, is the latest, and it's still an Internet-Draft, not an RFC. It defines two fields:

• RateLimit-Policy: "default";q=10;w=10 describes a policy with a quota of q units per window of w seconds.
• RateLimit: "default";r=0;t=1 reports r units left, and t, the seconds within which the client can use no more than r.

The names have changed along the way. Up to -06 (December 2022) the draft used separate RateLimit-Limit, RateLimit-Remaining and RateLimit-Reset fields, -07 merged them into one RateLimit field, and -08 (October 2024) brought today's syntax, so you'll still see all three styles in the wild. When both are present, the draft says clients must give Retry-After precedence, and Retry-After shouldn't point earlier than the end of the t window.

X-RateLimit-* is the older, unstandardized convention. GitHub sends x-ratelimit-limit, x-ratelimit-remaining, x-ratelimit-used and x-ratelimit-reset, with reset as a UTC epoch timestamp. Elsewhere reset can be seconds to wait, milliseconds or a date, so document what yours means.

The example code below returns this 429 (headers trimmed) for a bucket of 10 that refills 1 per second:

HTTP/1.1 429 Too Many Requests
RateLimit-Policy: "default";q=10;w=10
RateLimit: "default";r=0;t=1
Retry-After: 1
Content-Type: application/json; charset=utf-8

{"error":"rate_limited","retry_after":1}

A token bucket doesn't map exactly onto "q per w seconds", so the example sends the bucket size as q and the time to refill it from empty as w. Clients that space requests evenly at q / w per second or slower won't run out. If existing clients read X-RateLimit-*, keep sending those too.

Per-customer limits keyed by API key

When you rate limit an API per customer, middleware order decides what you're limiting:

1. Check the API key. Unknown keys get a 401 and never reach the limiter, so junk keys can't fill your bucket map. Add a per-IP limit on failed checks to slow down key guessing.
2. Resolve the key to its customer and key the bucket on the customer ID.
3. Read the limit from the customer's plan, with per-customer overrides for accounts that need more.

Expensive endpoints can cost more than one token: if image generation costs ten times what a lookup does, charge it 10 tokens.

Don't bill rejected requests, but do record them: a customer who hits the limit every afternoon is a candidate for a bigger plan. API analytics covers which counts to keep. Customers should also see their limits next to their keys, so a 429 doesn't come as a surprise, which is one job of a developer portal.

Rate limiting across multiple instances

An in-memory bucket counts what one process sees. With three instances behind a load balancer, you have three options:

Counting per instance is less sloppy than it sounds. GitHub's documentation says that because it handles requests in multiple regions, x-ratelimit-remaining can go up between responses in the same window. Most limits exist to stop runaway clients, and a bit over the limit still does that.

Dividing each limit by the number of running instances (rounded up, never below 1) gets close to the real limit and keeps the check local. It drifts when keep-alive pins a client to one instance behind a load balancer that balances connections, not requests.

A shared store is the only way to get an exact limit. Run the token bucket as a Lua script: Redis runs scripts atomically, so two instances can't both spend the last token. Read the time inside the script with Redis's TIME command, so clock drift between app servers doesn't matter.

Our advice: start per instance, and move to a shared store when limits are tight compared with normal traffic or autoscaling swings your instance count. UsageRail's SDK, which we're building now, takes the first two routes: a token bucket per customer inside your process, with per-second and per-minute limits, counted per instance by default or divided by the instance count if you turn that on. Rejections are 429s with Retry-After and rate limit headers, and they aren't billed.

Testing a rate limiter

• Inject the clock. Tests that sleep are slow and flaky, and a fake clock can jump half a second or an hour instantly.
• Test the edges: exactly capacity requests pass, the next fails, half a token isn't enough, and a long pause refills the bucket to capacity and no further. Assert exact header values.
• Check isolation: one customer at zero doesn't affect another, and unknown keys get a 401 without creating a bucket.
• Mind concurrency. A synchronous check in Node can't interleave within one process, but FastAPI runs plain def dependencies in a thread pool, so use async def or a lock.
• Before launch, load test with your real instance count to see the limit clients actually get.

Express code: token bucket middleware

Here's the limiter and middleware in plain JavaScript (ESM, Node 22 or later). Every response gets RateLimit-Policy and RateLimit, and each 429 also gets Retry-After.

// Per-customer token bucket rate limiting as Express middleware.
// Each customer gets a bucket that holds up to `capacity` tokens and refills
// at `refillPerSecond`. A request takes one token; an empty bucket means 429.

export function createTokenBucket({ capacity, refillPerSecond, now }) {
  // One bucket per customer. Requests are authenticated before they get here,
  // so this map grows with your customer count, not with made-up keys.
  const buckets = new Map();

  return function take(customerId) {
    const currentTime = now();
    const bucket = buckets.get(customerId) ?? { tokens: capacity, updatedAt: currentTime };
    // Lazy refill: add the tokens earned since the last request, up to capacity.
    const elapsedSeconds = currentTime - bucket.updatedAt;
    bucket.tokens = Math.min(capacity, bucket.tokens + elapsedSeconds * refillPerSecond);
    bucket.updatedAt = currentTime;
    buckets.set(customerId, bucket);

    if (bucket.tokens < 1) {
      // Rounded up, because retrying a moment too early just earns another 429.
      const retryAfterSeconds = Math.ceil((1 - bucket.tokens) / refillPerSecond);
      return { allowed: false, retryAfterSeconds };
    }
    bucket.tokens -= 1;
    return {
      allowed: true,
      remaining: Math.floor(bucket.tokens),
      secondsUntilFull: Math.ceil((capacity - bucket.tokens) / refillPerSecond),
    };
  };
}

// The default clock is monotonic, so a wall-clock jump (NTP, a VM resume)
// can't refill or drain every bucket at once.
export function rateLimit({ capacity, refillPerSecond, now = () => performance.now() / 1000 }) {
  const take = createTokenBucket({ capacity, refillPerSecond, now });
  // Header syntax from the IETF draft draft-ietf-httpapi-ratelimit-headers-11.
  const policy = `"default";q=${capacity};w=${Math.ceil(capacity / refillPerSecond)}`;

  return function rateLimitMiddleware(request, response, next) {
    // request.customer is set by the authentication middleware, which runs first.
    const result = take(request.customer.id);
    response.set('RateLimit-Policy', policy);

    if (result.allowed) {
      response.set('RateLimit', `"default";r=${result.remaining};t=${result.secondsUntilFull}`);
      return next();
    }
    // On a 429, t equals Retry-After: the draft says Retry-After shouldn't point
    // to a time earlier than the end of the window that t describes.
    response.set('RateLimit', `"default";r=0;t=${result.retryAfterSeconds}`);
    response.set('Retry-After', String(result.retryAfterSeconds));
    response.status(429).json({ error: 'rate_limited', retry_after: result.retryAfterSeconds });
  };
}

The app checks the key first, then mounts the limiter:

import express from 'express';
import { rateLimit } from './rate-limit.js';

// apiKeys maps a raw key to a customer ID. Fine for a demo; in production,
// store a hash of each key and look up the hash.
export function createApp({ apiKeys, capacity, refillPerSecond, now }) {
  const app = express();

  // Authenticate before rate limiting, so the limiter can key on the customer.
  app.use(function authenticate(request, response, next) {
    const apiKey = request.get('authorization')?.replace(/^Bearer /, '') ?? request.get('x-api-key');
    const customerId = apiKeys.get(apiKey);
    if (!customerId) {
      return response.status(401).json({ error: 'invalid_api_key' });
    }
    request.customer = { id: customerId };
    next();
  });
  app.use(rateLimit({ capacity, refillPerSecond, now }));

  app.get('/v1/hello', (request, response) => {
    response.json({ customer: request.customer.id });
  });

  return app;
}

The tests start the app on a free port with a fake clock and cover the 429 and its headers, refills, per-customer buckets and the 401:

cd examples/api-rate-limiting/express
node --test

Prefer a library? express-rate-limit counts per IP by default, in a window that starts at each client's first request, and keeps counts in process memory unless you give it a store such as Redis. Pass a keyGenerator that returns your customer ID.

FastAPI code: token bucket dependency

Here's the same limiter as a FastAPI dependency (Python 3.12):

"""Per-customer token bucket rate limiting as a FastAPI dependency.

Each customer gets a bucket that holds up to `capacity` tokens and refills at
`refill_per_second`. A request takes one token; an empty bucket means 429.
"""

import math
import time
from collections.abc import Awaitable, Callable
from dataclasses import dataclass
from typing import Annotated

from fastapi import Depends, HTTPException, Response


@dataclass
class Bucket:
    tokens: float
    updated_at: float


@dataclass
class Decision:
    allowed: bool
    remaining: int = 0
    seconds_until_full: int = 0
    retry_after_seconds: int = 0


class TokenBucket:
    # The default clock is monotonic, so a wall-clock jump (NTP, a VM resume)
    # can't refill or drain every bucket at once.
    def __init__(self, capacity: int, refill_per_second: float, now: Callable[[], float] = time.monotonic):
        self.capacity = capacity
        self.refill_per_second = refill_per_second
        self.now = now
        # One bucket per customer. Requests are authenticated before they get
        # here, so this grows with your customer count, not with made-up keys.
        self.buckets: dict[str, Bucket] = {}

    def take(self, customer_id: str) -> Decision:
        current_time = self.now()
        bucket = self.buckets.setdefault(customer_id, Bucket(self.capacity, current_time))
        # Lazy refill: add the tokens earned since the last request, up to capacity.
        elapsed_seconds = current_time - bucket.updated_at
        bucket.tokens = min(self.capacity, bucket.tokens + elapsed_seconds * self.refill_per_second)
        bucket.updated_at = current_time

        if bucket.tokens < 1:
            # Rounded up, because retrying a moment too early just earns another 429.
            wait_seconds = math.ceil((1 - bucket.tokens) / self.refill_per_second)
            return Decision(allowed=False, retry_after_seconds=wait_seconds)
        bucket.tokens -= 1
        return Decision(
            allowed=True,
            remaining=math.floor(bucket.tokens),
            seconds_until_full=math.ceil((self.capacity - bucket.tokens) / self.refill_per_second),
        )


def rate_limit(bucket: TokenBucket, authenticate: Callable[..., Awaitable[str]]):
    # Header syntax from the IETF draft draft-ietf-httpapi-ratelimit-headers-11.
    policy = f'"default";q={bucket.capacity};w={math.ceil(bucket.capacity / bucket.refill_per_second)}'

    # async def on purpose: FastAPI runs plain def dependencies in a thread pool,
    # where two requests could update the same bucket at once. With async def and
    # no await inside take(), each check runs start to finish on the event loop.
    async def rate_limited_customer(
        response: Response, customer_id: Annotated[str, Depends(authenticate)]
    ) -> str:
        decision = bucket.take(customer_id)
        if decision.allowed:
            response.headers["RateLimit-Policy"] = policy
            response.headers["RateLimit"] = f'"default";r={decision.remaining};t={decision.seconds_until_full}'
            return customer_id
        # On a 429, t equals Retry-After: the draft says Retry-After shouldn't point
        # to a time earlier than the end of the window that t describes.
        raise HTTPException(
            status_code=429,
            detail="rate_limited",
            headers={
                "RateLimit-Policy": policy,
                "RateLimit": f'"default";r=0;t={decision.retry_after_seconds}',
                "Retry-After": str(decision.retry_after_seconds),
            },
        )

    return rate_limited_customer

In app.py, authenticate is a sub-dependency of the limiter, so it always runs first:

rate_limited_customer = rate_limit(bucket, authenticate)

@app.get("/v1/hello")
async def hello(customer_id: Annotated[str, Depends(rate_limited_customer)]):
    return {"customer": customer_id}

The tests use FastAPI's TestClient and a fake clock:

cd examples/api-rate-limiting/fastapi
uv run pytest

For a library, slowapi adapts flask-limiter to Starlette and FastAPI, with memory, Redis or Memcached storage. Its documentation still calls it alpha quality.