How it works / Rate limits

Rate limits

Rate limits cap how fast one customer can call your API: a number of requests per second, per minute, or both. The SDK enforces them in memory, so a request over the limit is refused without a network call, and it never reaches your handler.

http

HTTP/1.1 429 Too Many Requests
Content-Type: application/json
Retry-After: 1
RateLimit-Policy: "per_second";q=10;w=1, "per_minute";q=300;w=60
RateLimit: "per_second";r=0;t=1, "per_minute";r=212;t=18

{"error":"rate_limited","retry_after":1}

Setting limits

Rate limits belong to a plan. Open the plan under Plans and fill in Requests per second, Requests per minute, or both. Every customer on the plan gets the same limits, each counted separately for that customer. Leave a field empty for no limit.

A change reaches every running SDK at its next sync, within 15 seconds. Customers without a plan, and anonymous requests in apps that use their own authentication, have no rate limit.

How the limit is counted

Each limit is a token bucket per customer. A bucket holds as many tokens as the limit and refills at that rate: a limit of 10 per second refills one token every 100 ms. Each request takes a token from each bucket, and a request that finds any of the customer's buckets empty is refused.

So a customer who has been quiet can send a burst as large as the limit, then continues at the limit's pace. With both limits set, the per-minute bucket caps sustained traffic while the per-second bucket caps bursts.

A refused request doesn't take a token, and it isn't billed or counted against the quota.

The 429 response

Header	Meaning
`Retry-After`	Seconds until the bucket that refused the request has a token again, rounded up.
`RateLimit-Policy`	Each limit: `q` is its size and `w` the window in seconds.
`RateLimit`	Each limit right now: `r` is the tokens left. `t` is the seconds until a request fits again for a limit that's run out, and until the bucket is full for the others.

RateLimit-Policy and RateLimit follow the IETF draft for rate limit headers (draft-ietf-httpapi-ratelimit-headers, version 11). It's still a draft, so the names could change before it becomes a standard; Retry-After is the one to rely on.

To change the body, pass onReject to the middleware, and keep the status and headers so that clients can still back off. See Express.

Several servers

The buckets live in each process's memory and aren't shared. Under Settings > Rate limits across servers, you choose what a limit means when your API runs on several servers:

Each server enforces the full limit, the default. With 4 servers and a limit of 10 per second, a customer whose requests are spread evenly can make up to 40 per second. Use this when you mostly want to stop runaway clients.
Split the limit across servers. Each server enforces the limit divided by the number of servers currently reporting, rounded up and never below 1. With 4 servers, each allows 3 per second, so a customer can make about 12. This stays close to the total you set, but a customer whose requests all land on one server is held to that server's share.

The number of servers comes from UsageRail at each sync. A new server is counted as soon as it syncs, so shares shrink within about 15 seconds when you add servers. A server that stops is still counted until 10 minutes after its last sync or report, so after a deploy or a scale-down the remaining servers keep the smaller share for up to about 10 minutes.

Things to know

Restarts. A restarted process starts with full buckets, so right after a deploy a customer can briefly get one extra burst.
New customers with your own authentication. A customer that UsageRail first sees through your consumer function has no rate limit until the next sync brings their plan to the SDK, which takes up to about 15 seconds. Their quota applies from the first request.
Memory. Each process keeps buckets for up to 40,000 recently active customers. A customer whose buckets were dropped starts again with full buckets.