FastAPI and Starlette

The FastAPI middleware checks each request's API key, rate limit and quota before your path operation runs, and counts the request when the response has been sent. It's a plain ASGI middleware, so it works the same with Starlette.

pip install usagerail

import os

from fastapi import FastAPI, Request
from usagerail.fastapi import UsageRailMiddleware

app = FastAPI()
app.add_middleware(
    UsageRailMiddleware,
    app_key=os.environ["USAGERAIL_APP_KEY"],
    auth="api-key",
)


@app.post("/v1/generate")
async def generate(request: Request):
    result = await run_model(await request.json())
    request.state.usagerail.track(result.tokens)  # optional: count tokens
    return result.body

The SDK needs Python 3.10 or newer and has no dependencies of its own. The key comes from the Authorization: Bearer header, or from x-api-key when the request has no bearer token.

Options

API keys and quotas explain the three availability settings.

Count units instead of requests

In apps that count units, call request.state.usagerail.track(n) with a whole number from 0 to 10,000,000. Call it again to replace the number. A billable request that never calls it counts as 1, or as the route's weight. In apps that count requests, track() does nothing.

Use your own authentication

Pass auth="custom" and a consumer function. It can return a string ID, a dict with an "id" key, or None. Return an identity your own code has verified, never a value the caller can set, such as a plain header: anyone could put another customer's ID there.

from starlette.middleware.authentication import AuthenticationMiddleware

app.add_middleware(
    UsageRailMiddleware,
    app_key=os.environ["USAGERAIL_APP_KEY"],
    auth="custom",
    consumer=lambda request: request.user.identity if request.user.is_authenticated else None,
)
# Added last, so it runs first and sets request.user
app.add_middleware(AuthenticationMiddleware, backend=YourAuthBackend())

The function runs before your path operations, which means before Depends(...), so the identity has to come from a middleware. FastAPI runs the middleware added last first, so add your authentication middleware after the line that adds UsageRail.

Returning None makes the request anonymous: it's recorded, but not rate limited, not counted against a quota and never billed.

Rejected requests

Refused requests get the statuses and JSON body listed on the Express page, such as {"error": "rate_limited", "retry_after": 1}. To send your own body, pass on_reject and keep the headers:

from fastapi.responses import JSONResponse


def reject(rejection, request):
    return JSONResponse(
        {"code": rejection.error},
        status_code=rejection.status,
        headers=dict(rejection.headers),
    )


app.add_middleware(UsageRailMiddleware, app_key=..., auth="api-key", on_reject=reject)

What gets counted

• Each request is counted under the route's path template, such as /v1/items/{item_id}, with its method and outcome. Prefixes from include_router and from Mount are part of the template.
• 2xx and 3xx responses are billable. 4xx and 5xx responses, refused requests and requests the client left before the response started are recorded but not billed.
• A request no route matched, including the redirect Starlette sends for a missing trailing slash, is recorded under _unmatched. That redirect is a 3xx, so it's billed as 1 unit.
• A streaming response counts once the body has been sent. Server-sent events count once when the stream starts, and track() units are added when it ends.
• WebSocket connections aren't checked or counted.
• While the middleware waits for an answer from UsageRail, such as for a key it hasn't seen yet, it reads at most 64 KB of the request body to notice a client that hangs up.

Running in production

• Counts are sent every 60 seconds, and once more in FastAPI's lifespan shutdown, which uvicorn runs on SIGTERM.
• If a process is killed outright (kill -9, out of memory), counts it hadn't sent are lost and never billed.
• With gunicorn --preload, each worker becomes its own SDK instance after the fork and keeps the key list it inherited.
• The key list is saved in a private folder in the system temp directory. Pass snapshot=False to turn this off.