Rate and concurrency limits

Requests per window and requests at a time, per client or shared, across processes. In fastapi_alcazaba.rate_limit.

In the guide: Rate limiting, Concurrency limits. In an app: the example.

fastapi_alcazaba.rate_limit.FastAPIRateLimiter

FastAPIRateLimiter(times: int = 1, milliseconds: int = 0, seconds: int = 0, minutes: int = 0, hours: int = 0, identifier: Identifier | None = None, callback: OnLimit | None = None, *, name: str | None = None, cache: Cache | None = None, fail_open: bool = True, cost: int = 1, headers: bool = True)

FastAPI dependency that lets each client make times requests per window to a route; the next ones get a 429 with Retry-After until the window slides past enough of them.

@app.get("/search", dependencies=[Depends(FastAPIRateLimiter(times=10, minutes=1))])

The counters live in the cache's backend (the cache given, else the one get_cache returns), so every process shares them, and go through its timeout and circuit breaker. Their keys are SHA-256 digests: identities never reach the backend in clear.

It's a sliding window: the count of the previous window is added in proportion to how much of it is still less than a window ago, so no client doubles its rate around the boundary between two windows. That assumes the previous window's requests were spread evenly; when they came at its end, it's a little stricter than times per window. Requests over the limit don't count: checking and counting is one atomic operation where the backend allows it. Each process then refuses a client with nothing left without asking the backend, until something fits again, so a flood doesn't reach the backend.

Responses carry the RateLimit and RateLimit-Policy headers of the IETF draft (draft-ietf-httpapi-ratelimit-headers), for clients to slow down before they get a 429.

Parameters:

Name	Type	Description	Default
times	int	requests allowed per window.	1
milliseconds	int	the window is the sum of milliseconds, seconds, minutes and hours.	0
identifier	Identifier | None	who the client is; its address by default (client_ip). E.g. a user id.	None
callback	OnLimit | None	called with the request and the seconds to wait when a request is over the limit; too_many_requests by default. If it returns, the request goes on.	None
name	str | None	the routes using limiters with this name and the same rate share one budget; by default each route (its methods and path template) has its own. Also the policy's name in the headers.	None
cache	Cache | None	the Cache whose backend keeps the counters.	None
fail_open	bool	when the backend fails, let requests through (the default, as the cache does) or refuse them with a 503.	True
cost	int	how much of the budget each request takes: an expensive route can take more of a budget it shares with others (name).	1
headers	bool	send the RateLimit and RateLimit-Policy headers.	True

forget

forget() -> None

Forget the clients this process refuses without asking the backend (those with nothing left): their next requests ask it again. The counters in the backend stay. It's done by itself when the cache changes, as each test's lifespan sets one up.

hit async

hit(connection: HTTPConnection, key: str = '', cost: int | None = None) -> float | None

Count a request: None if it's within the limit, else the seconds until it would be (and it isn't counted). Call it yourself to limit something else than whole requests: each message of a WebSocket, or the login attempts to an account (key=username). cost is how much of the budget it takes (the limiter's cost by default).

fastapi_alcazaba.rate_limit.client_ip

client_ip(connection: HTTPConnection) -> str

The client's address as the ASGI server reports it, IPv6 addresses by their /64 network: that's what an ISP gives a single customer, who can use any of its 2⁶⁴ addresses. Behind a proxy, let the server take it from the proxy's headers (uvicorn --proxy-headers --forwarded-allow-ips=<proxy address>): read here, X-Forwarded-For would let every client choose who it is.

fastapi_alcazaba.rate_limit.client_network

client_network(ipv4_prefix: int = 32, ipv6_prefix: int = 64) -> Identifier

An identifier like client_ip with other network sizes: client_network(24, 48) counts a whole IPv4 /24 or IPv6 /48 as one client.

fastapi_alcazaba.rate_limit.too_many_requests

too_many_requests(connection: HTTPConnection, retry_after: float, headers: Mapping[str, str] | None = None) -> NoReturn

The default reaction to a request over the limit: a 429 with Retry-After (and headers), or closing a WebSocket with 1008 (policy violation).

fastapi_alcazaba.rate_limit.FastAPIConcurrencyLimiter

FastAPIConcurrencyLimiter(limit: int, *, name: str | None = None, identifier: Identifier | None = None, wait: float = 0.0, ttl: float = 60.0, cache: Cache | None = None, fail_open: bool = True, retry_after: int = 1)

FastAPI dependency (with yield) that lets at most limit requests run at the same time; the slot is released when the request is done (see the module's docstring for streaming).

@app.post("/summaries", dependencies=[Depends(FastAPIConcurrencyLimiter(8, wait=5))])

The slots live in the cache's backend (the cache given, else the one get_cache returns), so every process shares them. Their keys are SHA-256 digests: identities never reach the backend in clear.

Parameters:

Name	Type	Description	Default
limit	int	requests that may run at the same time, 1 to MAX_LIMIT.	required
name	str | None	routes whose limiters have this name and the same limit share the slots; by default each route (its methods and path template) has its own. Needed by slot() outside requests.	None
identifier	Identifier | None	who the client is (e.g. client_ip, or a user id): each one gets limit slots of its own, and is refused with a 429 instead of a 503. None: every client shares them.	None
wait	float	seconds to wait for a slot when they're all taken; 0 tries once.	0.0
ttl	float	seconds after which the slot of a process that died is freed; renewed every ttl / 3 seconds while the request runs.	60.0
cache	Cache | None	the Cache whose backend keeps the slots.	None
fail_open	bool	when the backend fails, count the slots in each process instead (so at most limit per process run; the default), or refuse requests with a 503.	True
retry_after	int	the Retry-After of refused requests, in seconds.	1

slot async

slot(key: str = '', *, connection: HTTPConnection | None = None) -> AsyncIterator[None]

Hold a slot around a block, outside a dependency:

async with limiter.slot(key=tenant_id):
    await render(report)

key gives each value its own limit slots. With a connection, the route and the identifier apply as in the dependency; without one, the limiter needs a name. Raises ConcurrencyLimitExceeded when no slot frees within wait seconds, and BackendUnavailable when the backend fails and fail_open is False.

fastapi_alcazaba.rate_limit.ConcurrencyLimitExceeded

ConcurrencyLimitExceeded(status_code: int, retry_after: int)

Bases: HTTPException

Every slot stayed taken for as long as the limiter waits. FastAPI answers it with a 503 (a 429 when the limit is per client) and Retry-After.