PaginationScroll¶
An export of orders: a script reads every order in batches while the shop keeps selling, and
gets them as they were when it started. The query runs once; its results are kept in the cache's
backend, and each scroll_id reads the next batch of them. The strategy:
PaginationScroll.
Run it¶
From a clone of the repository:
make deps # .venv, with every extra
source .venv/bin/activate
uvicorn examples.pagination.scroll:app
Set SCROLL_SECRET and CACHE_SIGNING_KEY (secrets.token_urlsafe(32) each), the same in every
process. Without them each start makes up throwaway ones, and the scroll ids stop working when the
app restarts.
Try it¶
The export starts: the first four of the 12 orders, and the scroll_id of the next batch, in the
body and in the Link header:
$ curl -i "http://localhost:8000/orders/export?batch_size=4"
HTTP/1.1 200 OK
content-type: application/json
link: </orders/export?batch_size=4&scroll_id=WyJsR3NqcG9ZNGo5QVZIMGFoU3ZJZDVRIiw0XQ.3VRVV5FF93szTc6nwQegzA>; rel="next"
{"items":[{"customer":"customer-1","amount":10.0,"id":1,"status":"paid"},{"customer":"customer-2","amount":20.0,"id":2,"status":"paid"},{"customer":"customer-3","amount":30.0,"id":3,"status":"shipped"},{"customer":"customer-4","amount":40.0,"id":4,"status":"paid"}],"scroll_id":"WyJsR3NqcG9ZNGo5QVZIMGFoU3ZJZDVRIiw0XQ.3VRVV5FF93szTc6nwQegzA","total":12}
An order is placed meanwhile:
$ curl -X POST http://localhost:8000/orders -H "Content-Type: application/json" -d '{"customer": "late", "amount": 99.5}'
{"customer":"late","amount":99.5,"id":13,"status":"paid"}
The export goes on with the orders it found: still 12, nothing seen twice. The same request again, as a script does when the response is lost, gets the same batch:
$ curl "http://localhost:8000/orders/export?batch_size=4&scroll_id=WyJsR3NqcG9ZNGo5QVZIMGFoU3ZJZDVRIiw0XQ.3VRVV5FF93szTc6nwQegzA"
{"items":[{"customer":"customer-5","amount":50.0,"id":5,"status":"paid"},{"customer":"customer-6","amount":60.0,"id":6,"status":"shipped"},{"customer":"customer-7","amount":70.0,"id":7,"status":"paid"},{"customer":"customer-8","amount":80.0,"id":8,"status":"paid"}],"scroll_id":"WyJsR3NqcG9ZNGo5QVZIMGFoU3ZJZDVRIiw4XQ.FvL2Kiaq0Y5sccGOjqoMZQ","total":12}
$ curl "http://localhost:8000/orders/export?batch_size=4&scroll_id=WyJsR3NqcG9ZNGo5QVZIMGFoU3ZJZDVRIiw0XQ.3VRVV5FF93szTc6nwQegzA"
{"items":[{"customer":"customer-5","amount":50.0,"id":5,"status":"paid"},{"customer":"customer-6","amount":60.0,"id":6,"status":"shipped"},{"customer":"customer-7","amount":70.0,"id":7,"status":"paid"},{"customer":"customer-8","amount":80.0,"id":8,"status":"paid"}],"scroll_id":"WyJsR3NqcG9ZNGo5QVZIMGFoU3ZJZDVRIiw4XQ.FvL2Kiaq0Y5sccGOjqoMZQ","total":12}
The last batch has no scroll_id, and no Link:
$ curl "http://localhost:8000/orders/export?batch_size=4&scroll_id=WyJsR3NqcG9ZNGo5QVZIMGFoU3ZJZDVRIiw4XQ.FvL2Kiaq0Y5sccGOjqoMZQ"
{"items":[{"customer":"customer-9","amount":90.0,"id":9,"status":"shipped"},{"customer":"customer-10","amount":100.0,"id":10,"status":"paid"},{"customer":"customer-11","amount":110.0,"id":11,"status":"paid"},{"customer":"customer-12","amount":120.0,"id":12,"status":"shipped"}],"scroll_id":null,"total":12}
A scroll_id holds the snapshot's id and where the next batch starts, signed. One changed by
hand to skip a batch doesn't match its signature:
$ curl -i "http://localhost:8000/orders/export?scroll_id=WyJsR3NqcG9ZNGo5QVZIMGFoU3ZJZDVRIiw4XQ.3VRVV5FF93szTc6nwQegzA"
HTTP/1.1 400 Bad Request
{"detail":"Invalid scroll_id"}
A new export runs the query again, and finds the 13 orders. When the script is done with it, it deletes the snapshot rather than leave it until it expires (60 seconds after its last batch):
$ curl "http://localhost:8000/orders/export?batch_size=10"
{"items":[...],"scroll_id":"WyJwNXRWQU9uV0ZnS3VuM01iSTlNekp3IiwxMF0.9cQSX6b7Hpxm_wVvrAsyAg","total":13}
$ curl -i -X DELETE "http://localhost:8000/orders/export?scroll_id=WyJwNXRWQU9uV0ZnS3VuM01iSTlNekp3IiwxMF0.9cQSX6b7Hpxm_wVvrAsyAg"
HTTP/1.1 204 No Content
$ curl -i "http://localhost:8000/orders/export?batch_size=10&scroll_id=WyJwNXRWQU9uV0ZnS3VuM01iSTlNekp3IiwxMF0.9cQSX6b7Hpxm_wVvrAsyAg"
HTTP/1.1 404 Not Found
{"detail":"Expired or unknown scroll_id"}
Batches have a maximum size, checked before anything is queried:
$ curl "http://localhost:8000/orders/export?batch_size=500"
{"detail":[{"type":"less_than_equal","loc":["query","batch_size"],"msg":"Input should be less than or equal to 50","input":"500","ctx":{"le":50}}]}
What to look at¶
- One query per export: without a
scroll_idthe endpoint queries the orders andstartkeeps them; with one,nextreads the batch from what was kept, and the database isn't touched. - A snapshot: what was found when the export started, whatever is written meanwhile. Items added, changed or deleted don't shift the batches.
- Retries are safe: the
scroll_idsays where the batch starts, so the same request gets the same batch; reading a batch doesn't move anything forward. - Kept alive while it's read: each batch gives the snapshot another 60 seconds. A script that stops for longer gets a 404 and starts again.
- Bounded: the query asks for one row more than
max_items, andstartrefuses more than that, rather than keep a million rows in the backend. - Sealed in the backend: the cache's signing key seals the snapshot, so whoever can write to
the backend can't plant one; an
encryption_keyinstead also hides it, for personal data. - Not cached: the endpoint has no
@cache, since each start keeps a snapshot of its own.
The code¶
examples/pagination/scroll.py
"""An export of orders by scroll: a script reads every order in batches while the shop keeps
selling, and gets what was there when it started.
uvicorn examples.pagination.scroll:app
`PaginationScroll` keeps what the query found as a snapshot in the cache's backend, and each
`scroll_id` reads its next batch: orders placed meanwhile don't show, none is skipped or seen
twice, and a retried request gets the same batch. Set SCROLL_SECRET and CACHE_SIGNING_KEY
(secrets.token_urlsafe(32) each), the same in every process, in the environment or a .env file.
"""
import secrets
from collections.abc import AsyncIterator
from contextlib import asynccontextmanager
from enum import StrEnum
from typing import Annotated
import uvicorn
from fastapi import Depends, FastAPI, Query, status
from pydantic import BaseModel, Field, SecretStr
from pydantic_settings import BaseSettings, SettingsConfigDict
from fastapi_alcazaba import FastAPICache
from fastapi_alcazaba.backends.inmemory import InMemoryBackend
from fastapi_alcazaba.pagination.scroll import PaginationScroll, ScrollPage, ScrollParams
def throwaway_secret() -> SecretStr:
"""A random secret, for trying the example: it changes at every start and differs between
processes."""
return SecretStr(secrets.token_urlsafe(32))
class Settings(BaseSettings):
"""The configuration, from environment variables or a .env file."""
model_config = SettingsConfigDict(env_file=".env", env_file_encoding="utf-8", extra="ignore")
scroll_secret: SecretStr = Field(default_factory=throwaway_secret)
cache_signing_key: SecretStr = Field(default_factory=throwaway_secret)
settings = Settings()
class Status(StrEnum):
PAID = "paid"
SHIPPED = "shipped"
class NewOrder(BaseModel):
customer: str = Field(min_length=1, max_length=100)
amount: float = Field(gt=0)
class Order(NewOrder):
id: int
status: Status = Status.PAID
class Database:
"""Stands in for the database: 12 orders, every third one shipped."""
def __init__(self) -> None:
self.orders = [
Order(
id=n,
customer=f"customer-{n}",
amount=10.0 * n,
status=Status.SHIPPED if n % 3 == 0 else Status.PAID,
)
for n in range(1, 13)
]
def find(self, state: Status | None, limit: int) -> list[Order]:
"""SELECT * FROM orders WHERE status = :state ORDER BY id LIMIT :limit"""
return [order for order in self.orders if state is None or order.status == state][:limit]
def add(self, order: NewOrder) -> Order:
saved = Order(id=len(self.orders) + 1, **order.model_dump())
self.orders.append(saved)
return saved
db = Database()
# Signed: clients can't make scroll ids up. Every process must have the same secret, or a
# scroll_id made by one is refused by the others.
scroll = PaginationScroll(
Order,
settings.scroll_secret,
default_batch_size=5,
max_batch_size=50,
keep_alive=60,
max_items=1_000,
)
@asynccontextmanager
async def lifespan(_: FastAPI) -> AsyncIterator[None]:
# Redis or Valkey in production, so that every process reads the same snapshots. The
# signing key seals them: whoever can write to the backend can't plant one.
FastAPICache.init(InMemoryBackend(), prefix="shop", signing_key=settings.cache_signing_key)
yield
app = FastAPI(lifespan=lifespan)
@app.get("/orders/export")
async def export_orders(
params: Annotated[ScrollParams, Depends(scroll)],
state: Annotated[Status | None, Query(alias="status")] = None,
) -> ScrollPage[Order]:
"""Without a scroll_id, the query runs and its results are kept for the next batches; with
one, the next batch comes from them and the query doesn't run again. Not cached: each start
keeps a snapshot of its own."""
if params.scroll_id is None:
# One more than the scroll keeps: if they're all there, start() refuses them.
return await scroll.start(db.find(state, scroll.max_items + 1), params)
return await scroll.next(params)
@app.delete("/orders/export", status_code=status.HTTP_204_NO_CONTENT)
async def stop_export(scroll_id: str) -> None:
"""The script is done (or gives up): the snapshot goes now, not when it expires."""
await scroll.clear(scroll_id)
@app.post("/orders", status_code=status.HTTP_201_CREATED)
async def add_order(order: NewOrder) -> Order:
"""A new order: the exports running don't see it, the next ones will."""
return db.add(order)
if __name__ == "__main__":
uvicorn.run("scroll:app", reload=True)