Repository navigation
Limit concurrent large /search requests #26
Description
Large /api/v1/search requests are CPU-bound on the single event loop (building rows, then formatting CSV or Pydantic models for JSON), so a few at once slow down every other request.
Measured in review of #20, against a local Postgres 16 container seeded with 100,001 small rows: a JSON request sent 2 s into a burst of 15 parallel requests took:
| Burst of 15 | JSON request during the burst |
|---|---|
CSV limit=100000 |
6.9 s |
| CSV with no limit, over the 100,000 cap (400) | 10.4 s |
JSON limit=100000 |
30.4 s |
#20 caps a JSON limit at 5,000, which removes the last case. CSV exports remain, and production rows are larger than these seeded ones, so the real numbers are likely higher.
Proposal: limit how many large /search requests run at once, for JSON and CSV alike, and return a 503 with Retry-After when the limit is reached. Small requests, like the frontend's 10/20/50-row pages, shouldn't count against it. To decide:
- What counts as large (a row threshold on the effective limit, e.g. 1,000) and how many may run at once.
- What the slot covers. For CSV, releasing it once the spooled file is written (before sending) seems right, since sending costs little CPU. For JSON, response serialization happens after the handler returns, so a slot held only inside the handler wouldn't cover it. Check where FastAPI 0.115 runs serialization relative to
yield-dependency exit before relying on a dependency for this. - Whether this belongs in the app or in the ingress (Traefik), which could also rate-limit many small requests.
Activity