[DustinTrap]

Closes #23.

What

A client can only count its own requests, so a fan-out cannot tell how many of the endpoint's 8 slots other consumers already hold. scripts/wait-for-slot.sh samples vllm:num_requests_running and vllm:num_requests_waiting through llama-swap and exits 0 once running + waiting + NEED <= LIMIT, or gives up after --max-wait.

• One GET of /upstream/<model>/metrics per sample. No inference request; nothing changes on the server.
• Key from an environment variable or a key file, passed to curl on stdin, never in argv, never printed (same as scripts/check-keys.sh).
• Defaults: --limit 6, --need 1, sample every 5 s, give up after 600 s. The 6 is a policy choice (leave 2 slots for latency-sensitive consumers), not a measurement.
• --check takes one sample: exit 0 room, 1 no room.
• No answer, a 5xx or a 429 counts as "no room". 401, 403, 404 or an answer without the gauges exits 2.
• Advisory only: two clients can take the same free slot. concurrencyLimit in llama-swap.yaml stays the hard limit.

Docs: the README lists the script and shows the call under Operations; examples/opencode/AGENTS.md runs the gate before launching more than one sub-agent; examples/opencode/README.md shows the wrapper that puts wait-for-slot on PATH.

Tested

• Against a local mock metrics server: room, busy, wait-then-room, give-up after --max-wait, 401, 404, 500, request timeout, missing gauges, key file with CRLF, usage errors.
• One read-only sample from the live endpoint (--check), exit 0.
• bash -n and shellcheck pass; gitleaks passes over the tree and full history.

Not verified

• That a model follows the new AGENTS.md line in a real session.
• That "one sub-agent at a time adds no request" holds (expected from how the tool call blocks the parent; not measured).
• The two items already listed as unverified on Client-side admission gate: wait for a free slot before a fan-out (scripts/wait-for-slot.sh) #23.

No change on the box and no change to llama-swap.yaml.

🤖 Generated with Claude Code