Repository navigation
Server P10a: measure the Live quote rate over 1, 16 and 256 instruments - #159
Conversation
…nds in 429
runtime_benchmarks gains live_throughput/{1,16,256} (bursts of 64 quotes
over N fixture bonds, through a 1,024-event source shaped like the
server ingress) and snapshot/assemble/{1,16,256}. live_ingress gains an
end-to-end test: quotes offered faster than the runtime values them end
in 429 CAPACITY_EXCEEDED, and every accepted quote is still valued.
No behaviour change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
No actionable comments were generated in the recent review.
 |
| Layer / File(s) | Summary |
|---|---|
Benchmark runtime and fixtures crates/convex-engine/benches/runtime_benchmarks.rs |
The Live engine helper now accepts an explicit runtime plan and quote source. New fixtures and quote-source helpers support plans with 1, 16, and 256 instruments. |
Throughput and assembly measurements crates/convex-engine/benches/runtime_benchmarks.rs |
Criterion now measures repeated 64-quote bursts and snapshot assembly across the configured widths. The throughput benchmark checks complete results, and the assembly benchmark checks the initial reference count. |
Live ingress capacity test
| Layer / File(s) | Summary |
|---|---|
Ingress setup and capacity behavior crates/convex-server/tests/live_ingress.rs |
A shared setup prepares the Live runtime fixture. The new capacity test checks for a retryable CAPACITY_EXCEEDED response and verifies that accepted quotes advance runtime generations while the runtime remains ready and failure-free. |
Priority:
Estimated code review effort: 3 (Moderate) | ~25 minutes
Change: Other
Possibly related PRs
- sujitn/convex#139: Adds the quote-ingress capacity response exercised by the new overload test.
- sujitn/convex#147: Changes the Live input to use an explicit
QuoteSource, which this PR passes to the benchmark engine helper. - sujitn/convex#157: Updates the runtime benchmark path that this PR extends with throughput and snapshot-assembly measurements.
Merge Risk: 
26e93
-
The new benchmarks and capacity test have no identified issue that needs resolution before merge.
Pre-merge checks: 
-
Check name Status Explanation Resolution Docstring Coverage Docstring coverage is 42.11% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 19 functions across 2 files. Write docstrings for the functions missing them to satisfy the coverage threshold. -
Check name Status Explanation Description Check Check skipped - CodeRabbit’s high-level summary is enabled. Title check The title clearly describes the primary change: adding Live quote-rate measurements for 1, 16, and 256 instruments. It is concise and specific. It does not mention the additional overload test, but th… Linked Issues check Check skipped because no linked issues were found for this pull request. Out of Scope Changes check Check skipped because no linked issues were found for this pull request. Full details: Title check
-
Explanation
The title clearly describes the primary change: adding Live quote-rate measurements for 1, 16, and 256 instruments. It is concise and specific. It does not mention the additional overload test, but the title does not need to cover every change.
-
-
Finishing Touches: 💡 1
-
📝 Generate docstrings: 💡
-
- Commit to this branch
- Create a new PR
🧪 Generate unit tests (beta)
-
- Commit to this branch
- Create a new PR
-
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Comment @coderabbitai help to get the list of available commands.
Summary
P10a measures the Live quote rate (D53). It adds benchmarks and one test, and changes no behaviour.
runtime_benchmarks.rs:live_throughput/{1,16,256}: a Live plan of N bonds (copies of the fixture bond under new identities, each with its own quote key) takes bursts of 64 quotes through a 1,024-event source shaped like the server'sQuoteIngress. Each sample waits for the result of the burst's last quote.snapshot/assemble/{1,16,256}: one snapshot over N instruments (configuration and reference reads, quote ingestion, content hash).live_ingress.rs::an_offered_rate_above_the_drain_rate_is_refused_with_429_and_nothing_is_lost: 256-event batches posted faster than the runtime drains end in429 CAPACITY_EXCEEDED(retryable). The runtime then values every accepted quote and stays ready.Numbers
Medians,
ciprofile, WSL2, 24 vCPU, rustc 1.97.1:snapshot/assembleEvery quote revalues every instrument (G21), so the rate falls roughly as 1/N. Against the 10k quotes/s target, 256 instruments reach 34.
Stage split of one tick at 256 instruments (a throwaway probe with the
/metricshistograms installed; not committed):snapshot_assemblecurve_build{graph}(verify + build of the 256-instrument snapshot)calculate(256 valuations on four threads, in awaited chunks of four)valuationCPU (110 µs each, against 82 µs for one bond alone)publishaccept, checkpoint and result clones, health)What the numbers decide (recorded in the plan)
assemble, and the 2 ms verify and build at graph activation) add to that. Only valuing fewer instruments per tick brings it under 1 ms.MarketSnapshotquality is derived from the issues of every quote, curve and fixing, andworker::valuefails every instrument when it is notValid. So one bond's stale or missing quote fails every result in the plan. Cross-quote source skew is the instance the 256-bond bench hit: the one-bond fixture allows 1 s, and the bonds ticked apart (the bench now uses the compiled Live value, 10 minutes). The decision is per-instrument quality (each result judged on the inputs it consumed) or the current snapshot-wide rule.calculateawaits each chunk ofmax_concurrent_calcsjobs before spawning the next.acceptandcalculateclone the whole checkpoint, results and outbox, on every tick.Verification
tools/assurance/check.pypassed; clippy-D warningson the engine and server (all targets) passed; thelive_ingresstests pass (3 of 3). The onlycargo fmt --checkfinding was one import's order in the bench, fixed before this commit.live_ingresspasses. The new bench cases pass in criterion test mode.🤖 Generated with Claude Code