perf(prover): default the cuda table scheduler to K = num_airs - #911
Open
MauroToscano wants to merge 2 commits into
Open
perf(prover): default the cuda table scheduler to K = num_airs#911MauroToscano wants to merge 2 commits into
MauroToscano wants to merge 2 commits into
Conversation
`table_parallelism()`'s cuda arm scaled K by `available_parallelism()`
(`cores * 2 / 3`). Measured over 881 runs on two RTX 5090 boxes, that is the
wrong shape. All eight core-count curves fit `T(K) = S + max(Tmax, W/K)` within
run-to-run noise, and the work K divides — W ≈ 5.3-8.0 s — is invariant to host
core count over an 8x range, to CPU model, and to rayon pool width: cutting
RAYON_NUM_THREADS 32 -> 4 leaves W alone and merely doubles S, with the best K
still num_airs at every pool width. `available_parallelism()` sizes precisely
that rayon pool, so it is the wrong quantity to scale K by. K is not a thread
count; each table's work runs on the one global pool.
Worst case against the best measured K, over four core counts on both boxes:
cores/3 +30.2 %
cores*2/3 +13.0 % (what this replaces)
constant 12 +7.0 %
num_airs +1.6 % (both non-zero cells inside noise, p = 0.88 / 0.80)
`cores*2/3` fails where it was predicted to: low core counts, K=2 at 4 cores
(+13.0 %) and K=5 at 8 cores (+8.1 %).
Taking the ceiling rather than solving for an optimum is right in both regimes
of the fit: if W/num_airs > Tmax more K strictly helps, and if W/num_airs < Tmax
the extra drivers are floor-limited and cost nothing — the one staging slab is
held 56 % of wall at K=31 and wall time still improves. The old doc comment's
mechanism ("in-flight tables mostly sit in GPU waits") is not what happens —
mean GPU utilisation never exceeded ~38 % at any K — so it is rewritten rather
than re-tuned. What is meant to bound concurrency is memory admission rather
than a count: that is what VramGate is for, and it never binds at the default
budget.
`table_parallelism` now takes `num_airs` and clamps to it, replacing the
`.min(num_airs)` the call site applied. `auto_storage::decide` keeps a bounded
figure through the new `storage_estimate_parallelism()`: `peak_bytes` sums the
transient bytes of the top-k tables, so an unbounded k there sums every table —
measured +27 % at 128 PAGE tables, +44 % at 512 — and would spill proofs to disk
that fit in RAM. Its value is unchanged, so no storage decision moves.
The CPU arm keeps `cores / 3`. The sweep ran only on cuda builds, where the
parallelized work is device-bound; on a CPU-only build every table is pure host
work and none of this evidence transfers.
The measurement behind the previous commit, under `scripts/profiling/table-parallelism-sweep/`: both write-ups, every timing CSV (881 runs over two RTX 5090 boxes, one row per run), the analysis scripts, the server-side sweep harnesses, the box/toolchain fingerprints, the frozen job orders and the diffs for the three experimental builds. `rules2.py` reproduces the rule-cost table from the committed CSVs; `amdahl.py` reproduces the fits. `thoughts/` is gitignored, so this lands next to the profiling tooling instead. Build logs, the per-run logs for the stages whose hypotheses were refuted, and the Stage 1a stack samples (gdb could not attach — the rented container dropped cap_sys_ptrace) are left out; the README says what is included and what is not.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The cuda arm of
table_parallelism()scales K byavailable_parallelism()(
cores * 2 / 3). Measured over 881 runs on two RTX 5090 boxes, that is the wrongshape — not the wrong constant.
All eight core-count curves fit
T(K) = S + max(Tmax, W/K)within run-to-run noise,and the work K divides (W ≈ 5.3–8.0 s) is invariant to host core count over an 8×
range, to CPU model, and to rayon pool width. The decisive one: cutting
RAYON_NUM_THREADS32 → 4 leaves W alone and merely doubles S, with the best K stillnum_airsat every pool width.available_parallelism()sizes precisely that rayonpool, so it is the wrong quantity to scale K by — K is not a thread count, each
table's work runs on the one global pool.
Worst case against the best measured K, over four core counts on both boxes:
cores/3cores*2/3(what this replaces)num_airscores*2/3fails where predicted: low core counts, K=2 at 4 cores (+13.0%), K=5 at 8cores (+8.1%). No K below
num_airsis significantly faster thannum_airsanywhereon either box (every such cell p ≥ 0.38 at n=8).
Taking the ceiling rather than solving for an optimum is right in both regimes of the
fit: if
W/num_airs > Tmaxmore K strictly helps; ifW/num_airs < Tmaxthe extradrivers are floor-limited and cost nothing — the one staging slab is held 56% of wall
at K=31 and wall time still improves.
The old doc comment's mechanism ("in-flight tables mostly sit in GPU waits") is not
what happens — mean GPU utilisation never exceeded ~38% at any K — so it is rewritten
rather than re-tuned. What is meant to bound concurrency is memory admission rather
than a count: that is what
VramGateis for.auto_storageis held boundedtable_parallelismnow takesnum_airsand clamps to it, replacing the.min(num_airs)the call site applied.
auto_storage::decidekeeps a bounded figure through the newstorage_estimate_parallelism():peak_bytessums the transient bytes of the top-ktables, so an unbounded k there sums every table — measured +27% at 128 PAGE tables,
+44% at 512 — and would spill proofs to disk that fit in RAM. Its value is unchanged,
so no storage decision moves. Two tests pin both halves.
The CPU arm is untouched
cores / 3stays. The sweep ran only on cuda builds, where the parallelized work isdevice-bound; on a CPU-only build every table is pure host work and none of this
evidence transfers.
Evidence
scripts/profiling/table-parallelism-sweep/— both rounds' write-ups, result CSVs andre-runnable harnesses, so every number above is checkable.
Honest limits
epoch-size-log2 21, ~31 tables. Production tables are much larger, so memory per
concurrently-admitted table is larger and the
auto_storageinteraction is unverifiedat that scale — a big-block check is a merge gate, not something this PR establishes.
taskset -c 0-7is oneCCD with 32 MB L3 vs the 3D part's much larger cache).
T(1) − T(best), not instrumented per phase.S + W/Kwith nosaturating resource, so a tolerance-knee is derived and reads 8 on one box and 16 on
the other at the same 5% tolerance.
Verified
cargo fmt --checkclean;cargo clippy --all-targets -p stark -p lambda-vm-proverclean.
auto_storage_tests8 passed (needs--features disk-spill);table_parallelism_stays_within_one_and_num_airspasses. The cuda-gated test and thecuda arm itself cannot compile here — no local CUDA toolchain — so both were verified by
reading plus a non-cuda build.