Repository navigation
Admission lanes: light jobs stop waiting behind heavy ones - #116
Merged
Merged
Conversation
….1, 0.2, 0.6, 3.3) One host-wide FIFO made a 5-second job wait behind 12-minute Xcode builds; on ultra load rises ~8 per concurrent job on 30 cores and 74 % of jobs finish in under 90 s. - hooks: FLEET_ADMIT_LANES=off|observe|enforce. Each job is simulator (runner rule, also takes a heavy slot), heavy or light: HEAVY/LIGHT patterns (repo:job shell globs on GITHUB_JOB), then the dashboard's /api/job-class (median of recent ran_s from the admission log, cached in .admission/job-class), else heavy. Limits HEAVY (4), LIGHT (12; SHORT_MAX_CONCURRENT is the alias), TOTAL (default MAX_CONCURRENT), FIFO per lane with the lockless check reading the line from the waiter's name. observe keeps the single lane deciding and logs lane_would_hold / lane_reason / lane_wait_s; off logs exactly the old line. Hold reasons name the lane; legacy reasons unchanged. - dashboard: /api/job-class, /api/load-by-busy, lane columns on admission_events, lanes block in /api/admission and the Capacity tab. - cockpit load: load1 p50/p90 by busy runners. - cleanup.sh: erase idle runners' _work/<repo>/DerivedData-*, spm-*, .build-cache-* unused 7 days (all under pressure); write missing Spotlight markers; ephemeral runners get the marker too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
addisdev
force-pushed
the
ci/admission-lanes
branch
from
October 23, 2026 03:38
a5d8382 to
71f30a0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI throughput plan (
CI-OPTIMIZATION-PLAN-2026-10.md) Phase 0.1, 0.2, 0.6 and 3.3. Owner decision WB-617: heavy 4, Simulator 2, light up to 12.What changes
Lanes in the admission hook (
FLEET_ADMIT_LANES=off|observe|enforce, defaultoff)simulator(existing runner rule; also takes a heavy slot),heavyorlight, in this order: Simulator runner,FLEET_ADMIT_HEAVY_PATTERNS,FLEET_ADMIT_LIGHT_PATTERNS(space-separatedrepo:jobshell globs onGITHUB_JOB, repo with or without owner), then the dashboard'sGET /api/job-class(median of the job's last 30ran_sfrom the admission log's release lines; light underFLEET_ADMIT_SHORT_JOB_S, ≥ 5 samples), else heavy. The hook caches the answer in.admission/job-class/for 300 s and gives the daemon 3 s.FLEET_ADMIT_HEAVY_MAX_CONCURRENT(4),FLEET_ADMIT_LIGHT_MAX_CONCURRENT(12; the deadFLEET_ADMIT_SHORT_MAX_CONCURRENTis its alias), Simulator unchanged, underFLEET_ADMIT_TOTAL_MAX_CONCURRENT(defaultFLEET_ADMIT_MAX_CONCURRENT).<ts>-<pid>-<light|heavy>-<runner>so the lockless check finds its own line's head from one listing; old-format names read as heavy. Mutex, reaping, cancel polling, disk floors, timeout semantics and the strict Simulator rule are unchanged.observe: the single lane decides exactly as today; every row also recordslane,lane_source, lane counts and limits,lane_would_hold,lane_reasonandlane_wait_s(when the lanes would first have admitted a held job).offwrites byte-for-byte the old line.4 heavy job(s) already running, at the heavy limit of 4,an older job is waiting for the next heavy slot); the legacy reasons keep their words, so the lake'sfct_admission_holdclassifier still files them.Dashboard:
/api/job-class,/api/load-by-busy, 11 lane columns onadmission_events(addColumn) plus an index on(repo, job, ts), alanesblock in/api/admission, lane rows and a lane column on the Capacity tab.cockpit load: host load1 p50/p90 (and p90/core) by busy-runner count over--days N, fromhost_samples.Phase 0.6:
cleanup.sherases an idle runner's_work/<repo>/DerivedData-*,spm-*,.build-cache-*with nothing written inFLEET_CLEANUP_RUNNER_CACHE_DAYS(7), all of them under pressure (never a building runner's, never_work/_*, never a git checkout), and writes any missing_work/.metadata_never_index.register.sh(and somirror-runners.sh) already wrote it;ephemeral-runner.shnow does too.Tests
hooks/tests/run.sh: 150 pass locally (55 new: classification incl. cache and fallback, heavy/light/total limits, per-lane FIFO, Simulator counts as heavy, legacy alias, observe vs enforce, lockless per-lane check).scripts/test-cleanup.sh: 77 pass (15 new).job-class.test.js, lane cases inadmission.test.js, route tests for both endpoints; related suites pass.LoadTableTestsrun in CI only (laptop under load).Rollout
Merge, deploy to ultra in observe (single lane keeps deciding at 8), review the observe data, then
FLEET_ADMIT_LANES=enforce.🤖 Generated with Claude Code