Repository navigation
Scheduler stage 3: job fairness, with per-CPU two-level EEVDF and creation authority - #979
Conversation
|
Metal, head
|
Review r1: PR #979 at
|
… exact Review r1 of #979. The lopsided placement cases bounded migrations by the run's length, so a ping-pong fifty times worse still passed. A trace of every move over the 20 seeds per width shows two causes before the first exit, neither a bounce. At start-up, an idle CPU steals from a neighbour that busy balancing has just given a second thread. At the end of the drain, cpu0 answers a burst of requests posted in one instant, drops below a CPU that took threads earlier, and that CPU hands one on. Each other CPU sources at most two moves before an exit. After the first exit, only an exit opens a pair more than one thread apart, the quarter-thread margin's threshold. The bounds are now at most even_drain + 2 * (cpus - 1) moves before an exit and at most one move per exit after it. Measured worst: even + 8 before (8 CPUs, 64 threads), and 53 moves after 64 exits (4 CPUs, 64 threads). Dropping pop_surplus's margin fails it: 21 moves after 8 exits at 2 CPUs. old_commit_before_pass passed only on a 3/4 threshold, with an unverified account of the misses. Counting call-site commits that go on to park splits the 500 schedules exactly. All 415 caught schedules commit at least once. None of the 85 others does: every consumer block there found a token waiting (507 at entry, 2 at the re-check). The gate now asserts that a schedule fails if and only if it commits at the call site, and keeps a reach floor so the workload cannot go quiet. Pointing it at the fused shape, which the harness cannot see, fails at seed 1. Also: the decision comment in preempt_if_due no longer claims a pass's queue changes only bring the decision forward. windows.rs no longer cites the deleted Frontier. The scheduler track's three stage-3 leftovers each get an exit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r2: PR #979 at
|
… exact Review r1 of #979. The lopsided placement cases bounded migrations by the run's length, so a ping-pong fifty times worse still passed. A trace of every move over the 20 seeds per width shows two causes before the first exit, neither a bounce. At start-up, an idle CPU steals from a neighbour that busy balancing has just given a second thread. At the end of the drain, cpu0 answers a burst of requests posted in one instant, drops below a CPU that took threads earlier, and that CPU hands one on. Each other CPU sources at most two moves before an exit. After the first exit, only an exit opens a pair more than one thread apart, the quarter-thread margin's threshold. The bounds are now at most even_drain + 2 * (cpus - 1) moves before an exit and at most one move per exit after it. Measured worst: even + 8 before (8 CPUs, 64 threads), and 53 moves after 64 exits (4 CPUs, 64 threads). Dropping pop_surplus's margin fails it: 21 moves after 8 exits at 2 CPUs. old_commit_before_pass passed only on a 3/4 threshold, with an unverified account of the misses. Counting call-site commits that go on to park splits the 500 schedules exactly. All 415 caught schedules commit at least once. None of the 85 others does: every consumer block there found a token waiting (507 at entry, 2 at the re-check). The gate now asserts that a schedule fails if and only if it commits at the call site, and keeps a reach floor so the workload cannot go quiet. Pointing it at the fused shape, which the harness cannot see, fails at seed 1. Also: the decision comment in preempt_if_due no longer claims a pass's queue changes only bring the decision forward. windows.rs no longer cites the deleted Frontier. The scheduler track's three stage-3 leftovers each get an exit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… least virtual time The queue kept its entities in a vector and scanned it for every pick, eligibility query, charge and lag read. Every operation is now O(log n): an AVL tree ordered by (virtual deadline, issue order) whose nodes carry their subtree's least virtual time answers the pick (the first eligible entity in deadline order) and `overtaken_in` (the least virtual time among entities due before the running one) by one descent, and a key-to-slot map replaces the linear lookups. Semantics are unchanged, tie-breaks included. `end_request` ends an entity's request where its service stands, which a voluntary yield gives up. Checks: - `the_tree_answers_as_a_scan_of_every_entity_does`: 20,000 random joins, leaves, rejoins, charges, reweights and slice changes, each followed by the tree's shape and augmentation checked whole and its pick and `overtaken_in` compared against a scan. - The stage-1 model (`job_sched`) prints the same reports to the byte with the tree as with the vector. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the job groups it holds Stage 3 of the scheduler plan (owner decision J1): the per-PID fair shares, the global vruntime frontier and the run queue's frozen insertion keys go together, replaced by jobs and two-level per-CPU EEVDF. Jobs. A job is the owner a process's threads and its children share CPU time under (`kernel/pure/sched/job.rs`, kernel object `Job`). A spawn puts its child in its own job unless it endows another under `JOB_LABEL`; making a new job takes a system capability's `JOBS` right (`SYS_JOB_CREATE`), so names and pids grant nothing. `SYS_JOB_OWN` answers a handle to the caller's job, which a launch carries so the supervisor starts the program in its caller's job. The supervisor makes every service a job of its own, and the compositor every program it starts (its row now holds `jobs`). A job lives while a thread of it runs or a handle to it is held. There is no hierarchy above it. Scheduling. Each CPU's fair band is EEVDF over the job groups it holds and EEVDF over each group's threads (`kernel/pure/sched/queue.rs`). A group weighs its job's weight times its share of the job's runnable threads, refreshed at each decision, so a helper buys no entitlement. Lag goes with whatever leaves: a thread's to its next queue, a dissolving group's with its last thread, clamped to `LAG_BOUND_NS`. A CPU decides again at a request's end, at an earlier-due group's eligibility, and at a wake due first — the last two once the running thread has run `MIN_SLICE_NS`. Every change to a CPU's queues happens in its own pass and every pass arms the timer last, so a remote steal re-arms it as a wake does (the J1 compositor fix). Busy CPUs ask the heaviest CPU for the thread whose move narrows their summed group weight, once per `BALANCE_NS`; idle steals and the push stay. A yield ends the yielder's request at both levels; a running fair thread lent real time leaves the fair band at the lend, its real-time turn starting then. Parameters: `SLICE_NS` 4 ms, `MIN_SLICE_NS` 250 us, `LAG_BOUND_NS` 4 ms, `BALANCE_NS` one slice — the stage-1 model's, under which it met the J1 criteria; `QUANTUM_NS` (10 ms) stays the real-time band's turn. Deleted: `fair.rs` (`FairShare`, `Frontier`, `ShareState`), `SYS_SCHED_INFO` and `SchedInfo`, `PROTECTED_RUN_NS`, `FairOrder`, the PID share table and its removal hook, sched_stress's vruntime-lag check and abuse_page_straddle's `SchedInfo` case (its `ProcessStats` case covers the same copy-out). Simulator. I5 is now equal jobs on a machine holding them alike, against `cpus × 2 × (slice + lag bound + kernel section + 2 chunks)`; I13 is runnable- thread progress, the service a ready thread's CPU owed it while it waited, against the model's `3 × slice + 2 × lag bound` plus two levels of decision lateness; I6 checks each job's runnable count against the bands. Gates: `fair_job_per_thread` and `fair_double_charge` red on I5 alone, `fair_lowest_key_within_group` on I13 alone. `fairness_storm` from 1 to 32 CPUs, 200 seeds each: I5 worst 4/20, 10/40, 14/60, 18/80, 24/120, 26/160, 38/240, 42/320 and 52/480 ms at 1, 2, 3, 4, 6, 8, 12, 16 and 24; I13 worst under 9.3 ms of 24 at every width. The policy suite is restated for the new bounds: a single-threaded rival finishes one slice late at every swarm size from 2 to 64, where it used to fall N+1 quanta behind. `old_commit_before_pass` is caught in 415 of 500 schedules rather than all: the explorer may leave a CPU un-stepped while it holds a task, so a consumer's request can end before it took a step and no block finds the queue empty. Two corpus traces were re-shrunk for the new decisions. Step budgets for the wide workloads follow the measured pass counts. Closed, their mechanism gone with the frozen keys: per-process-fair-split-is-the-policy, granularity-bound-crossed-at-four-widths, i13s-margin-at-32-cpus-is-down-to-two-milliseconds, and interactive-mix-crosses-the-fairness-bound-at-two-cpus (no crossing in 2,000 seeds of either stream with its allowance removed; I5 opens no window there now, and its sleeper's waits are I13's, at worst 8.7 ms of 24). fairness-is-per-thread-so-thread-count-wins is answered by jobs, and the hierarchy it proposed the owner declined. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A launch now carries a job, and the two process_tree builds by hand gave none, which only the guest build compiles. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… exact Review r1 of #979. The lopsided placement cases bounded migrations by the run's length, so a ping-pong fifty times worse still passed. A trace of every move over the 20 seeds per width shows two causes before the first exit, neither a bounce. At start-up, an idle CPU steals from a neighbour that busy balancing has just given a second thread. At the end of the drain, cpu0 answers a burst of requests posted in one instant, drops below a CPU that took threads earlier, and that CPU hands one on. Each other CPU sources at most two moves before an exit. After the first exit, only an exit opens a pair more than one thread apart, the quarter-thread margin's threshold. The bounds are now at most even_drain + 2 * (cpus - 1) moves before an exit and at most one move per exit after it. Measured worst: even + 8 before (8 CPUs, 64 threads), and 53 moves after 64 exits (4 CPUs, 64 threads). Dropping pop_surplus's margin fails it: 21 moves after 8 exits at 2 CPUs. old_commit_before_pass passed only on a 3/4 threshold, with an unverified account of the misses. Counting call-site commits that go on to park splits the 500 schedules exactly. All 415 caught schedules commit at least once. None of the 85 others does: every consumer block there found a token waiting (507 at entry, 2 at the re-check). The gate now asserts that a schedule fails if and only if it commits at the call site, and keeps a reach floor so the workload cannot go quiet. Pointing it at the fused shape, which the harness cannot see, fails at seed 1. Also: the decision comment in preempt_if_due no longer claims a pass's queue changes only bring the decision forward. windows.rs no longer cites the deleted Frontier. The scheduler track's three stage-3 leftovers each get an exit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem
Stage 3 of the scheduler plan (owner decision J1). CPU time was shared out per PID: every process, helper or not, got its own pot, kept against one global vruntime frontier, with ready threads queued at the key their pot held when they were inserted. A build's helper processes each bought a share, and the frozen keys let a process run ahead by one dispatch per runnable thread.
What changes
Jobs. A job is the owner a process's threads and its children share CPU time under (
kernel/pure/sched/job.rs, kernel objectJob).JOB_LABELwithWRITE.JOBSright makes a job (SYS_JOB_CREATE). Names and pids grant nothing.SYS_JOB_OWNanswers a handle to the caller's own job. A launch carries it (or the spawn's endowed job), and the supervisor starts the program in that job. Launched programs are therefore their caller's helpers unless the caller holdsJOBS.system.tomlrow now holdsjobs.Scheduling. Each CPU's fair band runs EEVDF over the job groups it holds, then EEVDF over each group's threads (
kernel/pure/sched/queue.rs), with oneEevdfimplementation at both levels.LAG_BOUND_NS, so no sleeper credit beyond it.MIN_SLICE_NS.BALANCE_NSper CPU, with a margin so floor rounding cannot bounce an odd split. Idle steals and the push are unchanged.Eevdfqueue is now an AVL tree ordered by deadline, each node holding its subtree's least virtual time. Pick, eligibility query and update are each O(log n); before, every operation scanned a vector.Parameters, and why. These are engineering values, the stage-1 model's under which it met the J1 criteria.
SLICE_NSMIN_SLICE_NSLAG_BOUND_NSBALANCE_NSQUANTUM_NS(10 ms) remains the real-time band's turn.Deleted:
fair.rs(FairShare,Frontier,ShareState), the PID share table and its removal hook.SYS_SCHED_INFO/SchedInfo,PROTECTED_RUN_NS,FairOrderand its gates.sched_stress's vruntime-lag check, andabuse_page_straddle'sSchedInfocase (itsProcessStatscase covers the same copy-out).issues/records whose mechanism was the frozen keys or the PID shares. The commit message lists each and why. The audit split the busy-CPU wake record intono-row-holds-a-wake-beside-seven-spinners-to-100-usanda-woken-thread-waits-out-a-sibling-spinners-quantum-when-every-cpu-is-busy. Both now describe the minimum-run cut in place of the protected run, and the second's mechanism, the frozen keys, is gone, which leaves only its measurement owed.PR boundaries. One PR. The job is the scheduler's accounting object, and the authority is how a job comes to exist, so neither is useful without the other. The deletions are required to land together. The tree rewrite of
Eevdfis a separate first commit.Checks
cargo run -- --ci hostat731df67c4(rebased onto maineeffc8b38, after #978 and audit batches #982, #985 and #986)cargo test -p kernel --lib sched::cargo test --release -p kernel-simcargo run -- --build-onlyat28ef305a2cargo test --test toyos-build -- console_image_boots desktop_without_x2apic a_launch_gives_each_unsent_stdio_slot_a_null libc_children vector_state_survives_a_switch the_compositor_drives_the_virtio_gpu a_drives_row_holds_one_process_per_function28ef305a2Simulator. The fairness invariants are restated for the new policy:
cpus × 2 × (slice + lag bound + kernel section + 2 chunks).3 × slice + 2 × lag boundplus decision lateness at both levels.Gates:
fair_job_per_threadandfair_double_chargered on I5 alone;fair_lowest_key_within_groupreds on I13 alone.fairness_storm, 200 seeds per width, worst spread / bound:I13's worst is under 9.3 ms of its 24 ms bound at every width. Before this change I5 crossed its own bound at four of these widths.
The policy suite is restated for the new bounds. A single-threaded process now finishes one slice late against every swarm from 2 to 64 threads; it used to fall N+1 quanta behind.
Negative controls
due(decide only on a recomputed decision;begin's charge renews the request):a_fair_thread_is_decided_again_at_its_armed_request_endreds, exit 101. The simulator found the defect first, as an I9 lend held past its bound.pop_surplusmay hand over the loaded task.a_cpu_does_not_hand_over_the_context_it_is_still_standing_onreds athand_off's assertion.job_fairnessunder QEMU gives the heavy job 0.70, 0.78 and 0.78 of the machine at 2, 4 and 8 vCPUs and fails at each width. Unmutated it measures 0.329–0.338.Evidence for the stage's exit ("two independent builds versus helper-heavy jobs, at multiple CPU widths").
job_fairness(new) starts three jobs with thejobsright:It measures each job's CPU time over 3 s of a saturated machine. Each is owed a third; the tolerance is 0.1 either way.
two_builds_and_a_helper_heavy_job_split_the_machine_in_thirds).110877546: the jobs commit plusprocess_tree'sLaunchfield, before the rebase onto fd: one descriptor table numbers every raw fd, and a child's descriptors are its fd:<n> endowments #978. Thetestcasesboot judgesjob_fairness,wake_onto_busyandwake_storm_cost; thelatencycaseboot judgeslatency_wake.job_fairnesswake_onto_busywake_storm_costlatency_wakeThe T14's
latency_waketail is not this change's. Booted back to back with the base (6f4d8dbdb), head gives p99.9 2466 µs and max 4465 µs, and base 2529 µs and 4527 µs. Judges: T14 head, T14 base, AMD. The first round'sjob_fairnessandlatency_wakegave the same results (T14, AMD).28ef305a2, which has731df67c4's code: on the T14,process_treeandlaunch_toctouPASS on theproctreecaseboot, with boot complete in 182 ms. Run by the orchestrator, and covering the supervisor's launch decode, which now splits off a job handle.latencycaserow completes in 670 ms at base and 671 ms at head, equal stage by stage. The osbench probe's 693 ms against benchcase's 182 ms record comes from the probe's base, whose kernel still enumerated USB (486 ms), which usbd drives every xHCI controller; the kernel drives no USB #971 removed.Logs: index, host checks, simulator, I5/I13 widths, QEMU
job_fairness, its mutant, control: armed instant, control: loaded hand-over, control: no margin, control: blind commit gate. The index states each log's revision.Risks and open items
sshserverstarts its sessions as its own helpers, so all remote logins share its job.CommandExt::endow(JOB_LABEL, …). This waits on the fd work that rewrites spawn.old_commit_before_passis exact again, but reaches the window in 415 of 500 schedules. In the other 85, every consumer block finds a token waiting (507 at entry, 2 at the re-check), so no block commits at the call site. The gate now asserts that a schedule fails if and only if it commits at the call site. A reach floor of three in four keeps the workload from going quiet. Pointing the gate at the fused shape, which the harness cannot see, fails at seed 1.lopsided_placementat 8 CPUs and 64 threads moves up to 113 tasks against 63 stealable. A trace of every move shows causes, not bounces. Before the first exit there are start-up chains (an idle CPU steals from a neighbour busy balancing just gave a second thread) and end-of-drain corrections (cpu0 answers a burst of simultaneous requests and drops below a CPU that then hands one on); after it, at most one move per exit. The test asserts both: at mosteven_drain + 2·(cpus − 1)moves before an exit, and at most one per exit after it. The time-proportional ceiling, which could not see a ping-pong, is gone. Droppingpop_surplus's quarter-thread margin fails the new bound with 21 moves after 8 exits.pop_surplus's own refresh is not exercised by these tests. Removing it passes, becausepick_fairrefreshed earlier in the same pass. It matters on a pass the real-time band picks, which the lopsided workload never has.per-cpu-scheduler-and-pmm-state-shares-lines-every-cpu-writes.🤖 Generated with Claude Code