[Japabu]

Records a placement starvation that osbench's wake.onto_busy shows on the T14, routed to the scheduler track (issues/placement-has-no-migration-cost-or-wake-affinity.md) for its diagnosis.

• What happens: at 22.7–29.0 s of the benchcase boot, the span in which netstack's DHCP lease and ready also land, the row's placement onto the CPU holding its parked partners runs for seconds.
• Main: main 544f9cc3c's rows all measured in 8 boots, one of them taking 3.87 s for the whole row.
• kernel: a pipe keeps its bytes in 4 KiB frames it takes as it fills #1074's images: they refused in 3 of 10 boots.
• Causation: whether kernel: a pipe keeps its bytes in 4 KiB frames it takes as it fills #1074 raises the rate is not established (one-sided Fisher exact p = 0.15).
• What the record lists: placement's rotating start, CpuHandles::place's order (least load, then a sleeping CPU, then the fewest busy SMT siblings) and its two readings: the scheduler following its rule, so the bench's staging premise is false, or a CPU that stops answering.
• Exit: a time bound over 20 main boots.

Records only; no code changes.

🤖 Generated with Claude Code

[Japabu]

Review r1 — #1087 at 9bdbb8675 (issue record only)

Assessment. It is right to record this. The slow placement appears on the base (544f9cc3c, session 4 base2), so it predates #1074 and is not a defect #1074 introduced. The record is honest that the cause is unknown. Four parts of the record are wrong, though: the head boot count, the p-value, the independence argument and the exit. The slug and opening sentence also call the wanted CPU busy when it is not. The diff adds 42 lines, one new issue file, and changes no code.

I checked every figure against the readbacks under tcp-connect/{metal,session2..5,metal-diag} and the orchestrator's t14-mconn* logs:

• Refusals: session 2 p1 refused partner 14 after 145,621 threads, session 5 head1 refused partner 33 after 138,904, and diagmerge refused partner 7 after 143,340. In all three, osbench_sched exited 1 and the judge reported FAIL.
• The slow base boot: session 4 base2's row ran from 23.156 to 27.025 s (3.87 s), and netstack's ready came at 24.991 s. These match the record.
• The diagnostic session: it ran one boot per image, so its readbacks were not overwritten.

Findings

BLOCKER 1 — issues/a-spawn-onto-a-busy-cpu-can-miss-it-for-seconds-at-boot.md:23,28 — the head count leaves out a passing head boot.

• The base row's 8 boots are: session 1's metal/run/base-bench, diag-base, and two boots each from sessions 3, 4 and 5. That count therefore includes session 1.
• Session 1 also booted a kernel: a pipe keeps its bytes in 4 KiB frames it takes as it fills #1074 head, 1f828f331 (metal/run/head-bench, according to metal/request.txt). Its row measured in 61 ms (23.605 → 23.666 s). The record does not count it.
• The head family is therefore 3 of 10, not 3 of 9.
• Line 28 says the readbacks are in "sessions 2–5", but the base row needs session 1 and the unnumbered diagnostic session.
• Fix: correct the count to 3 of 10 and name the sessions the counts come from. The commit message and the PR body repeat "nine", so correct those too.

BLOCKER 2 — …:26 — "one-sided p ≈ 0.09" is not reproducible and names no test.

• The usual test for a 2×2 table this small, one-sided Fisher's exact, gives 0.124 for 3/9 against 0/8, and 0.147 for the correct 3/10 against 0/8.
• Boschloo gives 0.074 and 0.070. Barnard with a pooled z gives 0.053 and 0.057. None of these gives 0.09.
• The conclusion, "not established", holds under every one of these tests.
• Fix: name the test and give its value (Fisher one-sided 0.15 for 3/10 against 0/8), or drop the number.

BLOCKER 3 — …:26-28 — "touches no pipe" is true, but it is offered as evidence of independence, and the path it lists is incomplete.

• sdk/std/sys/thread.rs shows that each candidate also calls thread_join and vm_release (reap) and thread_exit. The std channel is futex parking.
• Creating and releasing each candidate's stack takes and frees page-table frames through pmm. kernel: a pipe keeps its bytes in 4 KiB frames it takes as it fills #1074 rewrote exactly that path: free_table_frame became free_held, and allocation goes through alloc_held. Under kernel: a pipe keeps its bytes in 4 KiB frames it takes as it fills #1074, pipes also take the allocator's lock for every 4 KiB frame.
• So the path does reach code kernel: a pipe keeps its bytes in 4 KiB frames it takes as it fills #1074 changed. "No pipe" does not show that kernel: a pipe keeps its bytes in 4 KiB frames it takes as it fills #1074 is irrelevant.
• The independence claim the record can make is already in it: the slow placement occurs on 544f9cc3c.
• Fix: delete the sentence, or list the path completely, including the pmm page-table frames.

BLOCKER 4 — …:41-42 — the exit checks for refusals, but the defect is placements that take seconds.

• The title's defect is a miss "for seconds". The base occurrence took 3.87 s and still "measured".
• Twenty consecutive boots in which wake.onto_busy measures would therefore pass with this defect still present in up to 4.9 s of each row.
• In the 14 rows that measured, the row took 32–63 ms. Session 4 base2's 3.87 s row also measured.
• Fix: bound the confirmation on time. For example, every row in the 20 boots completes within a stated bound, such as 1 s, read from the log's timestamps or reported by the bench. A refusal-only criterion is not enough.

BLOCKER 5 — slug and …:10 — the wanted CPU is not busy while the partners are placed.

• rounds_onto_busy places all 200 partners before it starts the spinner (wake.rs). During the misses, the wanted CPU holds only parked partners.
• "Places partner threads on one busy CPU" and the slug "a spawn onto a busy CPU" therefore misdescribe the condition.
• If the CPU really were busy, CpuHandles::place (kernel/pure/sched/cpu.rs:2635) would avoid it by design. It keys on answering, then load, then not-sleeping, then busy siblings. So the slug's claim is either false or describes intended behaviour.
• issues/README.md treats a slug as a claim.
• Fix: rename the slug to what was observed, for example that a spawn can avoid one idle CPU for seconds at boot. Say that the CPU holds parked partners. Leave "busy" to the bench's name.

BLOCKER 6 — …:13 — the window "22–28 s" ends too early.

• The refused rows ran 23.963–28.967 s, 22.705–27.710 s and 22.802–27.803 s, and the slow base row ran 23.156–27.025 s.
• The window is therefore about 22.7–29.0 s.

SUGGESTION 1 — …:31-37 — the rule and the owner.

• The record quotes only the track's tie-break, "prefers a sleeping CPU". The rule that decides is place's order: CPUs that are answering, then least load, then not sleeping, then busy siblings, from a global rotating start (driver.rs:403).
• On that rule, a CPU that stays awake or loaded is never chosen while another answering CPU sleeps. If netstack ran on the wanted CPU, the scheduler is following its specification. What would then be false is place.rs's STAGE premise ("rotates over the CPUs … about n candidates"), and the work would belong to the bench, or to issues/no-test-can-hold-a-thread-on-a-named-cpu.md, the reason the bench stages at all.
• A CPU that is not answering for seconds would be a kernel defect, and that one does belong to the scheduler.
• Stating these alternatives in the record would make the exit's first step, "name the cause", specific.
• The scheduler track's listed work (migration cost, wake-affine placement, idle-state input, bounded scans) does not include this. Naming it as owner is acceptable as routing for diagnosis only.

SUGGESTION 2 — …:12 — the timing range. "30–60 ms" is slightly low. The rows that measured took 32–63 ms.

PR description (for the orchestrator)

• "3 of 9 boots" should be 3 of 10.
• "p ≈ 0.09" has the same problem as Blocker 2.
• "3.9 s of the 5 s budget" says too much. The budget is per partner, while 3.87 s is the whole row's duration. That a single partner's placement took it is an inference.

Evidence I checked: each benchcase kernel.log (the line before wake.onto_busy, the result or refusal line, netstack's DHCP: lease and ready lines, and TEST_END osbench_sched), verdict.txt, request*.txt, the orchestrator's t14-mconn-diag-{base,head}.log, and session5/judge-head1.log. I ran no tests or builds. The p-values are my own arithmetic from the counts.

SEND BACK

[Japabu]

Review r2 — #1087 at 7e9725876

This round checks only the points the r1 fixes cover.

Resolved

• B1, the counts: these are correct now, 8 base boots and 10 head boots. I checked the revision named for each session against request-session{2..5}.txt, metal/request.txt and request-diag.txt. Session 1's head-bench is now counted. The text says "sessions 1–5", and the commit message and PR body say 3 of 10.
• B2, the p-value: one-sided Fisher exact for 3/10 against 0/8 is 0.147, which matches the record's 0.15.
• B3, the pipe sentence: it is deleted, and the base's slow row is now the evidence that the behaviour predates kernel: a pipe keeps its bytes in 4 KiB frames it takes as it fills #1074.
• B4, the exit: it bounds each row at 250 ms over 20 consecutive main boots. The rows that measured took 32–63 ms, so the bound leaves room above them and still catches a miss of seconds.
• B5, the slug: the file is renamed to a-spawn-can-miss-a-cpu-holding-parked-threads-for-seconds-at-boot.md. The tree cites neither the old slug nor the new one anywhere else.
• B6, the window: it now reads 22.7–29.0 s. The three refusal windows and the slow base window match the logs.
• S2, the timing range: "32–63 ms" is correct.

Findings

BLOCKER 1 — issues/a-spawn-can-miss-a-cpu-holding-parked-threads-for-seconds-at-boot.md:31-32 — the sleeping preference is stated backwards.

• CpuHandles::place (kernel/pure/sched/cpu.rs:2641-2644) chooses with min_by_key on (load, !sleeping, busy_siblings). Because false sorts first, it prefers a sleeping CPU, as the scheduler track also says.
• "then a CPU that is not sleeping" says the opposite. It also contradicts the record's own next bullet, which says an awake wanted CPU is never chosen while another CPU sleeps.
• The last key counts busy SMT siblings, so the CPU with the fewest is preferred. "then busy siblings" does not say that.
• My r1 wording "then not-sleeping" named the key without saying which way it sorts, and that produced this error.
• Fix: say "then a sleeping CPU, then one with the fewest busy SMT siblings".

SUGGESTION — …:30. placement (kernel/src/sched/driver.rs) only picks the rotating start. The keys are applied in CpuHandles::place (kernel/pure/sched/cpu.rs). Naming both would point the reader at the code the rule is read from.

SEND BACK

[Japabu]

Orchestrator: r2's one blocker (the order of CpuHandles::place's keys) is verified fixed at 05da59f. The record now says it prefers a sleeping CPU and then the fewest busy SMT siblings, and names placement and CpuHandles::place. Landing.