[Japabu]

Stacked on #1117 (#1111 → #1116 → #1117). Review only the last four commits (hashes after the rebase onto main 8122a4ef5): 48d2f04b5 (the rule), 34c12a7eb (review r1), 7c69daeea (review r2: comments, the issue and the simulator test only), 8a1357642 (ab6: the move stays off the waker's core) and cebbdedd5 (a thread leaving its process names no waker). ab7 measured the same code at 9dcc24ee8, before this rebase.

Problem

A wake goes back to the CPU its task parked on, and spawn placement does not weigh LLCs. So a request/reply pair that starts split across an LLC boundary stays split, and every round trip moves its lines between L3s.

On the AMD bench such a pair reads about 25% slower (ab3 placement table):

• ipc.reqrep.8: 8.4–8.5 µs split against 6.7–6.9 µs in one LLC.
• fs.disk.read_4k_at: 11.6–11.7 µs split against 9.7 µs.

This is the first half of issues/placement-has-no-migration-cost-or-wake-affinity.md's "wake-affine vs spread".

Behaviour

• A wake a thread posts names its CPU (WakeCause::waker, from CpuSched::waker). One that no thread posted names none, and neither does one a thread posts once it has begun leaving its process (thread_exit, process::leave): a dying waker is no partner. Without that, a parent woken by each child's exit, join or teardown closes followed children that were already gone, which ab8 read as work.build_shaped.j16 +4%. The running task carries this as one bit, set once and never cleared. Such a wake is neither counted nor breaks the run.

• The rule (CpuSched::toward_its_waker). The woken task's owner CPU counts how many of its parked wakes in a row came from one LLC. A wake from another LLC whose run is at or past WAKER_RUN = 8 sends the task to a CPU of that LLC:

  • one the task's mask names (sched: a thread runs only on the CPUs its process's mask names #1116);
  • asleep with no kick on its way (Doorbell::sleeping_unrung, the test a wake forward uses), so a burst of moves spreads;
  • off the waker's own core. A request/reply waker has usually parked by the time its wake is placed, so its SMT sibling looks like a free core and comes first in the scan. ab6 measured what a pair on one core costs (below);
  • a free core first, then the quickest to wake, scanned in the waker's LLC alone (CpuHandles::idle_in_llc).

• What follows from it:

  • A split pair shares an LLC after 8 round trips. It ends on two CPUs of that LLC, and ordinary wake placement keeps it there.
  • Only a wake that finds the task parked reaches the rule and is counted. One that finds it running or about to park is a flag the task consumes unseen (park::deliver). So a server with a client in each LLC follows the far client when the near client's requests keep finding it running. In a traced moving seed, all 24 of the near client's requests became flags and every counted wake came from the far one. Once the server has moved, both clients' requests find it parked (71 counted from LLC 0 and 63 from LLC 1 in that seed), and nothing moves it back.
  • When parked wakes come from both LLCs independently and evenly, a run of k moves the server on about 1 in 2^(k+1) − 2 of them (checked by direct simulation).
  • The run is not reset by a move, so a task stolen out of the LLC it was woken from goes back on that LLC's next parked wake.
  • Real-time, stopped and killed tasks keep their own placement. With no such CPU in the waker's LLC, nothing moves.
  • The home CPU makes the move after its own wake, so a move costs two wakes in series.

• Why 8. A split pair pays k split round trips once, about 1.7 µs each on the AMD bench, so 8 costs a pair about 14 µs. Too short a run makes a server bounce: at ten times the rounds (240, 40 seeds), a run of 4 moved the server 4 times in one run, while 8 moved none more than once. Under independent, even parked wakes, k sets the rate: 1 in 6 at 2, 30 at 4, 126 at 6, 510 at 8. The simulator at 24 rounds, 40 seeds:

  k	server moves / most in a run / runs that moved
  1	407 / 45 / 34
  2	60 / 5 / 27
  3	31 / 3 / 24
  4	17 / 1 / 17
  6	10 / 1 / 10
  8, 10, 12	5 / 1 / 5

  The pair moved exactly once in every run for every k up to 12. The 5 moves at 8 to 12 are the flag mechanism above: one client's wakes are never counted, so k does not limit them. They exceed the formula's 3.8 for independent wakes, which does not describe this scenario.

• Owner ruling, 2026-11-01. One u16 of per-task placement state is admitted, kept by the CPU that owns the task. It is the single exception to the 2026-10-28 scheduling model's "no per-task state". It now holds the LLC in ten bits and the run's count in six (WakerStreak), a choice made inside that ruling. issues/placement-has-no-migration-cost-or-wake-affinity.md records both.

• Cost: an Option<CpuId> in the wake message; the u16 in the task; per wake that names a waker, one topology lookup and a count; on a move, one scan of the waker's LLC.

Which wakes name a waker

Every wake reaches park::deliver. Each poster below runs on the poster's own CPU, and none is deferred.

poster	context	names a waker
Watch::post and its boosted form: pipe write, port connect (syscall/ipc.rs, object/port.rs), event signal, console/log watches, quiesce progress	a thread's syscall or kernel path	yes
anything a thread posts from thread_exit or process::leave on: its join, its process's exit notice (publish_exit), its teardown's closes and releases, its unmaps	the leaving thread	no
Watch::post_n: futex wake, sleeplock hand-off	thread	yes
Watch::revoke: futex revoke	thread	yes
IrqWatch::post_in_place from a thread: pmm frames returned, and the inbox, when a client's send completes a server's poll	the poster's thread	yes
IrqWatch::post_in_place from a device interrupt: isa rows, pcidev interrupts and faults, counters answers (kick IPI)	device interrupt	no
console notify	thread, or interrupt	as its context
a deadline's expiry (fire_deadlines) and a retire's wake	the scheduler pass	no

• Poll-delivered wakes need no extra plumbing. A poll completes inside its poster's post, so Inbox::wake runs in the client's syscall and names the client's CPU. The file server waits on its inbox.
• How a device interrupt is told apart: by a per-CPU interrupt depth. On x86, device_irq_entry raises it beside the preempt count; it brackets the kick, ISA, DMA-fault and user-device vectors. The LAPIC timer, TLB IPI, spurious, unclaimed and NMI vectors have their own entries and leave it alone, which is harmless because none of them posts a wake. On aarch64, irq raises it around a device arm.
• Every Watch post names its waker inside the preempt-off section, so a migration between the two cannot name the previous CPU.

Checks

Evidence: pr-1129/r7 at 2a0a47fc (this round), pr-1129/r3 at 3c713712 (review r2: the traces and controls), and pr-1129/r2 at 7009cfc7.

check	revision	result
cargo run -- --ci host	cebbdedd5 (stack top), clean tree	109 steps, all green, exit 0 (log)
a_leaving_threads_wakes_leave_the_task_home	cebbdedd5	pass: 7 counted wakes from cpu1's thread, then 8 after it begins leaving, and the task stays home
QEMU q35, osbench_system on the ab9 head arm (33df293ad; the capability-groups entry pointed at that group, uncommitted)	33df293ad	exit 0; thread.spawn_join's joiner took 1047 wakes and none named a waker, while pipe wakes were named (log)
cargo run -- --ci host	8a1357642 (stack top, on main 8122a4ef5), clean tree	109 steps, all green, exit 0 (log)
cargo run -- --ci host	9dcc24ee8 (on main 8520204a0), clean tree	109 steps, all green, exit 0 (log)
cargo run -- --ci host	b89f78bee, clean tree	108 steps, all green, exit 0 (log)
cargo run -- --ci host	1178599c4, clean tree	108 steps, all green, exit 0
the kernel's library and kernel-loom/kernel-sim suites	1178599c4, inside the host CI run above	green
a_split_pair_moves_once_into_one_llc (AMD shape, 40 seeds)	1178599c4	exactly 1 move in every seed
a_server_with_a_client_in_each_llc_seldom_follows_them (40 seeds)	b89f78bee	at 240 rounds no run moves the server twice (12 runs move it once); at 24 rounds at most 10 in all (5 measured)
host tests: a run of 8 moves; a wake from home breaks the run; masked out stays home; two runs ending in one drain go to two CPUs	1178599c4	pass

The QEMU runs of affinity_pins_hold_on_every_cpu, std_threads_count_… and libc_children at b8bd0ea02 predate this round and are not repeated: the round changes only which wakes move and where, which those tests do not observe.

Controls, each a checked patch, applied, run and reversed in the same run:

mutation	reds
the old predicate (sleeping(), a rung CPU accepted)	the two-run test
no core check	a_move_stays_off_the_wakers_core (the task goes to cpu5, the waker's sibling)
WAKER_RUN = 1 (follow every wake)	the server test: 100 moves in one run at 240 rounds
WAKER_RUN = 4	the server test: 4 moves in one run at 240 rounds
no rule	a_split_pair…, the run test and the two-run test
mask ignored	the mask test and the server test
CpuSched::waker ignores the leaving bit	a_leaving_threads_wakes_leave_the_task_home: the eighth wake moves the task (log)
driver::current_leaves sets nothing (QEMU, same run shape as above)	the joiner's wakes are named again, 1043 of them in its LLC (log)

The host tests are written in terms of WAKER_RUN, so k = 1 leaves them green; the simulator pins its value.

Fan-out, simulator only (uncommitted scenario): one producer on cpu0 of the AMD shape wakes 16 futex waiters for 16 rounds under the shipped balance policy. Over 20 seeds that is 213 migrations with the rule and 182 without, the same most in a run (17). That is about 1.5 more moves per run of 256 wakes, and no sustained ping-pong against the balancer.

Metal

ab5 (AMD, 4+4 benchcase boots at b8bd0ea02, the rule before this round: run of 2, a rung CPU accepted) (judgment). All 8 osbench verdicts PASS. The two I219 reds are T14-only rows that boot:benchcase takes in, as the request said beforehand.

• Placement holds. The ipc.reqrep pair ran in one LLC in 4 of 4 head boots against 2 of 4 base. fs.disk.read_4k_at: 4 of 4 against 2 of 4. The head p50s sit in the one-LLC mode (6.85–6.93 µs, 9.76–9.93 µs), except one boot of each where the pair landed on SMT siblings (8.0 and 11.4 µs). Base has a sibling boot too.
• ipc.reqrep.65536 (IPC phase 2: a connection carries whole messages #1121): base 31.4, 31.7, 51.1 and 52.7 µs, with the two split boots slow; head 32.6–33.1 µs and 41.4 (the sibling boot). The 2× AMD reading is consistent with a split pair.
• Of 144 rows, 20 resolve against about 4 by chance. Better: pipe.throughput.256/4096/65536 2.1–2.5×, wake.storm.drain.w4 −10%, four fs.mem rows −1 to −2%. Worse: wake.storm.call.w1/w4/w64 +6–7%, thread.spawn_join.default +5% and .stack64k +3%, mem.munmap_touched_remote.4k +8%, wake.storm.drain.w1 +3%, and five rows +1–3% (wake.periodic.idle, time.sleep_late.10ms, mem.mprotect.4k, pingpong.pipe.cross_core, fs.mem.mkdir_rmdir). wake.storm.drain.w16's median is +23%, unresolved. The fair rows and wake.onto_busy are unchanged.
• Not explained. The timer-only rows cannot reach the rule, so their shifts are layout (one image per side) or chance. My inference, not established: thread.spawn_join is a joiner following its dead children's LLCs under a run of 2. wake.storm's waiters, all woken by one producer, move into its LLC.

ab6 (AMD, 4+4 at 1178599c4, before the core check) (judgment). osbench PASSes on all 8; the I219 reds are the T14-only rows.

• Every pair is in one LLC on head, but in 5 of 8 it shares a core. ipc.reqrep.8 ran on cpu3+cpu2 and twice on cpu11+cpu10, at 7.9–8.0 µs, against 6.9 µs on two cores; base splits it at 8.4. fs.disk.read_4k_at ran on cpu0+cpu1 and cpu2+cpu3, at 11.7 µs, the same as split, against 9.6–9.8 on two cores. pipe.throughput.16384 read 3.7 GB/s against the base's 8.6, which fits a writer and reader sharing a core. No base pair shared a core.
• Cause: found in the code and reproduced by a host test. busy_siblings counts only awake siblings, and the scan starts at the waker. 9dcc24ee8 keeps the move off the waker's core; without that check the test sends the task to the waker's sibling (cpu5).
• The ab5 slowdowns: wake.storm.call.w1/w4 and both thread.spawn_join rows no longer resolve. wake.storm.call.w64 +5.6%, drain.w64 +2.7% and mem.munmap_touched_remote.4k +10.5% still resolve. pingpong.futex.cross_core, pinned on cpu1 and cpu2, read 491 and 4469 ns in two head boots against 160–171; not explained.
• A head bias: tail.*, timer wakes no change here reaches, read slower on head by 0.7–1.7% in ab5, ab6 and the T14 A/B of sched: a wake its poster parks or exits behind runs where it was posted #1111. In each A/B the base boots first and the images differ, so shifts of 1–3% are not attributable in them.

ab7 (AMD, 4+4 at 9dcc24ee8 in balanced order, with a metal-only diagnostic on both arms) (judgment). osbench PASSes on all 8.

• Request/reply gains, every pair on two cores: ipc.reqrep.8 6.56–6.63 µs against base 7.83–8.00 (base split in 4 of 4), −16%. reqrep.65536 −42%, reqrep.4096 −24%, reqrep_handle.8 −18%. No head pair shares a core.
• pipe.throughput.16384 is still 0.43× (3.4–3.7 against 8.1–8.4 GB/s), but writer and reader never share a core (one_core 0/7 in every boot). One head boot even places its sides as the base does. So the mechanism is unknown. ab5 at a run of 2 read this row at base speed.
• pingpong.futex.cross_core: both arms now read 4.5 µs, the regime where each side parks; head is +1.6%, the bias's size. ab6's 491 and 4469 ns were the row tipping from its no-park regime (about 170 ns) into the parking one. ran= shows no other process on cpu0–3 in any head boot, so it is not interference from moved work.
• tail.* still reads about 1.5% slower on head with balanced order, so the bias is not boot order.
• Also worse: work.build_shaped.j16 +4.2%, fs.disk.seq_read and fs.mem.seq_read −2 to −6% MB/s.

ab8 (AMD, 6 boots: base, head and head with a run of 2, each with a diary-based diagnostic; judgment). osbench PASSes on all 6.

• The pipe slowdown comes from the run length, not from placement or bouncing. pipe.throughput.16384: head 3.6 GB/s, run of 2 8.4–8.5, base 8.2–8.4. Every arm's rounds are bimodal (about 3.5 against 8.4 GB/s). A run of 8 makes the slow mode more frequent. Each round has 0–1 migrations and 2–10 wakes. One run-of-2 boot placed reader and writer on the same CPUs as head and read 8.5. Why the slow mode is more frequent at 8 is not established.
• The rule costs a parallel build about 4% whatever the run length: work.build_shaped.j16 269–270 ms on all four rule boots against 258–260 on base. Inference: the build's parent is woken only by exiting children and follows them.
• The gain holds at either run length: ipc.reqrep.8 6.6–6.7 µs against 8.2–8.3 split.

ab8 refit, from its own rows (fit). Each pipe round's writer is a fresh thread, and spawn placement starts it in the same rotation in every arm (cpu13, 14, 15, 0, 1, 2, 3 at 16384). The reader is osbench's main thread, wherever earlier rows left it. In all 18 ab8 pipe rows (3 sizes × 6 boots), the number of slow rounds that n, min, max and mean imply equals the number of rounds whose writer ran in the other LLC (0–7 against 8–15). The p50 is slow exactly when 4 or more of the 7 rounds were split, and the rows with no split round have min ≈ max. So at 16384 head read slow because its reader sat in LLC 1, where 4 of the 7 rotation slots are across the boundary, against 3 of 7 for base's reader in LLC 0. A round has too few parked wakes (2–10) for a run of 8 to bring the reader over, while a run of 2 did. This is an inference from means; ab9 measures it per round. If it holds, the pipe rows measure the writer's spawn placement and not a cost of the rule, and choosing k on them would tune to the bench's rotation.

ab9 (6 AMD boots, base, head, head, base, base, head, all passed; base f06df08b7 = c9f88d36e + diagnostic, head 33df293ad = cebbdedd5 + the same diagnostic; judged against the rule fixed before staging; judgment).

• Build cost gone: work.build_shaped.j16 head/base 0.976. The parent's wakes on head are still mostly named by a waker that is not leaving, with an unidentified source, so how much the leaving rule earns is not isolated.
• Gain holds: ipc.reqrep.8 head/base 0.856.
• Pipe: every split-LLC round is slow in both arms at every size. The writer's CPU follows spawn placement's rotation whatever LLC the reader sits in. Per class, head matches base. One same-LLC base round read 5.6 GB/s with nothing in its record to explain it, so the mechanism is not confirmed by the rule's letter. On head, a run of far wakes carried across rounds moved the reader beside its writer, which then read fast.
• Blocks: thread.spawn_join +11% (+2.2% without one fast base boot), with non-overlapping ranges, although the joiner's wakes are all unnamed and it never moves on head. fs.disk.seek +4.9%, mem.anon.map +6.2%, mem.munmap_touched_remote.4k +4.9%, wake.storm.call.w64 +3.8% and wake.storm.drain.w4 +8.6% are not explained.

The T14 has one LLC, so the rule never fires there.

Risks

• Not landable on ab9: thread.spawn_join and five rows slower by 3.8–8.6% block it. Each is a row the rule does not demonstrably reach, so their cause (image layout, or a per-post cost of the new waker() path) is the open question. k stays 8.

• The leaving bit is approved as lifecycle state, not placement accounting (orchestrator, 2026-11-02). No existing state already says it. KILL is set only on a thread someone else retires, and process::exit excludes its caller. TaskState::Dead comes only after the exit pass. In the kernel, the process table's thread mark comes after release_thread's unmaps, and TaskHandle.releases is held only during a process teardown.

• Spawn placement starts a new thread across the LLC from its spawner while the spawner's LLC has free cores. That is what the pipe rows measure, if ab9 confirms it. It is sched: a spawn wakes the shallowest of its equal free-core sleepers #1117's domain and is not changed here. If ab9 confirms it, it gets an issue with evidence and a small separate change.

• Spreading a waker's partners for parallelism is not decided. A fan-out whose producer wakes the same workers 8 times in a row moves them into its LLC while sleeping CPUs are there. The simulator shows the balancer and the rule not fighting, but no metal row isolates it.

• No metal row has a server with concurrent clients in both LLCs. That a server follows its far client once, and never back, rests on the simulator alone.

• console's notify from an interrupt is left to the depth. On aarch64 only a device arm raises it, so an interrupt outside a device arm is attributed to the interrupted thread. Those arms post no watch today.

🤖 Generated with Claude Code

[Japabu]

Review r2 of #1129 at 1178599c4

Architecture. The rule lives where the wake is placed (handle_wake on the owner CPU), in the owner-approved u16, and adds no new mechanism. It reuses idle_among (idle_near's body, now shared with idle_in_llc) and sleeping_unrung. The waker is a fact named by the poster's own context, not a hint. The design fits.

Round-1 required findings

• !kick_pending: fixed. The move filters on mask.contains(..) && sleeping_unrung() (kernel/pure/sched/cpu.rs:1216-1218). two_runs_ending_in_one_drain_go_to_two_cpus holds it, and the old-predicate control reds it.
• Scan of the waker's LLC only: fixed. idle_in_llc → idle_among(me, None, Level::Llc, ..) (cpu.rs:2804-2806). The extracted idle_among is unchanged against the old loop body.
• waker() read under preempt-off: fixed. post_as, post_n, revoke and post_in_place evaluate waker() inside preempt_off (kernel/src/watch.rs:90-130), and console::post_wake inside irq_off. The WakeCause::new/.from grep turns up no other poster in kernel/src.
• Issue, comment and body: mostly fixed. One claim is false, below.
• k = 8 justified: not yet. See B1.

Correctness checks

• Encoding. count saturates at MAX_RUN = 63, and 63 << 10 fits in a u16. llc_index is the position of the LLC's first CPU, below MAX_CPUS ≤ 1024. A const assert and a runtime assert fence it, so there is no overflow. There is no reset: the streak travels with the task, as intended.
• Locks and preempt. The rule runs inside the owner's pass, reads only atomics and takes no lock. Two homes moving to the same sleeper at once is a benign doorbell race, and the target adopts both.
• RT and fair bands. RT (including a boost-lent wake), stopped and killed tasks are excluded after they are counted. hand_off keeps its kill check. A move during the hold bypasses held, which is correct because the move leaves the LLC whatever the disposition.
• The sched: a thread runs only on the CPUs its process's mask names #1116 mask. refresh_affinity runs before the rule (cpu.rs:1180), and the candidate filter checks mask.contains. A mask that changes in flight is corrected by the adopter's place_behind relocation. No path places the task outside its mask.
• Interrupt depth. On both arches it brackets the same set: kick, DMA fault, user device and unclaimed. On x86 the add/sub pairs nest inside the preempt count with IF=0, and the deferred-preempt epilogue runs after the decrement.

BLOCKER

B1. The simulator contradicts the k = 8 justification, and the PR explains that away incorrectly.

• Where: kernel/sim/tests/policy.rs:504 with kernel/sim/src/scenarios.rs:1228. The claim is in the PR body ("Why 8": "so it moves the server less than the formula does") and in the commit message of 1178599c4 ("Past 8 the server's 5 moves stay put, so the streak no longer drives them").

• What the measurement shows: the sweep (r16/measure-k.txt) reads 5 server moves at k = 8, 10 and 12 alike. Nothing in server_of_two_llcs except the rule can move the server:

  • balancing is off;
  • both clients are pinned;
  • cpu0 runs only the server, so no wake forward applies;
  • the sim's migrations counts Migrate trace events (kernel/sim/src/vm.rs:1103).

  So in 5 of 40 seeds, 12 or more consecutive parked wakes came from cpu9. The streak does drive those moves; k simply does not limit them. At k = 8 the formula allows at most 48 parked wakes × 40 / 510 ≈ 3.8 moves, so the simulator moves the server more than the formula, not less.

• Likely mechanism (inference): kernel/pure/sched/park.rs:117. Only a wake that finds the task parked reaches the owner with its cause. A wake that finds the server running or committing is a flag and is never counted. When the near client's wakes systematically arrive while the server is still running, they drop out of the count, and the far client alone makes the run.

• Failure scenario: a file server with one client in each LLC, where the local client's requests usually arrive before the server parks. The server follows the remote client whatever k is. After the move the roles may reverse, and if they do the server ping-pongs, paying two serial wakes and a cache refill per move. The 1-in-510 model, the WAKER_RUN doc and the server test's ≤1-per-run bound over only 24 rounds cannot show whether that happens.

• The comment at cpu.rs:1190 ("Every wake that names a waker counts, and only one that finds the task parked reaches here") says the opposite of what matters here. Only parked wakes are counted.

• Cheapest sufficient check (host simulator, seconds):

  1. For one of the 5 moving seeds, print the server's parked-wake waker sequence, and its flagged wakes, before the move.
  2. Run server_of_two_llcs at about 10× WAKE_LLC_ROUNDS and report moves per run.

• Then:

  • If moves stay at ≤1 per run, correct the body, comment and issue: name the mechanism and drop "less than the formula" and "no longer drives them".
  • If the server ping-pongs, the rule needs changing before it lands. The owner's u16 ruling is settled, so a change that stays inside it, or a return through the coordinator, is the implementer's call.

SUGGESTION (optional)

• S1 cpu.rs:1207: there is no reset, and the test is run < WAKER_RUN. So after a move the rule did not make (a steal, or a forward that crossed LLCs), a task whose run from its old LLC is saturated moves back on its first parked wake from that LLC, not on "the wake that makes WAKER_RUN". Scenario: a pair runs in LLC 0 with a run of 63, and a steal from an idle LLC 1 CPU takes the server. The client's next request moves it back at once if LLC 0 now has an unrung sleeper. That is arguably what the rule should do, since the run proves a single source. Either way, the doc at :1187 and WAKER_RUN's doc should say "a run at or past WAKER_RUN".

SEND BACK on B1. The code changes otherwise verify. ab6 is not judged here.