[Japabu]

Two defects in ToyOS's std, found by osbench (#983, which is stacked on this).

available_parallelism answered 1

It returned a constant 1 on every machine. The comment claimed no syscall says how many CPUs there are, but SYS_CPU_COUNT does, and libc already answers _SC_NPROCESSORS_ONLN from it. It now returns the kernel's count of CPUs online. This closes issues/std-says-this-machine-has-one-cpu.md.

Every Rust program on ToyOS now sees all of the machine's CPUs, so anything that sizes a pool from std starts more threads than before. Today no in-tree program does: tokio's only in-tree user, system/sshserver, builds a current-thread runtime, and no lockfile carries rayon, crossbeam-deque, num_cpus or threadpool. A future multi-threaded runtime or pool would start one worker per CPU, and each worker takes a 2 MiB stack. On the benches that is 8 on the T14 and 16 on the AMD bench, as std counted them in the osbench metal boots (bench-cpu-counts.txt in the evidence below, from an image that carries this PR's available_parallelism).

No thread std started gave its stack back

• Joined threads. join now frees the 2 MiB stack once the kernel has reaped the thread.
• Detached threads. Dropping a handle without joining detaches the thread. So does every thread::scope thread whose handle is not joined, because scope waits on its own counter, not on native joins. A detached thread now hands its block to a lock-free list as it ends, and the next spawn joins it and frees its stack. This is libc's pthread_detach pattern (sdk/libc/src/pthread.rs).
• Thread-table places. That join also collects the thread from its process's table, so detached threads no longer use up MAX_THREADS.
• Caveat: until a later spawn, an ended detached thread keeps its stack and its table place.

This closes issues/std-leaks-a-thread-stack-per-spawn.md and issues/a-thread-std-detaches-holds-its-place-in-the-process-for-good.md. It also corrects issues/a-processs-memory-is-a-byte-total-that-reads-zero-under-contention.md, which cited the first, and the comment in abuse_tls_alloc.rs that blamed this leak.

Checks

std_threads (tests/guest/rust-tests) holds three things:

• std's CPU count. The QEMU check holds the printed count to the CPUs it booted the guest with, which is independent of the syscall. On a bench, the binary asserts it against SYS_CPU_COUNT.
• 64 threads spawned and joined one at a time are not all on distinct stacks.
• MAX_THREADS + 8 threads with dropped handles, each ended before the next spawn, are all spawned and are not all on distinct stacks.

Stacks that are still allocated cannot share a base. Reuse also depends on dlmalloc returning the mapping and on the kernel placing the next one at the same address, so a future mmap randomisation would turn this red without any std defect.

check, head 3a5fc40	result
cargo test --test toyos-build -- std_threads_count (QEMU q35, TCG, 2 vCPUs)	0: 2 CPUs; 64 joined threads ran on 1 stacks; 4104 detached threads were all spawned and ran on 3 stacks
the same with sdk/std/sys/thread.rs as on main	1: available_parallelism answers 1, and the kernel counts 2 CPUs
the same with only the stack free removed	1: 64 threads spawned and joined one at a time had 64 distinct stacks
the same with only the detach removed	1: detached thread 923 of 4104 was refused: thread stack allocation failed
cargo run -- --ci host	0, 87 steps green
shared metal boot, T14 and AMD bench	staged; the orchestrator runs it

Each control was a patch applied to the fixed file and restored, and the file was compared byte-for-byte with the fixed copy afterwards. Logs and the control patches: https://github.com/ToyOSOrg/evidence/tree/a64c9f71b772a9b5388f7c70656a3999a78ed10a/std/r2.

🤖 Generated with Claude Code

[Japabu]

Review r1 of #987 at f866fe3

I read the commit, the whole diff, sdk/std/sys/thread.rs in full, the kernel's SYS_CPU_COUNT, SYS_THREAD_JOIN and thread_exit paths, the fork's sys/thread/mod.rs and thread/scoped.rs at the pinned rust commit, libc's pthread.rs, the harness registration and the evidence at ToyOSOrg/evidence@66061a9. I ran no build or test.

Architecture

Both fixes are in the right place. std's ToyOS backend (sdk/std/sys/thread.rs) owns the stack it allocates and now frees it, and it asks the kernel for the one machine fact that it can't derive itself. No kernel or ABI change was needed. The production change is +9 net lines. The tests add 32 lines for the guest binary and 28 for the harness. The issue records go from 69 lines to 30.

• The CPU count is correct. smp::cpu_count() is ROSTER.count(), the number of CPUs committed online (kernel/src/smp.rs:47). It is not the MADT capacity. Every AP is committed in start_other_cpus (kernel/src/main.rs:442) before the supervisor is spawned (:462). There is no hotplug, so no user process can see a count that grows later. An AP that fails to start stays offline and is excluded, which is the right answer for "online" and matches libc's _SC_NPROCESSORS_ONLN. There is no affinity or quota mechanism to honour. The Err arm can't be reached while the BSP is counted.
• Freeing the stack on join is safe. The trampoline runs the TLS destructors and rt::thread_cleanup on the stack and then enters SYS_THREAD_EXIT. From there, release_thread and leave run on the kernel stack and never touch the user stack range. The range is not one of the mappings release_thread_mappings releases: the kernel only records user_stack_base for SYS_STACK_INFO. sys_thread_join returns 0 only after collect_zombie, which comes after leave. So dealloc can't race the thread's last user-mode use of its stack. The joiner's munmap shootdown covers the exiting CPU. On Gone (a cancelled join) or NotFound, the stack is kept, which is the right way to fail. std's generic join then hits its own unwrap on the packet, but that predates this PR.
• Fork pins. rust is unchanged (11e31a5 at both the merge base and HEAD). sdk/std reaches the fork through sys/thread/mod.rs:118-121, so no fork commit is involved. main has since moved rust to a56b405 (fd: one descriptor table numbers every raw fd, and a child's descriptors are its fd:<n> endowments #978). GitHub reports the PR as MERGEABLE/CLEAN.
• Effect on pools. The PR body states it. In tree, nothing over-allocates. The only tokio user, system/sshserver, builds new_current_thread() and does not enable rt-multi-thread. No lockfile carries rayon, crossbeam-deque, num_cpus or threadpool. The only consumers of available_parallelism are guest tests and host-side build code. The sentence "Shipped programs built on tokio's default runtime will start more threads" describes no program in the tree today (see the PR-description notes).

Findings

BLOCKER issues/std-leaks-the-stack-of-every-thread-it-detaches.md:17-30: the record's account of the detached case is false.

• It says the answer is "a design question: the kernel reclaiming a thread stack it already knows, or std refusing to detach … and not a missing line in std". ToyOS already solves this exact case in userland. libc's pthread.rs has a detached thread hand its block to REAPED as it exits (sdk/libc/src/pthread.rs:13-15, :116-118), and the next pthread_create joins it and frees its stack (:130-132). The record this one cross-links, issues/a-thread-std-detaches-holds-its-place-in-the-process-for-good.md, cites that mechanism as the thing std lacks. The same std-side change would close both records with no kernel change, so the record must name it as the exit and stop presenting this as a kernel design choice.
• The record also leaves out the most common detaching path. thread::scope waits on its running-thread counter, not on native joins (fork library/std/src/thread/scoped.rs:42-68, :141-149). Every scoped thread whose handle is not explicitly joined therefore drops its Thread and leaks its stack. The record should say so, because that is what makes the leak routine and not exotic.
• "It cannot be closed without a plateau to measure" (:28-30) contradicts this PR. The PR closes the joined half with a stack-base-reuse oracle, and that oracle applies to a detached thread just as well (spawn and drop, let it end, spawn again, compare bases). Either the plateau requirement goes, or the PR has to explain why reuse is insufficient for one half and sufficient for the other.
• kernel/src/process.rs:565, :569-570 are stale; the field is at :634-635 and the store at :967-968. This file is newly written in this PR, so it owns these references.
• issues/a-processs-memory-is-a-byte-total-that-reads-zero-under-contention.md:9-12 must then stop calling this record its "immediate consumer". Its :55-58 ("the record's exit condition currently closes only the joined path. The plateau must cover both.") also needs correcting.

BLOCKER tests/guest/rust-tests/src/bin/abuse_tls_alloc.rs:57-59: this comment is now false. It says std's thread::spawn "leaks its 2 MiB stack on every spawn (… join never frees it)". After this PR, join does free it. The PR deleted the equivalent stale comment in demand_window_race.rs but missed this one, and issues/a-processs-memory-…:41-43 cites it. Delete the comment, or reduce it to whatever reason still holds for using a raw thread.

BLOCKER (evidence; the orchestrator runs it): std_threads is a discovered binary, so it now joins both benches' testcases shared boots (tests/toyos.rs:1477, DRIVEN_AND_SHARED). The PR and the registration comment call that shared run "each bench's verdict". The evidence has only the QEMU runs and --ci host; there is no --metal-readback preparation and no metal run. Before landing, run the shared boots once on the T14 and the AMD bench, or show why an existing run already covers this member. That is the declared target for the binary.

SUGGESTION Implement the detached half now, mirroring libc's reap. That means a Drop on Thread that marks the thread detached, an exit-time push onto a reap list, and the next spawn joining and freeing what is on that list. It is roughly libc's 30 lines. It closes this record and a-thread-std-detaches-holds-its-place-in-the-process-for-good.md together, and std_threads can hold it with the same base-reuse oracle using dropped handles. Whether to widen the scope is the orchestrator's decision. If it stays out, the first blocker's record correction is what is required.

SUGGESTION tests/guest/rust-tests/src/bin/std_threads.rs:17-18: "a freed one is the next spawn's to reuse" overstates the guarantee. Reuse depends on dlmalloc munmapping the mmapped chunk and on the kernel's mmap placing the next one at the same address. The negative direction is sound: live stacks can't share a base. But a future mmap placement change, such as randomisation, would turn the test red with no std defect. Say "can be", and name that dependency in one clause.

SUGGESTION The CPU half compares std with the same syscall that std itself calls. That is enough to catch a regression to a constant, but not a kernel miscount (capacity versus online). The harness already knows the guest has two vCPUs. Asserting the printed count against that number would make the oracle independent at no extra cost.

SUGGESTION tests/guest/rust-tests/src/bin/std_threading.rs:17-20 asserts available_parallelism() > 0, which NonZero already guarantees by type. It is now superseded by std_threads. Delete it.

Tests

QEMU is the cheapest place for the stack claim, because sdk/std has no host build. The registration comment says so, and std's actual join path is what has to be exercised. On its own the CPU half would be settled by inspection, but here it runs in the same boot at no extra cost, and it answers the old record's point that nothing would catch a regression. Both controls are meaningful:

• With the whole change reverted (control-whole.patch applied in reverse), the run fails at std_threads.rs:15.
• With only the free removed (control-join.patch), it fails at :26.
• The fixed run prints 2 CPUs; 64 joined threads ran on 1 stacks and exits 0.
• --ci host at f866fe3 is green (87 steps).

The evidence README names the revision and states that the tree was restored after the controls. Only x86-64 under QEMU was run. That is acceptable here: the code involved is architecture-neutral std code and a generic kernel counter.

PR description (for the orchestrator)

• No in-tree shipped program uses tokio's multi-thread runtime. sshserver is current_thread, and the workspace doesn't enable rt-multi-thread. Say that explicitly so the warning about thread growth isn't read as describing current programs.
• "A detached thread still keeps its stack" should add that every unjoined thread::scope thread is detached.
• "8 on the T14 and 16 on the AMD bench" is not in the evidence. Cite a boot log or label it as recalled.

SEND BACK

[Japabu]

Metal, head 3a5fc40e8, run by the orchestrator, shared testcases boot:

• T14 (ThinkPad T14 Gen 2): rc 0, std_threads PASS, "8 CPUs".
• AMD (Lenovo 82A2): rc 0, std_threads PASS, "16 CPUs".

[Japabu]

---

Review r2 of #987 at 3a5fc40 (rebased on eeffc8b)

I read the commit's whole diff, sdk/std/sys/thread.rs in full, the kernel join and exit paths again, and the r2 evidence at ToyOSOrg/evidence@a64c9f7/std/r2. The control patches apply to the committed blob (e18a7d405). I ran no build or test.

r1 blockers

• The issue records: resolved. Both detached-thread records are closed by the fix, not reworded. The memory record no longer claims a consumer and no longer demands a plateau for the detached half. No reference to the three deleted slugs remains outside issues/.
• abuse_tls_alloc.rs:57-59: resolved. The comment now says something true: the test reuses one mapped stack so the address space holds still.
• Metal: still open, and the orchestrator is running it. It stays required until both benches' shared boots report.

The optional suggestions were all taken. The harness now holds the printed count against BootOptions::default().smp, which is an independent oracle. The test comment states its dependence on mmap placement. The > 0 check is gone from std_threading.rs. bench-cpu-counts.txt supports the PR body's 8 and 16.

The reap list

• Lock-free correctness and ABA: sound. Producers only push, with a CAS on the head. The consumer takes the whole list with swap(null) and never pops one node with a CAS. ABA arises when a pop compares against a stale head->next, and there is no such pop here. If a push sees head A, and A is reaped, freed, reallocated and pushed back as the head, the CAS still links the new block to the actual current head. Release on the push and Acquire on the swap make next, tid, stack and layout visible to the reaper. Concurrent spawns each take a disjoint list.

• One reaper per block.

  • join reaps under ManuallyDrop and never touches state. The thread's swap then sees JOINABLE and does not push.
  • Drop CASes JOINABLE to DETACHED. If the thread swapped EXITED first, the CAS fails and Drop reaps. Otherwise the thread sees DETACHED and pushes.
  • Only Drop writes DETACHED, and only the thread writes EXITED, so exactly one party reaps.

• The stack is freed only after the thread has left it. The thread's last user-mode actions are the TLS destructors, thread_cleanup, the state swap, the push and the call into thread_exit. The block is not touched after the push succeeds. Every reaper frees only after thread_join returns 0. That happens only after collect_zombie, which follows leave in process::thread_exit (kernel/src/process.rs:1250-1251), and by then the thread runs on its kernel stack. A reaper that takes a block which was pushed but has not yet exited simply blocks in the join until it has. The wait is bounded: the thread holds no user lock and runs no user code after the push. A refused join (Gone, NotFound) frees nothing, which is correct.

• The thread-ends/next-spawn race: benign. A detached thread that ends after a spawn's drain (or between its EXITED swap and its push) is collected by the following spawn. The thread-table place and the stack that are held over are bounded by the threads that ended since the last spawn. The doc comment at :14-15 states this.

• MAX_THREADS accounting is now correct. The ABI counts exited-but-unjoined threads against MAX_THREADS (sdk/abi/src/syscall.rs:1019-1021). Each path ends in thread_join, which collects the zombie:

  • a joined thread, in join;
  • a handle dropped after its thread ended, in Drop;
  • a handle dropped before its thread ended, in the next spawn's drain, before the spawn syscall.

  So no spawn is refused for an ended std thread that had already pushed when the spawn drained the list. The QEMU pass spawns 4104 detached threads, more than MAX_THREADS = 4096, and none is refused.

  • Gap, not a blocker: the detach control fails at 923 on stack OOM before it reaches the table limit, so no negative control shows the slot half on its own. A control that freed stacks without joining would be unsound, and the pass above MAX_THREADS together with the ABI's documented zombie counting is enough.

• The wording of the closed exit, "collected once it exits". Literally, the thread is collected at the next spawn after it exits. I accept the close. A thread-table place only matters to a spawn, and every spawn collects first. The record's stated test (more than MAX_THREADS dropped handles, none refused) now passes. The residual stack hold until the next spawn is deferred release, the same as libc's, and it is documented at the site.

Findings

BLOCKER sdk/std/sys/thread.rs:27-28, :55-58, :114-122: the published handshake is dead machinery copied from libc, and its comment is false.

• libc needs it because a pthread can detach itself before its creator has written the tid (sdk/libc/src/pthread.rs:9-11).
• A std thread cannot detach itself. A block reaches a reaper only through the handle that Thread::new returns after storing tid, either by join or Drop on that handle or by a push that follows Drop's CAS. The chain is: the tid store, then the handle transfer, then Drop's AcqRel CAS, then the thread's AcqRel swap, then the Release push, then the Acquire swap. This already orders the tid store before every reaper's read.
• So the comment "once published, the thread may end detached and be reaped, freeing the block" (:114-115) describes something that cannot happen. The futex wait in reap never waits, and the futex_wake costs one syscall on every spawn for nothing.
• Fix: delete published, the wait loop, the futex_wake and both comments, and store tid before returning Thread { block }.

SUGGESTION In issues/a-processs-memory-is-a-byte-total-that-reads-zero-under-contention.md:11-13, drop "which no other leak offers". It is an unsupported generalisation, and the sentence works without it.

Tests

The three controls are the right ones and red on the right assertions:

• the whole file as on main fails at :23 (answers 1);
• the free removed fails at :31 (64 distinct stacks);
• the detach removed fails at :49 (refused at 923).

The fixed run prints 2 CPUs and 64 joined threads ran on 1 stacks; 4104 detached threads … ran on 3 stacks, and exits 0. --ci host is green. Deleting published changes no behaviour the test observes. After that edit, compiling and rerunning the QEMU check is enough; the controls don't need repeating.

SEND BACK. The published deletion is the remaining code item, and the metal results are the remaining evidence.