Repository navigation
Conversation
Review r1 of #1150 at
|
…fused (S0-a) A user copy reaches its pages under its space's lock, populating any not yet touched, and in memory stage 7b that fault-in will park on a userland pager. A copy made under any spin lock would then park holding it. Several syscalls copied that way: reads and writes under the process lock and the object's (a pipe's, a kernel file's cache, a console's line, a device claim's description), message send and receive under the process and queue locks, and getcwd, query_modules, sysinfo and readlink under the process, process-table or VFS lock. The gate. Every copy entry (View::window, View::pieces, translate_user) refuses a copy above the preemption depth its syscall's entry runs at, whether or not a page needs populating: every spin lock raises the count, and both architectures' trap entries raise it to BASELINE_TRAP (x86_64's syscall_entry and common_entry, AArch64's trap::syscall and trap::user_abort). It is a kernel-bug fence; no user action reaches it. This replaces #1146's per-space holder, which caught only a copy under its own space's lock. The moves. - Syscalls work on their object cloned out of the process's lock (syscall::handles::object, and with_object now clones). A device claim is read under it, since describing one installs handles, into a DeviceRead copied out after. - getcwd clones the cwd, readlink lets the VFS lock go first, and query_modules and sysinfo lay their answer out under their lock and copy it out after. - A kernel file's position and a console's line are sleeping locks (SleepLock), which raise nothing, so a read or a write stays atomic with its position update (XSH 2.9.7) and a console write keeps its lines whole. file_cache reads copy from the page's Arc with the cache's lock let go (a write meanwhile writes a copy), and a write's bytes are copied in a page at a time before the cache's lock. ByteSource had no other user and goes. - Pipes and connections claim their bytes (kernel::pipe::Bytes): one write claim and one read claim at a time, each copying with no lock held. The lock claims, hands out a chunk's frames and ends the claim. A claimed write's frames count as used, so no surplus takes them, and only the read's own consume lets claimed bytes go; a last reader leaving under a claim leaves the frames to the claim's end. A second copy of either kind waits for the first on its end's watch under a TURN token, cancellable by a kill and waking no poll. Frames and a longer frame list are taken before the lock and spares given back after it (frame_bytes::supply, give_back), so neither the allocator nor the heap is reached under it. - A receive claims the front message, copies its bytes out, then under the process and queue locks pops it and installs its handles, with the table's room made first. A claim dropped first leaves the message queued whole, so a refusal still takes nothing. A send claims its room, copies in, then moves its handles and queues the message whole in one hold; a receiver gone by then hands the batch back at its own numbers. The console's partial line is now flushed in Drop, at the last reference: the deferred release queue runs on_zero_handles where no sleeping lock is taken, and a write in flight holds a reference. Records: the finding a-user-copy-demand-pages-under-whatever-its-caller-holds is closed (its order is refused by a check that fails). The pipe issue keeps what is left: the hold time unread on a bench, the queue's two lists grown under its lock, and audio_tone's worst wake. The process-memory track records the orchestrator's ruling of 2026-11-01 (on the owner's delegation) that 7b comes before stage 4, its S0-a prerequisite met, and that a copy under a sleeping lock parks holding it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… abandoned write's room announced A copy that found its end's claim held asked to be woken once, before its wait; the wait's re-checks did not ask again. A claim made between a wake and the re-check then ended without waking it. The turn now lives in `kernel::pipe::Bytes`: `turn(end)` asks to be woken every time it finds the claim held, and each claim's end answers `Ended`. `frame_bytes::turn` is the one wait, shared by pipes and connection directions, and `frame_bytes::wake` the one post. A claim is made only in its turn, so `claim_read` and `claim_write` no longer answer a refusal. An abandoned write gives back room that a wait for room or a writable poll found taken: `Ended::room` posts the end's watch for every waiter (a pipe write refused for memory; a send refused for memory, for a receiver gone or for a handle a sibling closed). The rest of the review: - `frame_bytes::supply` hands an unused longer list out of the hold, so no `pmm::uncharge` runs under the object's lock. - A real-time sender's lent window is taken with the message, not at the claim, so a receive refused for table room leaves it for the next. - A file's id and volume are fixed at its open and its mtime is atomic, so `pread`, `pwrite`, `fstat` and `ftruncate` take no lock; the sleeping lock holds the position alone. - A pipe write looks its handle up again after each wait for room, as it did before the claims, so a sibling's close is answered rather than a write landing past its reader's EOF. - The S0-a records say only its no-copy-under-a-spin-lock half is met. - The fsync comment that said it parks goes; the file row's comment says where a file's read or write parks; a test's doc comment is back on it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Japabu
changed the title
…in one hold A claim costs a small transfer at least two holds of the object's lock where the copy under the lock took one, and on the T14 that cost small pipe writes 28-30% and request-reply IPC 12-26%. A transfer of at most `BOUNCE` (4096) bytes is now copied between its caller and a `Bounce` on the stack with no lock held. Between the bounce and the frames the copy is kernel to kernel, in one hold: - a pipe write puts its bytes whole (`Bytes::put`), unless it would want frames, which it takes with the lock let go as a claim does; - a pipe read takes its bytes whole (`Bytes::take`) and copies them out after; - a send's bytes are copied in before any lock, and the message is queued whole as its handles move, in the sender's process hold, as before the claims; - a receive pops the front message and installs its handles in the hold that looks the connection up, where the table has room without growing (`HandleTable::has_room`), and copies its bytes out after. Larger transfers, a transfer that meets a claim in flight, and a receive whose table must grow keep the claim path, which kept their gain (large pipe writes +71-83% on the T14). The bounce is uninitialized and filled through `read_raw`, so a 64-byte write zeroes no 4 KiB. A put or a take owes no wake: a copy asks to be woken only while a claim is held, and none is held across either. The track records the orchestrator's 2026-11-03 ruling: 7b needs the met half of S0-a and, after #1152, the gate extended to sleeping locks; the rest of S0-a is built as part of 7b. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A pipe read or message receive of at most `BOUNCE` bytes lets its bytes go, and a receive installs its handles, before copying out. A lost page met in that copy, which 7b answers, must not lose them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…othing it can borrow `Bounce::read` answered the bounce by value, and the compiler moved it into the caller's place: every bounced pipe write and send copied all 4104 bytes of it, whatever it held (a `memcpy` of 0x1008 bytes in `PipeRef::write` and `sys_message_send`). The bounce is now filled where it lies. The wake after a pipe read or write borrows the pipe from the object the call already holds, rather than cloning it, and a bounced receive keeps no reference to its queue: the spares it took free themselves, and any past their batch stay listed for the next take. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`lock_word` slept in `FUTEX_WAIT` at the first collision. The descriptor table's lock, taken twice by every descriptor call, is held for a few hundred nanoseconds; a sleep and its wake cost a kernel round trip each way. In #1150's metal A/B the pipe benchmark's two threads, no longer serialized by the kernel's process lock, met on it 128,000-153,000 times a run where main met it about 150 times, and that was much of the small-pipe-write regression. A contender now looks again up to 100 times while the word says held with no sleeper, as competitive spinning has it, and sleeps behind any sleeper. The lock's sleep and wake go through the module's `wait`/`wake_one`, so a host test contends for it. The process-wide lock itself is recorded in `issues/every-descriptor-call-takes-one-process-wide-lock-twice.md`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… comment names whose lock The eleven sightings all fall between 2026-10-04 and 2026-10-10; the 2026-11-02 reading of #1150's scratch kernel was not one of them. The copy's comment names the readers that want the lock: the caller's own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A write of at most `BOUNCE` bytes is copied into the stack before any lock; then one hold of the process's lock looks the handle up and, for a pipe's write end, puts the bytes in under the pipe's lock (`PipeRef::put_now`). A read takes at most a bounce's worth in the same way (`PipeRef::take_now`) and copies it out after. That is the lock shape a pipe call had before any copy left the locks, with no user copy inside it: a process lock and the pipe's, nested, once each. Anything that hold cannot answer at once falls to the general path: a claim in flight, a write that wants frames or room, an empty pipe or more than a bounce to read, and any other object. A small write to another object is copied in twice, which ends with #1152 and #1165 for the kernel's files and console. `Bytes::claim_held` asks after a claim without asking to be woken, since these never wait. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Memory track S0-a, its no-copy-under-a-spin-lock half: no user copy runs under any kernel spin lock, and one that would is refused by name. The rest of S0-a's row (a chunk faulted in before any lock and pinned after a re-check, the lost-page outcomes, its H model) is not built and stays open. This half is what memory stage 7b needs from S0-a; whether 7b also waits for the rest is not yet ruled. It matters to 7b because there, a copy's fault-in will park on a userland pager, and a copy made under a spin lock would park holding it. Folds and closes #1146, keeping its gate and dropping its per-space holder.
Problem
A user copy reaches its pages under its space's lock, populating any it has not touched (
kernel/src/user_ptr.rs). Several syscalls copied while holding a spin lock:read/write(blocking and not) under the process lock and then the object's lock: a pipe's, a kernel file's cache, a console's line, a device claim's description;getcwd,query_modules(process lock),sysinfo(process table) andreadlink(VFS lock; the scout missed this one, a static audit found it).Now
The gate. Each copy entry (
View::window,View::pieces,translate_user) panics, naming the depth, when the preemption count is above the depth its syscall's entry runs at, whether or not a page needs populating. Every spin lock raises the count, and both architectures' trap entries raise it toscheduler::BASELINE_TRAP= 1: x86_64'ssyscall_entryandcommon_entry, AArch64'strap::syscallandtrap::user_abort. The WIP'scount() != 0was wrong, since every syscall runs at 1. A sleeping lock (sleeplock::SleepLock) raises nothing, so a copy may run under one. The gate is a kernel-bug fence: no user action reaches it once the copies below moved.The moves.
Objects are cloned out of the process's lock before any copy (
syscall::handles::object;with_objectnow clones). A device claim is read under it, since describing one installs handles, into aDeviceReadthat is copied out after.Laid out, then copied:
getcwdclones the cwd,readlinklets the VFS lock go first, andquery_modulesandsysinfolay their answers out under their locks.Sleeping locks where atomicity is the contract. A kernel file's position and a console's line are now
SleepLocks, so a read or write stays atomic with its position update (POSIX XSH 2.9.7), and a console write's lines stay whole. A file's id and volume are fixed at its open and its mtime is atomic, sopread,pwrite,fstatandftruncatetake no lock. (Consolidate storage around native bcachefs and userland filesystems #1152 deletes the kernel file cache and lands first; these parts go at the rebase.)file_cachereads copy from the page'sArcwith the cache lock let go (a concurrent write writes a copy), and writes bounce a page at a time before the cache lock.ByteSourcehad no other user and is deleted.Pipes and connections claim their bytes (
kernel::pipe::Bytes, host-modelled). One write claim and one read claim at a time; each copies with no lock held. The lock does only three things: claim, hand out a chunk's frames (≤ 17 runs; the first chunk's in the claim's own hold), and end the claim.consumelets claimed bytes go. A read that empties the pipe beside a write claim does not reset the start under it. A last reader leaving during a claim leaves the frames for the claim's end to give back.frame_bytes::turn,frame_bytes::wake). A second copy of either kind waits for the first on its end's watch, under aTURNtoken. The wait is cancellable by a kill and wakes no poll.Bytes::turn(end)asks to be woken every time it finds the claim held, under the lock its wait re-checks in, so a claim made between a wake and its re-check still wakes it as it ends (review r1's lost wakeup). Each claim's end answers a#[must_use]Ended.space(), so a wait for room or a writable poll parks. A write abandoned (a pipe write or a send refused for memory, a send whose receiver left or whose handle a sibling closed) answersEnded::room, which posts the end's watch for every waiter.frame_bytes::supply,give_back). So neither the frame allocator nor the heap is reached under a pipe's lock; a longer list a concurrent read made unnecessary is handed out of the hold too. A claimed transfer takes two holds or more in steady state; a bounced one takes one.A transfer of at most 4 KiB bounces through its stack in one hold (
frame_bytes::BOUNCE,Bounce). Round 1's metal showed the claims' extra lock holds cost small pipe writes 28–30% and request-reply IPC 12–26% on the T14. A small transfer is now copied between its caller and an uninitialized stack buffer with no lock held. Between the buffer and the frames the copy is kernel to kernel, in one hold:Bytes::put, unless it would want frames, which it takes outside the lock as a claim does.Bytes::take, then the copy out.HandleTable::has_room). The bytes are copied out after.A large receive keeps "a refusal takes nothing". It claims the front message (queue lock), copies its bytes out (no lock), then under the process and queue locks pops it and installs its handles. The table's room is made before the queue lock. A real-time sender's lent window is taken with the message, so a receive refused for table room leaves it for the next. A claim dropped before its take leaves the message queued whole. That covers no table room now and, in 7b, a lost file page answered
EIO. The one visible change: handles are installed after the bytes land. A send claims room, copies in, then moves its handles and queues the message whole in one hold; a receiver gone by then hands the batch back at its own numbers.Console: the partial line is flushed in
Drop, at the last reference. console: the VM's console is a userland virtio-serial driver's, and the UART goes quiet while it carries the log #1062 made the console's release deferred, soon_zero_handlesruns from the release queue, where no sleeping lock may be taken. A write in flight holds a reference, so the line is free by then.A bounce is filled in place, and a small call clones only what it must. A bounce returned by value had been moved, with all 4104 bytes copied, on every bounced pipe write and send. The wake after a pipe call borrows the object's pipe, and a bounced receive keeps no queue reference.
The SDK's lock spins briefly before it sleeps (
toyos::sync::lock_word, 100 looks while held with no sleeper, as competitive spinning has it). Every descriptor call takes the process's descriptor-table lock twice. With the kernel no longer serializing a process's pipe calls under its process lock, the pipe benchmark's two threads met on it, and each meeting cost a futex sleep and wake. Recorded:issues/every-descriptor-call-takes-one-process-wide-lock-twice.md.A small pipe call is made in the hold that looks its handle up (
PipeRef::put_now,take_now). A write of at most 4 KiB is copied into the stack first. One hold of the process lock then finds the pipe and puts the bytes under its nested lock; a small read takes its bytes the same way and copies them out after. That is the lock shape base had, with no user copy inside it. A claim in flight, a want of frames or room, an empty pipe, or any other object goes to the general path, which copies a small non-pipe write in twice until Consolidate storage around native bcachefs and userland filesystems #1152 and kernel: no kernel thread but the idle loop; the console UART is a userland driver's (K4, K6) #1165 remove the kernel's files and console.Semantics a reviewer should know.
readkeeps its cloned object across its park, as Linux'sfdgetdoes. A pipewritelooks its handle up again after each wait for room, as before this PR, so a sibling's close of the writer's last handle is answered rather than the write landing past the reader's EOF. One window is left: a write already past its lookup when that close runs lands its bytes after the reader may have seen EOF. It is bounded by one claim wait and one copy. POSIX leaves a close concurrent with another thread's call on the descriptor to the application to avoid (my reading, not verified against the text); Linux would complete the write before EOF.Records
issues/a-user-copy-demand-pages-under-whatever-its-caller-holds.md: its order is refused by a check that fails.issues/a-file-write-beside-a-read-of-its-page-can-be-refused-for-memory.md. A write that finds its page shared with a reader in flight copies it under the cache lock and can failExhausted. Consolidate storage around native bcachefs and userland filesystems #1152 removes the file cache, and this issue with it.issues/a-pipes-lock-spans-a-user-copy-and-its-first-allocation.mdkeeps what is left: the hold time is unread on a bench (QEMU's clock cannot read it), the connection queue's twoVecDeques still grow under its lock (reserving their charged bound at creation is the fix, at that memory per direction: an owner tradeoff), andaudio_tone's worst wake is unread.Checks
Round 5 (head
05f6c884f; evidence https://github.com/ToyOSOrg/evidence/tree/350a3bd22d010685d4bb15c0bbd2fb533273c778/r5):r4/ab3-table.txt; base a0e8f13, head5572386b8):FUTEX_WAITs a run against base's 94–136.ipc.reqrep.4096+16%,reqrep_handle.8+12%. AMD: pipe 64/1024 −16/−19%, and IPC +15 to +45% (one of each AMD head arm's two boots in a slow mode).std_fsfailure was its DATA volume, overwritten after base boot 1 and independent of this change.cargo run -- --ci hostat05f6c884f: rc=0, 109 green.05f6c884f, per the owner's rule: PENDING (queued). The--debugmembers run on the Mac before the rule gave 230/238 with the same 8SYS_DEBUGmembers, and C 143/143.ab4/), with rules set before any boot. Base and head both carry a measurement-only per-syscall probe (branchess0a-probe-base,s0a-probe; never to land), which reports mean ns per call by syscall number for the IPC residual without a boot of its own. Five boots per bench; the AMD half can run first while the T14 is down.Round 4 (head
5572386b8; evidence https://github.com/ToyOSOrg/evidence/tree/77bd65bd3e59d17b55254b0074f0ef938dd67270/r3):ab2-table.txt). The gate holds on both benches. The hot rows still regress:pipe.throughput.64..4096−8 to −15%;ipc.reqrep.8/4096/reqrep_handle.8+8 to +16%.pipe.throughput.64−35% and1024−24%.FUTEX_WAITcounts for the pipe benchmark's process: main 125–205 a run, head 17,000–161,000 (ab2-futex.txt).memcpyinPipeRef::writeandsys_message_send(disasm-pipe-write-before.txt). The rebuilt kernel has neither.cargo run -- --ci hostat5572386b8: rc=0, 109 green. The newcontenders_for_a_lock_each_hold_it_alonehost test fails under control N8 (a spinner takes a held lock).--debugat5572386b8: 230 of 238, the same 8SYS_DEBUGmembers. C corpus 143 of 143. No gate panic.ab3/), with rules set before any boot. Per the owner's ruling, any hot row that moved worse blocks landing. The futex count is an oracle for the spin. Round 2's threshold comparison was confounded by the bounce's move, so round 3 measures it again.Round 3 evidence (the bounce): https://github.com/ToyOSOrg/evidence/tree/4d8d75f1006b28a727f5d7fa608125fd5583c2a1/r2. Code at
0d0a03065;fa5c9d257adds an issue note, and1321274d7a test lint.whole_transfers_in_one_hold_keep_the_stream_and_the_claimscovers small puts and takes beside a large write claim in flight, the stream byte-exact, frames bounded and accounted. Controls are red on exactly that test and reverted: N6 (putplaces though frames are wanted) and N7 (a refusedputkeeps its claim).cargo clippy -p kernel --bin kernel -- -D warnings: rc=0 on x86_64 and AArch64.--debug(x86_64 QEMU) at0d0a03065: 230 of 238, the same 8 needingSYS_DEBUG. The C corpus: 143 of 143. No gate panic.qbench-table.txt). One red, head's osbench stalled afteripc.reqrep.4096at load 62, which the harness classed as its liveness ceiling. It did not reproduce: 3 of 3 head runs at load 21–25 completed. The phase that stalled runs the unchanged claim path.cargo run -- --ci hostat1321274d7gave rc=0, 109 steps green. At0d0a03065its one red was clippy'slen_zeroin the new test, which1321274d7fixes; the change is test-only and leaves the staged images unchanged.ab2/), rules set before any boot. On both benches: base, b4096, b1024, base, b4096, b1024 benchcase, then b4096 testcases. b1024 is the same head atBOUNCE= 1024, staged because QEMU could not choose the threshold. 4096 stands unless b1024 moves better on the 4096-byte rows.Round 1 evidence (head
42e2716c3, base058bea2d3): https://github.com/ToyOSOrg/evidence/tree/18c2541c946ee50f72a9f8bda0c47633142525dc/r0Round 2 evidence (head
19c814f63, rebased on maina0e8f1333): https://github.com/ToyOSOrg/evidence/tree/98ad7c5154481d8af4a08eaba1e3bfb297d08f77/r1Round 2 (review r1's fixes):
cargo run -- --ci hostat19c814f63: rc=0, 109 steps green.cargo clippy -p kernel --bin kernel -- -D warnings: rc=0 on x86_64 and AArch64.a_claim_made_before_a_woken_waiter_looks_again_still_wakes_it(A ends, C claims, B re-checks, for both ends),an_abandoned_write_gives_its_room_back,a_second_claim_waits_until_the_first_ends,an_unwaited_claim_wakes_nobody.--debug(x86_64 QEMU) at19c814f63: 230 of 238 exit 0. The 8 others are the members that needSYS_DEBUG, as in round 1. The C corpus: 143 of 143 match. No gate panic, no kernel panic.Round 1 (still standing for what round 2 did not change): host model controls N1-N3; guest control G1 (the gate fires at depth 2 through
sys_getcwd); the whole QEMU suite at0d7a6ecfc, 83 of 90, with every red investigated (the recorded virtsmpcase counters-read silence, session_lifecycle under load, two load-only reds green alone); theBytes::surplusunderflow found there and fixed with a red-then-green host test.Metal A/B, round 1 (T14 and AMD; base, head, base, head benchcase, then head testcases and head shared-debug; all boots passed; judged by the rules set before the boots,
ab1-table.txt):pipe.throughput.64/256/1024−28 to −30%,4096−5%;ipc.reqrep.8/4096+25 to +26%,65536+12%,reqrep_handle.8+19%.fs.mem.fstat+10% (T14) and +17% (AMD);fs.mem.seek+17 to +19%;fs.mem.pread_8+7 to +8%;fs.mem.seq_write.4096−12 to −14%;seq_write.1048576−6 to −8%.fs.mem.read_4k_at+8%,write_4k_at+20%.pipe.throughput.16384/65536+73 to +83% on the T14;16384+71% on AMD. AMD's small-pipe rows did not move: their bases straddle the heads.fs.disk.*,persist.*) also flag "regressions" on the T14 (fs.disk.direct.*,stat,fstat+14 to +20%), but its two base boots differ from each other by about as much. The T14's durable-path variance is a recorded bench issue. Some of these rows (fstat,pread,write_4k_at) run this change's file paths, so I do not read them as clean controls either way.fstat,preadandpwrite. Its expected effect is unmeasured, and Consolidate storage around native bcachefs and userland filesystems #1152 deletes those paths anyway.ab2-rules.md): a new base at the rebased parent, benchcase ×4 and testcases on both benches.Risks
Page::own), which can fail withExhausted. This is recorded in its issue and goes with Consolidate storage around native bcachefs and userland filesystems #1152.on_zero_handleswhere no sleeping lock may be taken, and a dying carrier drains nothing either way.🤖 Generated with Claude Code