Repository navigation
Wakeups are kernel events, and a one-handle timed wait is SYS_HANDLE_WAIT - #1108
Conversation
Review of #1108 at
|
|
Review round 1, head
Suggestions taken:
No boot is needed for these fixes. Evidence: https://github.com/ToyOSOrg/evidence/tree/0af72a5d05789abaa11d8d52e1e5551d27054b7c/ipc-events/review-r1-044a7ae23 |
Review round 2 of #1108 at
|
Round 2 answered at
|
…nd a wait on one handle
A wakeup carried no data, yet every one was a pipe: two objects, a
pipe's four watches and, once rung, a frame, a byte per wake and a
64-byte drain loop to take it, EOF standing for "the waker is gone" and
a refused write for "the bell is gone". And a wait with a timeout on one
handle built a poller for it: an inbox page made, mapped, and given back
through the all-CPU flush shared memory takes.
An event is `EventSignal` and `EventWait` over one `Event`: a pending
bit, each side's end and one watch (`kernel::event`, host-tested). The
signal side raises it with one atomic or, posting only as it goes from
idle to pending, and never blocks or allocates; it answers `Gone` once
no wait side is left. The wait side takes it with one atomic and; a
wait side whose every signaler has gone is readable for good, and its
take answers `Gone`, never a signal nobody raised. Readiness observes
and takes nothing. The two authorities are separate by type: a wait
handle has no signal path, nor a signal handle a take. A real-time
signaler lends its window through the one path pipe wakes lend theirs
(`scheduler::post_lending`), bounded by the same quantum.
`SYS_HANDLE_WAIT` parks on one handle's one condition with a timeout,
answered from the readiness an inbox watch looks at, the notice's
listening included, so a one-handle timed wait needs no inbox. A claim
lends its watch to polls alone and is refused `NotSupported`.
ABI: `ObjectKind::{EventSignal, EventWait}`; `SYS_EVENT_CREATE`,
`SYS_EVENT_SIGNAL`, `SYS_EVENT_TAKE`, `SYS_HANDLE_WAIT`. SDK:
`toyos::wake` keeps its names over events, with `Hungup` for the other
side's end, and `poller::wait_one`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pipes that carried nothing but a wake are events now, each made by the side that hands the other its part, so no server ever acts on an event handle a client chose: - window's `Waiter`, the compositor's session opens, soundserver's command and device-loop bells and each client's signal, the supervisor's app-end watchers, its file worker (a counter beside the event, which coalesces), each disk driver's stop and each program's log `ALIVE`, whose waker's end is the program's, and libc's wake and control signals. - netstack's holder bells of listeners and datagram sockets, and a stream's receive and send bells. - A stream holder's `RING` frame is gone: netstack hands each holder a third bell's waker and watches that bell, rung for a cursor or a want and after an `EXPORT`; its wakers' end is the holder's leaving, so one watched handle a place stays one. - A block session's doorbell bytes on its connection are two events the driver makes and sends with `MSG_OPENED`; their wakers' end is the session's, and the connection goes once the session is open. Deleted: netstack's `pipes.rs`, `stream::SERVER_WATCH` and `RING`, the 64-byte drain loops, and a poller per socket end. A connection's raw byte calls stay: `FrameRx`, netstack's `Client::gone` and a guest test still use them, until the whole-message receive. A holder's bells hold no page, so netstack's places cost two pipes, not four. A guest test, `event_sides`, holds what the host tests cannot: a parked wait woken by another process's ring and by its end, a watch that takes nothing, and an end that is never a ring; `kernel_objects_refused` is refused an event too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
libc's interruptible sleep, diskserver's wait for a block service's answer, inspect's for a snapshot, the supervisor's for logkeeper's flush, hda's for soundserver's acceptance and the terminal's for its shell each made an inbox for one wait and gave it back after it. A refused wait now says so where the poller's would have spun or been ignored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fork's ToyOS waker wrote a byte per wake to a pipe its selector never read, so a woken waker stayed readable for good. It is an event now, and the selector takes it as it reports the waker's token. No program in the image builds mio for ToyOS, so the guest test that holds it, `mio_waker_coalesces`, depends on the fork itself: 10,000 wakes before a poll are one event, the next poll reports nothing, and a wake after it is reported. This closes `issues/mio-s-toyos-waker-never-drains-its-pipe.md`. Only mio's source line in Cargo.lock moves: `cargo update -p mio` re-resolves windows-sys for four unrelated crates. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d while it lives Until a connect is answered netstack holds the waker of the holder's rung bell itself, so that bell cannot read as the holder leaving, and a client killed meanwhile kept its place until netstack gave the connect up. netstack watches the first holder's connection until the answer is handed over and the rung after it: still one watched handle a place. A guest test on the real netstack, `a_client_killed_mid_connect_gives_its_place_back`, connects two children to an address nothing answers and kills one: its place comes back while the other's connect is still pending. With the rung watched instead, the killed child's place stays until the connects end, and the test fails. An event's shared state and watch were charged to its signal side, whose charge went with the last signaler while a wait side kept the memory. The charge is the `Event`'s now, as a pipe's is the `Pipe`'s. Comments and records that still said pipe where an event is now, a sentence that did not parse, broken wraps and a module out of order; `closing-a-pipe-end-ends-every-holders-polls-on-it.md` now covers an event's wait side. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The connect to an unanswered address fails when netstack gives up resolving it, about 3 s (nud::BROADCAST_SOLICIT requests nud::RETRANS apart), not at the connect's own give-up. The comments said the 20 s ceiling sat far inside the give-up; they now say the kill has to land within those 3 s and that a run missing it is red at "a child ended before it was killed". A peer that resolves and stays silent would remove that dependence, but on QEMU's user network only the slirp gateway and DNS addresses resolve, and a connect to them reaches the host's own stack, so it is not hermetic. Also: lent.rs's ring comment ends at the rung bell's end, and the issue record's edited lines are rewrapped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rebased onto main
|
|
Orchestrator: r2's comment blocker checked at 18060b5. The ceiling comments now state the ~3 s address give-up (nud::BROADCAST_SOLICIT), and both small suggestions are taken. The rebase onto f45dc5e (after #1106) is green: host CI 107/107, QEMU netstack and kernel tests including the virtio-net and 82574 arms. lib/link's byte doorbell migrates in phase 2 (#1121), which deletes byte writes on connections; that's within "whoever lands second migrates the other". Landing. |
session_lifecycle's remaining red: the job's last line and the supervisor's forced-stop line, written just before the forced reboot, sometimes never reach the serial console, while the kernel's records of the forced stop do. It is red at #1108 18060b5 too (1 of 4 runs on the remote QEMU runner), so it predates whole messages, which make it likelier (3 of 6 at e396bb5) by shortening the time from the terminate to the stop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
session_lifecycle's remaining red: the job's last line and the supervisor's forced-stop line, written just before the forced reboot, sometimes never reach the serial console, while the kernel's records of the forced stop do. It is red at #1108 18060b5 too (1 of 4 runs on the remote QEMU runner), so it predates whole messages, which make it likelier (3 of 6 at e396bb5) by shortening the time from the terminate to the stop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
session_lifecycle's remaining red: the job's last line and the supervisor's forced-stop line, written just before the forced reboot, sometimes never reach the serial console, while the kernel's records of the forced stop do. It is red at #1108 18060b5 too (1 of 4 runs on the remote QEMU runner), so it predates whole messages, which make it likelier (3 of 6 at e396bb5) by shortening the time from the terminate to the stop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
session_lifecycle's remaining red: the job's last line and the supervisor's forced-stop line, written just before the forced reboot, sometimes never reach the serial console, while the kernel's records of the forced stop do. It is red at #1108 18060b5 too (1 of 4 runs on the remote QEMU runner), so it predates whole messages, which make it likelier (3 of 6 at e396bb5) by shortening the time from the terminate to the stop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem
A wakeup carries no data, yet every one in ToyOS was a pipe. That meant two kernel objects and a pipe's four watches. Once rung it also took a 4 KiB frame (since #1074), a byte per wake and a 64-byte drain loop to take it. EOF stood for "the waker is gone" and a refused write for "the bell is gone". Block sessions and TCP streams also wrote raw doorbell bytes onto their connections (
Connection::{read,write}_nonblock, netstack'sRINGframe). Separately, about a dozen one-handle timed waits each built a fresh poller: an inbox page made, mapped and given back for one wait. mio's ToyOS waker wrote to a pipe its selector never drained, so a woken waker stayed readable for good (issues/mio-s-toyos-waker-never-drains-its-pipe.md).What changes
kernel::event, pure and host-tested;kernel/src/object/event.rs,kernel/src/syscall/event.rs).EventSignalandEventWaitshare oneEvent: a pending bit, each side's end and one watch.Goneonce no wait side is left.Gone, never a phantom signal.scheduler::post_lendingpath pipe wakes use, bounded by the same quantum.SYS_HANDLE_WAIT: park on one handle's one condition with a timeout. It is answered from the same readiness an inbox watch reads, including the notice-listener rule, so a one-handle wait needs no inbox. A claim is refusedNotSupported.ObjectKind::{EventSignal, EventWait},SYS_EVENT_CREATE,SYS_EVENT_SIGNAL,SYS_EVENT_TAKE,SYS_HANDLE_WAIT.toyos::wake::{pair, Waker, Bell}keeps its names over events, withHungupfor the other side's end. It also addspoller::wait_one.Waiter, compositor session opens, soundserver's command, device-loop and per-client bells;DRIVER_STOP, each program's logALIVE, and libc's wake and control signals;MSG_OPENED. The connection is dropped once the session is open.RINGis deleted. netstack makes a per-holder "rung" event, gives the holder its signal side, and watches the wait side instead of the holder's connection once the connect is answered. The holder's departure is that signal side's last handle going. Until the answer, netstack itself holds the signal side it will hand over, so it watches the first holder's connection instead, whose hang-up is the departure (review r1).EXPORTstays a message, followed by a ring. Each place still has one watched handle, so netstack's place bound is unchanged.SYS_HANDLE_WAIT: libc's interruptible sleep, diskserver's block-answer wait, inspect, the supervisor's logkeeper flush, hda's acceptance and the terminal's shell. This is a separate commit (080528966) so it can be reverted if the A/B says so (approved condition: it must win or tie against a cached poller).fe42defon the fork'stoyosbranch. This closes the mio issue.Deleted: the bell pipes and their drain loops, netstack's
pipes.rs,stream::SERVER_WATCHandRING, and a poller per socket end. A netstack holder's memory bound drops from four pipes to two.Consequential decisions
Gone; mio fork commit and pin in one landing.ResourceExhaustedarm are gone with the pipe.Checks
Head
18060b521, on mainf45dc5e3b(#1106). The first four commits have the tree of6250464e8, rebased, which is where the checks below were run unless noted; the fifth and sixth are review rounds 1 and 2. A connection's raw byte calls,read_nonblockandwrite_nonblock, stay until the whole-message receive (phase 2). So doeslib/link's doorbell byte, which phase 2 moves to events.Rebase onto #1106 (
18060b521) (evidence):cargo run -- --ci hostexit 0 (107/107). QEMU:netstack_holders_exported_to_children(virtio-net and 82574 arms) exit 0. QEMU:a_link_ends_under_a_held_stream_and_comes_back,stream_holders,a_client_killed_mid_connect_gives_its_place_back,udp_receive_times_outandkernel_objects_refused_alonegreen on virtio-net.Review round 1 (
044a7ae23) (evidence):cargo run -- --ci host: exit 0, 105 of 105.QEMU (x86-64):
stream_holders,netstack_holders_exported_to_childrenandkernel_objects_refused_alonepass. The newa_client_killed_mid_connect_gives_its_place_backpasses (exit 0).Its negative control is the rung watched while the connect is unanswered, with netstack rebuilt; the test fails (exit 1). A first version with one child passed that control, because the connect ends on its own within seconds. The test now kills one of two children and requires the place back while the survivor's connect is still pending, which only the kill can explain.
The event charge and the comment and record fixes need no boot: refusal under exhaustion is
kernel_objects_refused_alone's, rerun above.cargo run -- --ci host: exit 0, 105 of 105 steps green.cargo run -- --build-only: exit 0.QEMU,
cargo test --test toyos-build -- <9 names>: exit 0, 9 of 9 passed. The registrations arestream_holders,libc_children,netstack_holders_exported_to_children,udp_receive_times_out,the_data_partition_beside_the_stick_is_designated,a_host_disk_beside_the_stick_is_left_alone,session_lifecycle,hda_plays_to_the_controller_that_reaches_an_outputandkernel_objects_refused_alone. These were rerun after the rebase because kernel: a pipe keeps its bytes in 4 KiB frames it takes as it fills #1074 changed the pipe that connections still use.Negative controls, each a patch applied and restored in one run, on the pre-rebase tree (
f3fe154). The rebase changed none ofkernel::eventorkernel/src/object/event.rs.===READY===.===READY===.Metal, images built at
6250464e8anda5b330f39and run by the orchestrator, judged with the harness at a tree identical to6250464e8(evidence):event_sides,mio_waker_coalesces,cpal_drop_unreleased,hda_toneandhda_tone_resampledall pass on both (exit 0, 5 of 5 each).f762092failsmio_waker_coalesces(exit 1) on the defect itself: "a poll after the wake was reported reported [Token(7)]".osbenchjudge. Means of p50, head against base:ipc.reqrep.8-1.8%,.4096-2.1%,.65536-0.3%,ipc.reqrep_handle.8-1.9%;pingpong.pipe.*andpingpong.futex.*within 3%.ipc.reqrep.86613 against 6593 ns). head2 is slower on all four reqrep rows (ipc.reqrep.88546 ns, minimum 8256 against about 6500) and is also an outlier onwake.periodic.idleandtime.sleep_late.1ms, at the same clock as the others. This change does not touch the connection path reqrep measures, and head1 runs the same code at base speed, so I read head2 as one boot's machine state, not a regression. That is an inference from two boots: the cause is not established (cross-CCX placement would fit, unverified).persist.*, which is bimodal across base boots too (1.0 against 7.0 ms), and single-boot outliers.SYS_HANDLE_WAITagainst a cached poller (measurement-onlywaitonerows, head boots, p50 ns):080528966is kept.handle_waitwins on every ready row (4x) and on the T14's pingpong, and ties on the AMD's, whose cached-poller pingpong itself spans 5400 to 7004 across the two boots.Evidence: ToyOSOrg/evidence
pr-1108at e5371858a: logs with exit codes, control patches, the measurement-only waitone source and the metal request.Why the guest tests need their environment
a_client_killed_mid_connect_gives_its_place_backneeds a process's death and the real netstack's watch of it (tests/images/netcase.toml), which no host test has. Its peer is an address on QEMU's user network that nothing answers, so no public network. QEMU x86-64, order and counts, with QEMU's clock only a hang ceiling.event_sidescovers a parked wait woken by another process's ring and by its end, a watch that takes nothing, and an end that is never a ring. Host tests model the pure state, not cross-process parking. It runs on metal in the testcases shared boot (T14, AMD).mio_waker_coalesceschecks the fork itself: no image program builds mio for ToyOS. The check is 10,000 wakes before a poll coming back as one event, then nothing, then a later wake reported.Risks and open items
issues/a-tcp-connect-zeroes-twelve-mebibytes.mdon main still cites the issue kernel: a pipe keeps its bytes in 4 KiB frames it takes as it fills #1074 deleted (a-pipe-is-a-two-mebibyte-page-...). That is outside this change and left as is.🤖 Generated with Claude Code