diff --git a/Cargo.lock b/Cargo.lock index 64cca460091..0e24a367710 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1749,7 +1749,7 @@ dependencies = [ [[package]] name = "mio" version = "1.2.0" -source = "git+https://github.com/ToyOSOrg/mio?branch=toyos#f76209283c8133df0e56be2cd402e05ebdec41c8" +source = "git+https://github.com/ToyOSOrg/mio?branch=toyos#fe42def7bec9a1f01bfff764256fcdce77e7f7e1" dependencies = [ "libc", "toyos-abi", diff --git a/issues/a-childs-end-is-an-event-and-a-parent-takes-its-children-down.md b/issues/a-childs-end-is-an-event-and-a-parent-takes-its-children-down.md index 525e0218c98..54e88727092 100644 --- a/issues/a-childs-end-is-an-event-and-a-parent-takes-its-children-down.md +++ b/issues/a-childs-end-is-an-event-and-a-parent-takes-its-children-down.md @@ -174,8 +174,8 @@ an explicit force issues it, and never a timer's. and the stage reverted whole, beside `signal_alarm`; the routing laws are host tests (`signal_dispositions.rs`), and so is a wait deciding from one read of each child's end (`spawn_requests.rs`). libc's own thread also watches its - control pipe, so it watches at most 254 children: the 255th is refused - `EAGAIN`, a `poll` takes 255 descriptors beside its wake pipe, and `SIG_IGN` + control event, so it watches at most 254 children: the 255th is refused + `EAGAIN`, a `poll` takes 255 descriptors beside its wake event, and `SIG_IGN` is shown over 255 children. 5. **A login is a session under its owner**. **In review** (#1035, the P1b pull request). Per login the supervisor starts one session owner, diff --git a/issues/a-client-cannot-tell-soundserver-it-paused.md b/issues/a-client-cannot-tell-soundserver-it-paused.md index 3e3a6f69afc..dda1cb7d505 100644 --- a/issues/a-client-cannot-tell-soundserver-it-paused.md +++ b/issues/a-client-cannot-tell-soundserver-it-paused.md @@ -6,10 +6,11 @@ opened: 2026-08-01 # A client cannot tell soundd it paused, so soundd cannot tell paused from wedged -soundd infers a dead client from a pipe error: `NotFound` is a gone process, -`WouldBlock` is "merely behind". That detects exactly one failure. A client that -stops making progress while holding its fd — a wedged thread, a deadlock, a -callback that never returns — produces `WouldBlock` forever and is +soundd infers a dead client from its signal event alone: a ring answering +`Hungup` is a gone process, and any other ring coalesces into a signal the +client has not taken yet. That detects exactly one failure. A client that +stops making progress while holding its handle — a wedged thread, a deadlock, +a callback that never returns — takes no signal forever and is indistinguishable from one that deliberately paused. The ambiguity is not a soundd bug: pause is defined as the client's stream thread simply not reading, with no coordination, and the cpal backend confirms it — `pause()` stores a @@ -18,7 +19,7 @@ state word and parks on a futex, and soundd is never told. The cost is that `is_streaming` latches on the first delivered period and never clears, so one paused client holds soundd out of idle for the life of the process: a wake per period forever, the DMA engine running, the codec voice -open, and the client's shm, signal pipe and slot ring pinned. +open, and the client's shm, signal event and slot ring pinned. **Design, as far as it is settled.** Liveness is declared intent × observed progress. Progress already exists as `write_idx`; intent does not. One diff --git a/issues/anonymous-memory-swaps-to-an-encrypted-partition.md b/issues/anonymous-memory-swaps-to-an-encrypted-partition.md index 1f6372e695e..747d4424227 100644 --- a/issues/anonymous-memory-swaps-to-an-encrypted-partition.md +++ b/issues/anonymous-memory-swaps-to-an-encrypted-partition.md @@ -260,7 +260,7 @@ process is dead, counted as that job's `W`, and released only to its next start. - the block records (24 B per block); - reverse records (8 B per frame); - the resident chain; -- the session's doorbell pipe; +- the session's two doorbell events; - `M`. **Wiring.** Each job has a wire allowance (Sizes). @@ -301,7 +301,7 @@ Each check goes red under the mutation named after "red". | Stage | After | Builds | Exit | |---|---|---|---| | S0-a (`issues/process-memory-in-4-kib-pages-still-copies-each-programs-text.md`'s owner) | — | Every user copy faults or swaps in its chunk before any kernel lock, and pins after a re-check, with no pin across a wait. A lost anonymous page ends the caller, and a lost file-mapping page answers `EIO`. The futex re-translation never waits | Closes `issues/a-user-copy-demand-pages-under-whatever-its-caller-holds.md` and `issues/a-pipes-lock-spans-a-user-copy-and-its-first-allocation.md`. **H:** a chunk model in which an anonymous loss ends the caller with no partial result and a file-mapping loss answers `EIO`, red if either takes the other's outcome. **Q:** the pipe and futex sites, red when a copy runs under the pipe lock | -| S0-b (`issues/storage-is-layers-and-a-role-is-a-filesystem.md`'s owner) | — | The block-session client and driver side allocate nothing per I/O: tables sized by `DEPTH`, reads opened straight from the arena. A reopen refused `Held` by a closing session waits for its retirement. A doorbell pipe keeps its first frame if pipes take frames as they fill | **H:** `DEPTH` traffic with an allocator that fails after warm-up, red on any per-I/O allocation; a reopen during a dead session's close answered only after its last completion, red if the hold is released early | +| S0-b (`issues/storage-is-layers-and-a-role-is-a-filesystem.md`'s owner) | — | The block-session client and driver side allocate nothing per I/O: tables sized by `DEPTH`, reads opened straight from the arena. A reopen refused `Held` by a closing session waits for its retirement | **H:** `DEPTH` traffic with an allocator that fails after warm-up, red on any per-I/O allocation; a reopen during a dead session's close answered only after its last completion, red if the hold is released early | | P1 (`issues/process-memory-in-4-kib-pages-still-copies-each-programs-text.md`'s owner) | prerequisites | Both rules and the frame rule in `kernel/pure/frames.rs`; residency by charged job; `CHARGE`; `vm_wire`, the per-job allowance and additive charges; `resident` and `restart-hold`; the main stack sized by `PT_GNU_STACK` (8 MiB otherwise); resident by kind; unmovable frames in their own superframes; the ledger figures; `S = 0` | **H:** both rules at every `W` and `A` increase, red when a table build skips the backing rule; a resident fault never waits, red when a swappable fault may take a held frame; `SHM_FOR_JOB` to a resident job gives `W` and to an ordinary one `A`; wiring refused past the allowance; a helper shares its job's allowance; an additive wire, red if the charged job pays; two wirers where the first exits, red if any release can be refused or leaves a rule exceeded; superframe invariants under wire and lend; a hold kept whole. **Q:** a `CHARGE`-only handle creates an object `SHM_FOR_JOB` and is refused as a spawn's job and at `SYS_JOB_FLOOR`, red if either refusal accepts it; `mmap_stress`; each resident row's committed and populated totals reported | | P2 (`issues/process-memory-in-4-kib-pages-still-copies-each-programs-text.md`'s owner) | P1 | Reverse records and the RAM-object reverse index; `mem_scan`; `mem_reclaim` for clean pages and never-writable copied pages; `mem_read`/`mem_write` and `vm_wire_ticket` on the ticket table; watermarks; the kept fallback; superframe evacuation; the `reclaimer` role; service v0 (plain CLOCK) | **H:** revalidation, floor protection, revoke order, an evacuation case; `Pending` without waiting, every non-resident page requested at once, red if a call waits or requests fewer; an unmap cancels a parked ticket and removes the wirer's charge without waiting, red if the unmap waits or the charge stays; `Lost` removes the wirer's charge, red if it stays; a blocking wire of another job refused, red if accepted; a dropped copied page refilled byte-exact, red if a once-`ReadWrite` or resident range is dropped; a version not purged under a copy range, red if the range is not counted; the fallback with a stuck service; a remap between scan and reclaim. **Q:** a page-cache-heavy run with the controls usable | | C-wire (`issues/toyos-has-a-desktop.md`'s owner) | P1, P2 | Non-blocking wires of on-screen buffers from the loop; shown only once wired; the refusal points; window buffers charged to clients through `CHARGE`; the loop's file I/O (the launcher's app list, the kept scale) moved off it | **H:** red if the loop composes an unwired buffer; a resize keeps the old buffer until wired, red if it swaps early; one parked wire delays no other window; after `Lost` the compositor keeps running and drops that window, red if it ends; a close during a parked wire unmaps without waiting; the kept handle `CHARGE` alone, red if it carries `TRANSFER` or `DUP`. **Q:** with one buffer parked unresident, and with a parked file call, the other windows draw and the stop control works, red when the file call is made on the loop | diff --git a/issues/closing-a-pipe-end-ends-every-holders-polls-on-it.md b/issues/closing-a-pipe-end-ends-every-holders-polls-on-it.md index 2e467ff1557..e1d2964ff9b 100644 --- a/issues/closing-a-pipe-end-ends-every-holders-polls-on-it.md +++ b/issues/closing-a-pipe-end-ends-every-holders-polls-on-it.md @@ -19,6 +19,11 @@ watch it still needs as over. Found by reading, in review of pull request #861, which put the pipe end's leaving watch in that same loop. No test or machine has shown it. +An event's wait side has the same defect since #1108: `close_ends_polls` is +`true` for `EventWait`, and its one watch is every holder's, so closing a +duplicate of a wait side answers another holder's poll on it as gone. A +listener's descriptor is such a duplicate (found in review of #1108). + **Exit**: a close ends only the polls made through the handle it closes, and a -guest test that holds a duplicate of a pipe end, watches it, and closes the -other handle reads no answer on it. +guest test that holds a duplicate of a pipe end and of an event's wait side, +watches each, and closes the other handle reads no answer on either. diff --git a/issues/cpal-backend-hardcodes-the-format.md b/issues/cpal-backend-hardcodes-the-format.md index 89801c5893e..28a2ec071cf 100644 --- a/issues/cpal-backend-hardcodes-the-format.md +++ b/issues/cpal-backend-hardcodes-the-format.md @@ -15,7 +15,7 @@ other format is refused. **Client liveness waits on the same fork, not on soundserver alone.** The ambiguity between a paused and a wedged client is designed in: pausing is the -client's stream thread simply not reading its signal pipe, with no +client's stream thread simply not taking its signal event, with no coordination of any kind, and the cpal backend's `pause()` is a purely local store soundserver is never told about. No change confined to soundserver can separate the two, and landing the soundserver and SDK halves alone would kill diff --git a/issues/hda-client-stall-prints-a-stream-error-and-passes.md b/issues/hda-client-stall-prints-a-stream-error-and-passes.md index 9520f8b733a..4bcbb0ff65a 100644 --- a/issues/hda-client-stall-prints-a-stream-error-and-passes.md +++ b/issues/hda-client-stall-prints-a-stream-error-and-passes.md @@ -8,7 +8,7 @@ opened: 2026-10-03 `tests/guest/rust-tests/src/bin/hda_client_stall.rs` hands cpal an error callback that prints `audio error: …` and does nothing else. The fork reports -two things through it: a stream whose signal pipe went away, and a dropped +two things through it: a stream whose signal event went away, and a dropped stream soundd did not let go of within its fade and a ring of periods. Either leaves the job exiting 0, and the `hda_client_stall` T14 row reads the job's exit code. `tone.rs`'s `play_tone` asserts on the same callback. diff --git a/issues/mio-s-toyos-waker-never-drains-its-pipe.md b/issues/mio-s-toyos-waker-never-drains-its-pipe.md deleted file mode 100644 index 5faf47fdea6..00000000000 --- a/issues/mio-s-toyos-waker-never-drains-its-pipe.md +++ /dev/null @@ -1,23 +0,0 @@ ---- -status: open -kind: finding -opened: 2026-09-26 ---- - -# mio's ToyOS waker never drains its pipe - -`mio/src/sys/toyos/waker.rs` on the mio fork (`ToyOSOrg/mio`, branch `toyos`) -writes a byte to a pipe per `wake()` and ignores a full pipe, and the -selector registers the read end under the waker's token; nothing in -`mio/src/sys/toyos/` ever reads that pipe. `toyos::wake` is the wake pipe the -tree now has once (a `Bell` whose `take` empties it), and logd, soundd and -`window::Waiter` use it; mio could not take it as a swap, because draining is -the selector's to do on the waker's token and the selector has no such step. - -Not shown: whether a registration left readable makes the selector report the -waker's token on every poll, which is what an undrained pipe would do to a -level-triggered watch, or whether a full pipe ever loses a wake. - -**Exit**: the selector takes the bell when the waker's token comes up, through -`toyos::wake`, with a guest test that wakes a tokio runtime more times than the -pipe holds. diff --git a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md index 809d4c84708..7812d58ff67 100644 --- a/issues/netstack-moves-every-byte-through-two-kernel-pipes.md +++ b/issues/netstack-moves-every-byte-through-two-kernel-pipes.md @@ -56,7 +56,8 @@ The design and its roast are PR #863's body. in loom. - Stage 3, done (PR #905): a TCP stream, connected or accepted, is one region whose two rings are [tcp]'s buffers and the connection it was opened - on, which then carries only doorbells and the hang-up; std, libc, mio and + on, which then carries only exports; its doorbells and its holders' leaving + are events; std, libc, mio and socket2 reach it through `toyos::net::stream`, and `SYS_CONNECTION_JOIN` is gone. A stream is shared with a child as a holder of its own; other sockets are not yet (`issues/a-socket-libc-or-socket2-has-not-bound-is-not-shared-with-a-child.md`). A sent byte is copied diff --git a/issues/no-physical-memory-fairness.md b/issues/no-physical-memory-fairness.md index 9cacfae75ca..c47d286d76d 100644 --- a/issues/no-physical-memory-fairness.md +++ b/issues/no-physical-memory-fairness.md @@ -97,8 +97,9 @@ divide an eighth of physical memory by what one unit costs and refuse past it, and a pipe takes memory for its bytes as it holds them rather than at `create`. Their memory is counted to whichever job made it: a client's own connection is -its job's, but a pipe a server makes and hands out (fileserver's stream -pipes, netstack's bells) is the server's, which the server bounds. With no +its job's, but a pipe or an event a server makes and hands out +(fileserver's stream pipes, netstack's bells) is the server's, which the +server bounds. With no per-job limit a cap is still the only thing between one client and the machine. diff --git a/issues/null-sink-applies-one-connect.md b/issues/null-sink-applies-one-connect.md index 617a143ad92..e08920d5a0a 100644 --- a/issues/null-sink-applies-one-connect.md +++ b/issues/null-sink-applies-one-connect.md @@ -19,7 +19,7 @@ shape is the same in both and it is narrow: - soundd finds no device and presents the null sink; `null sink idle` prints. - The first client connects. The control thread prints `opening stream`, the **mix loop** prints `client 0 connected (id=0)` — so it woke, drained the - command pipe and applied the command. + command pipe (an event since #1108) and applied the command. - Nothing from the mix loop ever again. `stats.report` fires every 2 s while a client is present and the connect resets that window, so a single missing stats line is proof the loop stopped; boot7's client was a 2 s tone and no @@ -27,8 +27,8 @@ shape is the same in both and it is narrow: - Every later client is stranded: the control thread keeps running (it printed `opening stream` for doom 14 s later) and `open_stream` answers, but no `client N connected` follows, so the mixer never signals it and it blocks - reading its signal pipe forever. That is `tone` never exiting and doom - wedging with a black window before its render loop. + reading its signal pipe (an event since #1108) forever. That is `tone` never + exiting and doom wedging with a black window before its render loop. **Not reproduced in QEMU.** `tests/desktopaudiocase` was built for this: the T14's shape, with the client's three descriptors as pipes to a terminal that is @@ -41,11 +41,12 @@ with a terminal opened afterwards. **Eliminated, each with a run behind it.** CPU count. The cpal client path (`null_sink_shipped_client`, adopted from `wt/toyos-hdaprobe` `fa47241`, two `/system/bin/tone` in series at 1.16 s and 1.15 s). soundd blocking on a client -(`signal_clients` uses `write_nonblock`; there is no blocking write in the mix -loop). The accept path being held by a stuck client (accept and mix are -separate threads and the control thread ran for 14 s afterwards). A CQ overflow -(`Poller::wait`'s `dropped` assertion would have killed soundd, and soundd is -alive). A panic anywhere in soundd, for the same reason. +(`signal_clients` used `write_nonblock`, and an event's signal never blocks; +there is no blocking write in the mix loop). The accept path being held by a +stuck client (accept and mix are separate threads and the control thread ran +for 14 s afterwards). A CQ overflow (`Poller::wait`'s `dropped` assertion +would have killed soundd, and soundd is alive). A panic anywhere in soundd, for +the same reason. **Eliminated by reading, and recorded so it is not re-derived.** The mix loop holds no lock across its wait, and neither `PIPES` nor `IO_URINGS` was held on @@ -54,8 +55,8 @@ open its stream. The mixer's timeout while streaming is finite (one device period), so a park with that deadline is the only way to stop, and the timer that ends it is the same one the compositor's frame interval rides. The closed `io_uring::cancel_by_source` lost wake is *not* this: the mix loop's only -registration is on the command pipe, which soundd owns both ends of and nobody -closes. +registration is on the command pipe, now an event, which soundd owns both +sides of and nobody closes. **What settles it is where soundd's mix thread is parked and on what deadline, and nothing in the tree reads the deadline off a running T14.** soundd's mix thread parked with a sane diff --git a/kernel/pure/event.rs b/kernel/pure/event.rs new file mode 100644 index 00000000000..fca7420f34c --- /dev/null +++ b/kernel/pure/event.rs @@ -0,0 +1,147 @@ +//! An event's state: one coalescing pending bit, and whether each side still +//! has a holder. +//! +//! A signal sets the bit and a take clears it; any number of signals before a +//! take are one. A wait side whose every signaler has gone is ready for good, +//! but a take there answers [`Taken::Gone`], never a signal nobody raised. + +#![forbid(unsafe_code)] + +use core::sync::atomic::{AtomicU8, Ordering}; + +const PENDING: u8 = 1 << 0; +const SIGNALERS_GONE: u8 = 1 << 1; +const WAITERS_GONE: u8 = 1 << 2; + +/// What a signal found. +#[derive(Clone, Copy, PartialEq, Eq, Debug)] +pub enum Signaled { + /// The event was idle and is pending now: its waiters are owed a post. + Raised, + /// It was already pending, so this signal coalesced into that one. + Coalesced, + /// No wait side is left to take it. + NoWaiter, +} + +/// What a take found. +#[derive(Clone, Copy, PartialEq, Eq, Debug)] +pub enum Taken { + /// A signal was pending, and is taken. + Signal, + /// Nothing pending, and a signaler may still raise one. + Empty, + /// Nothing pending, and no signaler is left to raise one. + Gone, +} + +pub struct State(AtomicU8); + +impl Default for State { + fn default() -> Self { + Self::new() + } +} + +impl State { + pub const fn new() -> Self { + Self(AtomicU8::new(0)) + } + + /// Raise the event. Release, so what the signaler wrote before it is seen + /// by the take that answers [`Taken::Signal`]. + pub fn signal(&self) -> Signaled { + let was = self.0.fetch_or(PENDING, Ordering::AcqRel); + if was & WAITERS_GONE != 0 { + Signaled::NoWaiter + } else if was & PENDING != 0 { + Signaled::Coalesced + } else { + Signaled::Raised + } + } + + /// Take the pending signal, if any. + pub fn take(&self) -> Taken { + let was = self.0.fetch_and(!PENDING, Ordering::AcqRel); + if was & PENDING != 0 { + Taken::Signal + } else if was & SIGNALERS_GONE != 0 { + Taken::Gone + } else { + Taken::Empty + } + } + + /// Whether a wait on the event answers now: a signal is pending, or no + /// signaler is left. Observes; takes nothing. + pub fn ready(&self) -> bool { + self.0.load(Ordering::Acquire) & (PENDING | SIGNALERS_GONE) != 0 + } + + /// The signal side's last handle has gone. + pub fn end_signalers(&self) { + self.0.fetch_or(SIGNALERS_GONE, Ordering::AcqRel); + } + + /// The wait side's last handle has gone. + pub fn end_waiters(&self) { + self.0.fetch_or(WAITERS_GONE, Ordering::AcqRel); + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn signals_before_a_take_are_one() { + let event = State::new(); + assert_eq!(event.signal(), Signaled::Raised); + assert_eq!(event.signal(), Signaled::Coalesced); + assert_eq!(event.take(), Taken::Signal); + assert_eq!(event.take(), Taken::Empty); + assert_eq!(event.signal(), Signaled::Raised, "a taken event is idle again"); + } + + #[test] + fn readiness_observes_and_does_not_take() { + let event = State::new(); + assert!(!event.ready()); + event.signal(); + assert!(event.ready()); + assert!(event.ready()); + assert_eq!(event.take(), Taken::Signal); + assert!(!event.ready()); + } + + /// A pending signal is answered before the signalers' end, and the end + /// after it is never read as a signal. + #[test] + fn signalers_gone_is_ready_but_never_a_signal() { + let event = State::new(); + event.signal(); + event.end_signalers(); + assert!(event.ready()); + assert_eq!(event.take(), Taken::Signal); + assert!(event.ready(), "an event with no signaler left stays ready"); + assert_eq!(event.take(), Taken::Gone); + assert_eq!(event.take(), Taken::Gone); + } + + #[test] + fn an_end_with_nothing_pending_is_gone_at_once() { + let event = State::new(); + event.end_signalers(); + assert!(event.ready()); + assert_eq!(event.take(), Taken::Gone); + } + + #[test] + fn a_signal_with_no_waiter_left_says_so() { + let event = State::new(); + event.end_waiters(); + assert_eq!(event.signal(), Signaled::NoWaiter); + assert_eq!(event.signal(), Signaled::NoWaiter); + } +} diff --git a/kernel/pure/lib.rs b/kernel/pure/lib.rs index 6bc8a883667..adbb98577b4 100644 --- a/kernel/pure/lib.rs +++ b/kernel/pure/lib.rs @@ -14,6 +14,7 @@ pub mod clock; pub mod cpu; pub mod dirnodes; pub mod dmaguard; +pub mod event; pub mod frames; pub mod iova; pub mod message; diff --git a/kernel/src/object/event.rs b/kernel/src/object/event.rs new file mode 100644 index 00000000000..a47f1f26882 --- /dev/null +++ b/kernel/src/object/event.rs @@ -0,0 +1,101 @@ +//! An event's two sides, as two object types over one [`Event`]. +//! +//! [`EventSignal`] has no take and [`EventWait`] no signal, so the two +//! authorities are separate by type. Each side's last handle is that side's +//! end ([`kernel::event`]): the signal side's makes the wait side ready for +//! good, the wait side's makes a signal answer `Gone`. + +use alloc::sync::Arc; +use core::sync::atomic::{AtomicBool, Ordering}; + +use kernel::event::{Signaled, State, Taken}; + +use crate::mm::charge::{self, Charge, Exhausted}; +use crate::watch::Watch; + +use super::{KObjectVariant, ObjectCore, ZeroHandles}; + +pub struct Event { + state: State, + /// Posted when the event is raised and when its signalers end; the wait + /// side's readable watch. + watch: Arc, + /// A real-time signaler's window, owed to whoever takes the signal next: + /// a waiter already runnable misses the wake-time loan. + lend_rt: AtomicBool, + /// Itself and its watch, charged to the job that made it for as long as + /// either side holds it. + _charge: Charge, +} + +pub struct EventSignal { + pub(super) core: ObjectCore, + event: Arc, +} + +pub struct EventWait { + pub(super) core: ObjectCore, + event: Arc, +} + +/// A fresh event's two sides. +pub fn create() -> Result<(Arc, Arc), Exhausted> { + let charge = Charge::current(charge::arc::() + charge::arc::())?; + let signal = EventSignal::new_core(0)?; + let wait = EventWait::new_core(0)?; + let event = Arc::new(Event { + state: State::new(), + watch: Arc::new(Watch::new()), + lend_rt: AtomicBool::new(false), + _charge: charge, + }); + Ok((Arc::new(EventSignal { core: signal, event: event.clone() }), Arc::new(EventWait { core: wait, event }))) +} + +impl EventSignal { + /// Raise the event; never blocks and allocates nothing. `false` once no + /// wait side is left. + pub fn signal(&self) -> bool { + let event = &self.event; + match event.state.signal() { + Signaled::Raised => { + crate::scheduler::post_lending(&event.watch, || event.lend_rt.store(true, Ordering::Relaxed)); + true + } + Signaled::Coalesced => true, + Signaled::NoWaiter => false, + } + } +} + +impl EventWait { + /// Take the pending signal, granting the window a real-time signaler lent. + pub fn take(&self) -> Taken { + let taken = self.event.state.take(); + if taken == Taken::Signal && self.event.lend_rt.swap(false, Ordering::Relaxed) { + crate::scheduler::boost_current_rt_inherited(); + } + taken + } + + pub fn ready(&self) -> bool { + self.event.state.ready() + } + + pub fn watch(&self) -> &Arc { + &self.event.watch + } +} + +impl ZeroHandles for EventSignal { + fn on_zero_handles(&self) { + self.event.state.end_signalers(); + self.event.watch.post(); + } +} + +impl ZeroHandles for EventWait { + fn on_zero_handles(&self) { + self.event.state.end_waiters(); + } +} diff --git a/kernel/src/object/mod.rs b/kernel/src/object/mod.rs index e5c4a94ddac..d705a7ee287 100644 --- a/kernel/src/object/mod.rs +++ b/kernel/src/object/mod.rs @@ -18,6 +18,7 @@ use crate::mm::charge::{self, Charge, Exhausted}; use crate::sync::Lock; pub mod device; +pub mod event; pub mod file; pub mod handle; pub mod inbox; @@ -262,6 +263,9 @@ kobject! { deferred Device => device::DeviceClaim, deferred Acceptor => port::Acceptor, deferred Inbox => inbox::InboxObject, + // Each side's last handle is that side's end, which the other side reads. + deferred EventSignal => event::EventSignal, + deferred EventWait => event::EventWait, // The flush that makes releasing it safe waits for every CPU, so it runs from the queue, never inline. deferred SharedMem => shm::SharedMemObject, // A service with no clients right now is not a service that has stopped. diff --git a/kernel/src/object/ops.rs b/kernel/src/object/ops.rs index 98dfaa70bde..a239a15f0f3 100644 --- a/kernel/src/object/ops.rs +++ b/kernel/src/object/ops.rs @@ -61,6 +61,8 @@ pub fn initial_rights(object: &KObjectRef) -> Rights { KObjectRef::Job(_) => super::job::JobObject::RIGHTS, KObjectRef::Notice(_) => super::notice::NoticeObject::RIGHTS, KObjectRef::WakeLatency(_) => super::wake_latency::WakeLatencyObject::RIGHTS, + KObjectRef::EventSignal(_) => Rights::DUP.union(Rights::TRANSFER).union(Rights::WRITE), + KObjectRef::EventWait(_) => BASE.union(Rights::READ), } } @@ -219,7 +221,7 @@ pub fn pipe_read(object: &KObjectRef) -> Option<(&PipeRef, WaitClass)> { KObjectRef::PipeWrite(_) | KObjectRef::File(_) | KObjectRef::Device(_) | KObjectRef::Console(_) | KObjectRef::Acceptor(_) | KObjectRef::Inbox(_) | KObjectRef::SysCap(_) - | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::Namespace(_) + | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::EventSignal(_) | KObjectRef::EventWait(_) | KObjectRef::Namespace(_) | KObjectRef::SharedMem(_) | KObjectRef::Process(_) | KObjectRef::Job(_) | KObjectRef::Null(_) | KObjectRef::Notice(_) => None, } @@ -234,7 +236,7 @@ pub fn pipe_write(object: &KObjectRef) -> Option<(&PipeRef, WaitClass, Landing)> KObjectRef::PipeRead(_) | KObjectRef::File(_) | KObjectRef::Device(_) | KObjectRef::Console(_) | KObjectRef::Acceptor(_) | KObjectRef::Inbox(_) | KObjectRef::SysCap(_) - | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::Namespace(_) + | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::EventSignal(_) | KObjectRef::EventWait(_) | KObjectRef::Namespace(_) | KObjectRef::SharedMem(_) | KObjectRef::Process(_) | KObjectRef::Job(_) | KObjectRef::Null(_) | KObjectRef::Notice(_) => None, } @@ -282,6 +284,7 @@ pub fn read_watch(object: &KObjectRef) -> Option { KObjectRef::Acceptor(a) => Some(WatchRef::Shared(a.watch().clone())), KObjectRef::Process(p) => Some(WatchRef::Shared(p.watch().clone())), KObjectRef::Notice(n) => Some(WatchRef::Shared(n.watch().clone())), + KObjectRef::EventWait(e) => Some(WatchRef::Shared(e.watch().clone())), KObjectRef::Job(_) => Some(WatchRef::InPlace(crate::mm::pmm::returned())), // Nothing posts for a serial byte, so a console's readable poll has // no source (`issues/a-console-watch-waits-on-the-keyboard-not-the-serial-line.md`). @@ -295,7 +298,7 @@ pub fn read_watch(object: &KObjectRef) -> Option { // Named unconditionally: the watch alone cannot enforce rights. KObjectRef::SysCap(_) => Some(WatchRef::Static(&crate::log::user::WATCH)), KObjectRef::PipeWrite(_) | KObjectRef::File(_) | KObjectRef::Inbox(_) - | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::Namespace(_) + | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::EventSignal(_) | KObjectRef::Namespace(_) | KObjectRef::SharedMem(_) | KObjectRef::Null(_) => None, } } @@ -306,7 +309,7 @@ pub fn watched_readable(object: &KObjectRef) { match object { KObjectRef::Notice(n) => n.listen(), KObjectRef::PipeRead(_) | KObjectRef::PipeWrite(_) | KObjectRef::Connection(_) | KObjectRef::Device(_) - | KObjectRef::Acceptor(_) | KObjectRef::Inbox(_) | KObjectRef::SharedMem(_) | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) + | KObjectRef::Acceptor(_) | KObjectRef::Inbox(_) | KObjectRef::SharedMem(_) | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::EventSignal(_) | KObjectRef::EventWait(_) | KObjectRef::Namespace(_) | KObjectRef::File(_) | KObjectRef::Console(_) | KObjectRef::SysCap(_) | KObjectRef::Process(_) | KObjectRef::Null(_) | KObjectRef::Job(_) => {} } @@ -321,7 +324,7 @@ pub fn write_watch(object: &KObjectRef) -> Option { KObjectRef::PipeRead(_) | KObjectRef::File(_) | KObjectRef::Device(_) | KObjectRef::Acceptor(_) | KObjectRef::Inbox(_) | KObjectRef::SysCap(_) - | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::Namespace(_) + | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::EventSignal(_) | KObjectRef::EventWait(_) | KObjectRef::Namespace(_) | KObjectRef::SharedMem(_) | KObjectRef::Process(_) | KObjectRef::Job(_) | KObjectRef::Null(_) | KObjectRef::Notice(_) => None, } @@ -336,7 +339,7 @@ pub fn pipe_end_watch(object: &KObjectRef) -> Option { KObjectRef::Connection(_) | KObjectRef::Acceptor(_) | KObjectRef::Process(_) | KObjectRef::Job(_) | KObjectRef::Notice(_) | KObjectRef::Console(_) | KObjectRef::Device(_) | KObjectRef::SysCap(_) - | KObjectRef::File(_) | KObjectRef::Inbox(_) | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) + | KObjectRef::File(_) | KObjectRef::Inbox(_) | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::EventSignal(_) | KObjectRef::EventWait(_) | KObjectRef::Namespace(_) | KObjectRef::SharedMem(_) | KObjectRef::Null(_) => None, } } @@ -350,7 +353,7 @@ pub fn other_end_gone(object: &KObjectRef) -> bool { KObjectRef::Connection(_) | KObjectRef::Acceptor(_) | KObjectRef::Process(_) | KObjectRef::Job(_) | KObjectRef::Notice(_) | KObjectRef::Console(_) | KObjectRef::Device(_) | KObjectRef::SysCap(_) - | KObjectRef::File(_) | KObjectRef::Inbox(_) | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) + | KObjectRef::File(_) | KObjectRef::Inbox(_) | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::EventSignal(_) | KObjectRef::EventWait(_) | KObjectRef::Namespace(_) | KObjectRef::SharedMem(_) | KObjectRef::Null(_) => false, } } @@ -375,7 +378,7 @@ fn close_ends_polls(object: &KObjectRef) -> bool { }, KObjectRef::PipeRead(_) | KObjectRef::PipeWrite(_) | KObjectRef::Connection(_) | KObjectRef::Acceptor(_) | KObjectRef::File(_) | KObjectRef::Inbox(_) - | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::Namespace(_) + | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::EventSignal(_) | KObjectRef::EventWait(_) | KObjectRef::Namespace(_) | KObjectRef::SharedMem(_) | KObjectRef::Null(_) | KObjectRef::Notice(_) => true, } } @@ -582,7 +585,7 @@ pub fn try_read(object: &KObjectRef, buf: &mut UserBytesMut) -> Option { }), KObjectRef::PipeWrite(_) | KObjectRef::Acceptor(_) | KObjectRef::Inbox(_) | KObjectRef::SysCap(_) - | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::Namespace(_) + | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::EventSignal(_) | KObjectRef::EventWait(_) | KObjectRef::Namespace(_) | KObjectRef::SharedMem(_) | KObjectRef::Process(_) | KObjectRef::Job(_) => Some(SyscallError::PermissionDenied.to_u64()), } } @@ -613,7 +616,7 @@ pub fn try_write(object: &KObjectRef, buf: &UserBytes) -> Option { KObjectRef::Null(_) => Some(buf.len() as u64), KObjectRef::PipeRead(_) | KObjectRef::Acceptor(_) | KObjectRef::Inbox(_) | KObjectRef::SharedMem(_) | KObjectRef::SysCap(_) - | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::Namespace(_) + | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::EventSignal(_) | KObjectRef::EventWait(_) | KObjectRef::Namespace(_) | KObjectRef::Process(_) | KObjectRef::Job(_) | KObjectRef::Notice(_) => { Some(SyscallError::PermissionDenied.to_u64()) } @@ -688,7 +691,7 @@ pub fn fstat(object: &KObjectRef) -> Stat { node: 0, }, KObjectRef::Inbox(_) | KObjectRef::SysCap(_) - | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::Namespace(_) + | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::EventSignal(_) | KObjectRef::EventWait(_) | KObjectRef::Namespace(_) | KObjectRef::Process(_) | KObjectRef::Job(_) | KObjectRef::Notice(_) => plain(FileType::Unknown), KObjectRef::Device(d) => plain(match d.class() { device_registry::DeviceType::Framebuffer => FileType::Framebuffer, @@ -735,6 +738,7 @@ pub fn has_data(object: &KObjectRef) -> bool { KObjectRef::Process(p) => p.finished(), KObjectRef::Job(j) => j.ready(), KObjectRef::Notice(n) => n.readable(), + KObjectRef::EventWait(e) => e.ready(), KObjectRef::File(_) | KObjectRef::Null(_) => true, KObjectRef::Device(d) => match d.class() { // A claim its last handle has let go has nothing to read: a poll @@ -748,7 +752,7 @@ pub fn has_data(object: &KObjectRef) -> bool { device_registry::DeviceType::Framebuffer => true, }, KObjectRef::PipeWrite(_) | KObjectRef::Inbox(_) | KObjectRef::SysCap(_) - | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::Namespace(_) + | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::EventSignal(_) | KObjectRef::Namespace(_) | KObjectRef::SharedMem(_) => false, } } @@ -761,7 +765,7 @@ pub fn read_posts_are_readiness(object: &KObjectRef) -> bool { KObjectRef::SysCap(_) => true, KObjectRef::Console(_) | KObjectRef::PipeRead(_) | KObjectRef::PipeWrite(_) | KObjectRef::Connection(_) | KObjectRef::Acceptor(_) | KObjectRef::File(_) | KObjectRef::Device(_) - | KObjectRef::Inbox(_) | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::Namespace(_) + | KObjectRef::Inbox(_) | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::EventSignal(_) | KObjectRef::EventWait(_) | KObjectRef::Namespace(_) | KObjectRef::SharedMem(_) | KObjectRef::Process(_) | KObjectRef::Job(_) | KObjectRef::Null(_) | KObjectRef::Notice(_) => false, } @@ -777,7 +781,7 @@ pub fn has_space(object: &KObjectRef) -> bool { KObjectRef::Console(_) => crate::log::console::has_room(), KObjectRef::PipeRead(_) | KObjectRef::Device(_) | KObjectRef::Acceptor(_) | KObjectRef::Inbox(_) | KObjectRef::SysCap(_) - | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::Namespace(_) + | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::EventSignal(_) | KObjectRef::EventWait(_) | KObjectRef::Namespace(_) | KObjectRef::SharedMem(_) | KObjectRef::Process(_) | KObjectRef::Job(_) | KObjectRef::Notice(_) => false, } } @@ -796,7 +800,7 @@ pub fn mark_tty(object: &KObjectRef) -> u64 { KObjectRef::Connection(_) | KObjectRef::File(_) | KObjectRef::Device(_) | KObjectRef::Console(_) | KObjectRef::Acceptor(_) | KObjectRef::Inbox(_) | KObjectRef::SysCap(_) | KObjectRef::SharedMem(_) - | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::Namespace(_) + | KObjectRef::Connector(_) | KObjectRef::WakeLatency(_) | KObjectRef::EventSignal(_) | KObjectRef::EventWait(_) | KObjectRef::Namespace(_) | KObjectRef::Process(_) | KObjectRef::Job(_) | KObjectRef::Null(_) | KObjectRef::Notice(_) => { SyscallError::InvalidArgument.to_u64() } diff --git a/kernel/src/scheduler.rs b/kernel/src/scheduler.rs index 8962cc508eb..8c4605202a8 100644 --- a/kernel/src/scheduler.rs +++ b/kernel/src/scheduler.rs @@ -10,7 +10,7 @@ use kernel::sched::hw::{CpuId, Machine, Nanos}; use kernel::sched::task::{Refused, SafePoint, WaitClass, QUANTUM_NS}; use crate::arch::percpu; -use crate::watch::{self, Cancel}; +use crate::watch::{self, Cancel, Watch}; use crate::hw::HW; use crate::pipe::PipeRef; use crate::process::{self, Pid, Tid}; @@ -253,12 +253,17 @@ pub fn exit_current() -> ! { unreachable!("exit_current: returned from the exit pass"); } -/// Wake pipe readers, lending each an RT window if the writer holds one; the -/// pipe is also marked, so a runnable reader takes the window too. +/// Wake pipe readers, lending each an RT window if the writer holds one. pub fn wake_pipe_readers(pipe: &PipeRef) { - let watch = pipe.read_watch(); + post_lending(pipe.read_watch(), || pipe.set_rt_boost_pending()); +} + +/// Post `watch`, lending whoever it wakes one [`boost_window`] if the poster +/// is real-time; `mark` records the loan for a consumer already runnable, +/// which takes it with [`boost_current_rt_inherited`]. +pub fn post_lending(watch: &Watch, mark: impl FnOnce()) { if driver::current_is_rt() { - pipe.set_rt_boost_pending(); + mark(); watch.post_boosted(boost_window()); } else { watch.post(); diff --git a/kernel/src/syscall/dispatch.rs b/kernel/src/syscall/dispatch.rs index 457c84ea9dd..fd583a8a451 100644 --- a/kernel/src/syscall/dispatch.rs +++ b/kernel/src/syscall/dispatch.rs @@ -26,6 +26,7 @@ use super::device::{ sys_device_dma_limit, sys_device_dma_start, sys_device_dma_unmap, sys_device_reg_read, }; +use super::event::{sys_event_create, sys_event_signal, sys_event_take, sys_handle_wait}; use super::fs::{ spawn_cwd, sys_chdir, sys_delete, sys_getcwd, sys_mkdir, sys_open, sys_readdir, sys_readlink, sys_rename, sys_rmdir, sys_stat, sys_symlink, @@ -625,6 +626,10 @@ pub(crate) fn syscall_dispatch(num: u64, a1: u64, a2: u64, a3: u64, a4: u64) -> }, SYS_JOB_FLOOR => sys_job_floor(RawHandle(a1 as u32), RawHandle(a2 as u32), a3), SYS_WAKE_LATENCY_HOLD => sys_wake_latency_hold(RawHandle(a1 as u32), a2), + SYS_EVENT_CREATE => sys_event_create(), + SYS_EVENT_SIGNAL => sys_event_signal(RawHandle(a1 as u32)), + SYS_EVENT_TAKE => sys_event_take(RawHandle(a1 as u32)), + SYS_HANDLE_WAIT => sys_handle_wait(RawHandle(a1 as u32), a2, a3), SYS_ACCOUNT_INFO => { if a3 < core::mem::size_of::() as u64 { return SyscallError::InvalidArgument.to_u64(); } let Some(addr) = UserAddr::checked(a2) else { return bad_addr }; diff --git a/kernel/src/syscall/event.rs b/kernel/src/syscall/event.rs new file mode 100644 index 00000000000..8dae451a94a --- /dev/null +++ b/kernel/src/syscall/event.rs @@ -0,0 +1,97 @@ +//! Events, and the wait on one handle that any waitable object answers. + +use kernel::event::Taken; +use kernel::sched::task::WaitClass; +use toyos_abi::handle::{RawHandle, Rights}; +use toyos_abi::inbox::{OTHER_END_GONE, READABLE, WRITABLE}; +use toyos_abi::syscall::SyscallError; + +use crate::object::event::{EventSignal, EventWait}; +use crate::object::ops::{self, WatchRef}; +use crate::object::KObjectRef; +use crate::process; +use crate::time::{Deadline, Duration}; +use crate::watch; + +use super::cancelled; +use super::ipc::install_pair; + +/// The signal side's handle in the high word, the wait side's in the low. +pub(super) fn sys_event_create() -> u64 { + match crate::object::event::create() { + Ok((signal, wait)) => install_pair(KObjectRef::EventSignal(signal), KObjectRef::EventWait(wait)), + Err(e) => SyscallError::from(e).to_u64(), + } +} + +pub(super) fn sys_event_signal(h: RawHandle) -> u64 { + match process::with_process_data(|data| data.handles.get::(h, Rights::WRITE)) { + Ok(event) if event.signal() => 0, + Ok(_) => SyscallError::Gone.to_u64(), + Err(e) => e.refuse(), + } +} + +/// `1` for a signal taken, `WouldBlock` for none pending, `Gone` for none +/// pending and no signaler left. Never blocks: a wait is `SYS_HANDLE_WAIT`. +pub(super) fn sys_event_take(h: RawHandle) -> u64 { + match process::with_process_data(|data| data.handles.get::(h, Rights::READ)) { + Ok(event) => match event.take() { + Taken::Signal => 1, + Taken::Empty => SyscallError::WouldBlock.to_u64(), + Taken::Gone => SyscallError::Gone.to_u64(), + }, + Err(e) => e.refuse(), + } +} + +/// Park until `h`'s object answers the one condition `interest` names, or +/// `timeout_nanos` passes (`u64::MAX` never does); answers the condition's +/// bit, or `0` at the deadline. The answer is the one an inbox's watch looks +/// at, so a readable wait on a notice makes its process a listener as a +/// readable poll does. +pub(super) fn sys_handle_wait(h: RawHandle, interest: u64, timeout_nanos: u64) -> u64 { + let interest = match u32::try_from(interest) { + Ok(bit @ (READABLE | WRITABLE | OTHER_END_GONE)) => bit, + _ => return SyscallError::InvalidArgument.to_u64(), + }; + let object = match process::with_process_data(|data| data.handles.get_ref(h, Rights::WAIT).cloned()) { + Ok(object) => object, + Err(e) => return e.refuse(), + }; + let on = match interest { + READABLE if ops::read_posts_are_readiness(&object) => None, + READABLE => ops::read_watch(&object), + WRITABLE => ops::write_watch(&object), + _ => ops::pipe_end_watch(&object), + }; + // A claim lends its watch to polls alone: its release answers them, and + // a parked thread has no such answer. + let Some(on) = on.filter(|w| !matches!(w, WatchRef::Claim(_))) else { + return SyscallError::NotSupported.to_u64(); + }; + if interest == READABLE { + ops::watched_readable(&object); + } + let ready = || match interest { + READABLE => ops::has_data(&object), + WRITABLE => ops::has_space(&object), + _ => ops::other_end_gone(&object), + }; + let deadline = match timeout_nanos { + u64::MAX => Deadline::never(), + nanos => Deadline::at(crate::clock::now() + Duration::from_nanos(nanos)), + }; + let parkable = crate::scheduler::Parkable::at_entry(); + let waited = match &on { + WatchRef::Static(w) => watch::wait_until(&parkable, *w, 0, WaitClass::Other, deadline, ready), + WatchRef::InPlace(w) => watch::wait_until(&parkable, *w, 0, WaitClass::Other, deadline, ready), + WatchRef::Shared(w) => watch::wait_until(&parkable, &**w, 0, WaitClass::Other, deadline, ready), + WatchRef::Claim(_) => unreachable!("refused above"), + }; + match waited { + Ok(()) if ready() => u64::from(interest), + Ok(()) => 0, + Err(_) => cancelled(), + } +} diff --git a/kernel/src/syscall/ipc.rs b/kernel/src/syscall/ipc.rs index e81a94fe517..e6a7e6a00d5 100644 --- a/kernel/src/syscall/ipc.rs +++ b/kernel/src/syscall/ipc.rs @@ -37,7 +37,7 @@ pub(super) fn sys_pipe() -> u64 { } /// Install two objects or neither; the two handles come back packed into one word, which cannot be read as an error. -fn install_pair(first: KObjectRef, second: KObjectRef) -> u64 { +pub(super) fn install_pair(first: KObjectRef, second: KObjectRef) -> u64 { let installed = process::with_process_data(|data| { // Room for both before either: a refused pair installs nothing, so nothing is taken back. if !data.handles.make_room(2) { diff --git a/kernel/src/syscall/mod.rs b/kernel/src/syscall/mod.rs index 7de7afe4aec..42d1672cd8f 100644 --- a/kernel/src/syscall/mod.rs +++ b/kernel/src/syscall/mod.rs @@ -6,6 +6,7 @@ mod debug; mod device; pub(crate) mod dispatch; +mod event; mod fs; mod handles; mod io; diff --git a/lib/blockring/src/lib.rs b/lib/blockring/src/lib.rs index 31a75f5355e..a8d94ed35e4 100644 --- a/lib/blockring/src/lib.rs +++ b/lib/blockring/src/lib.rs @@ -10,9 +10,10 @@ //! and every field before it acts ([`entry::Request::decode`], the //! transport's cursor bounds). //! -//! **A doorbell is a byte on the session's connection**, written after the -//! entries it announces are published. The connection is also what tells each -//! end the other has gone: its hang-up is the session's end. +//! **A doorbell is an event each way**, rung after the entries it announces +//! are published. The server makes both and hands the client its sides with +//! the open's answer ([`wire::OPENED_HANDLES`]); each end's bell is also what +//! tells it the other has gone: its waker's end is the session's end. //! //! **A completion means the device took the request, never that it is //! durable.** Durability is a flush's answer, and a flush answers for its own diff --git a/lib/blockring/src/wire.rs b/lib/blockring/src/wire.rs index 85eca5ab47b..1209e973416 100644 --- a/lib/blockring/src/wire.rs +++ b/lib/blockring/src/wire.rs @@ -3,8 +3,8 @@ //! **One way in, one answer.** On the service's port a client opens a //! partition by its unique GUID ([`MSG_OPEN`]), sending the session's region //! with it, and is answered [`MSG_OPENED`] or [`MSG_REFUSED`] with a -//! [`Refusal`]. After `MSG_OPENED` the connection carries nothing but doorbell -//! bytes, each way. +//! [`Refusal`]. `MSG_OPENED` carries the session's doorbells, and the +//! connection carries nothing after it. //! //! **What a connection may open is its [`Grant`]**, the badge the holder of //! the service's acceptor minted its connector with, which the kernel stamps @@ -29,8 +29,12 @@ use crate::layout::{self, ServerRings, RING_WORDS}; /// Open the partition whose unique GUID is the payload, over the region sent /// with it, if the connection's [`Grant`] admits it. Handles: the region. pub const MSG_OPEN: u32 = 1; -/// The session is open; the payload is [`Opened`]. +/// The session is open; the payload is [`Opened`], and the handles are +/// [`OPENED_HANDLES`]. pub const MSG_OPENED: u32 = 2; +/// What [`MSG_OPENED`] carries, in order: the waker the client rings for its +/// requests, and the bell the server rings for its answers. +pub const OPENED_HANDLES: usize = 2; /// Refused; the payload is a [`Refusal`]'s word. pub const MSG_REFUSED: u32 = 3; /// What partitions the service serves that the connection's [`Grant`] diff --git a/lib/blockserve/src/sessions.rs b/lib/blockserve/src/sessions.rs index fae09e6023c..96f5df35acf 100644 --- a/lib/blockserve/src/sessions.rs +++ b/lib/blockserve/src/sessions.rs @@ -1,7 +1,7 @@ //! One disk's sessions, as its driver serves them. //! //! **A session is a partition held**, from the open the block service hands -//! over to the client's hang-up: one holder per partition +//! over to the client's leaving, which its doorbell's end says: one holder per partition //! (`toyos-blockhold`), every request bounded to the partition and the arena //! before the device sees it (`toyos_blockring::server`), the data moved by //! the device straight into and out of the client's own region, which the @@ -10,8 +10,8 @@ //! naming it has been answered, never before. //! //! **A server never blocks on a client.** The open is answered with one -//! `try_send` on the client's own connection, and a doorbell is one -//! non-blocking write whose refusal is a full pipe — a doorbell already rung. +//! `try_send` on the client's own connection, carrying the session's +//! doorbells, which the driver makes; a ring never waits. //! //! **A designation is the driver's own work on one partition** ([`Job`]): it //! holds the partition as a session would for as long as it runs, so nothing @@ -23,6 +23,7 @@ use std::collections::BTreeMap; use toyos::ipc::Connection; use toyos::poller::{Poller, READABLE}; +use toyos::wake::{self, Bell, Waker}; use toyos_abi::syscall::SyscallError; use toyos_blockhold::Holds; use toyos_blockring::disk::Part; @@ -68,13 +69,16 @@ pub trait Device { /// A partition held, over a client's region. struct Served { - conn: Connection, + /// Rung by the client for its requests; its waker going is the client's leaving. + heard: Bell, + /// Rung for the client's answers. + ring: Waker, region: Region, device_addr: u64, rings: ServerRings, state: ServerSession, unique: [u8; 16], - /// The client hung up, or broke the protocol: nothing more is taken, and + /// The client left, or broke the protocol: nothing more is taken, and /// the session ends once nothing of it is on the device. closing: bool, requests: u64, @@ -191,8 +195,16 @@ impl Sessions { return refuse(&conn, if why == SyscallError::ResourceExhausted { Refusal::Exhausted } else { Refusal::Malformed }); } }; + let bells = wake::pair().and_then(|requests| Ok((requests, wake::pair()?))); + let Ok(((client_ring, heard), (ring, answered))) = bells else { + if let Err(why) = dev.dma_unmap(device_addr) { + panic!("{}: the kernel would not take a refused session's region back: {why:?}", self.name); + } + self.holds.release(first); + return refuse(&conn, Refusal::Exhausted); + }; let (rings, opened) = Opened::over(region.words(), blocks, unique); - if conn.try_send_bytes(wire::MSG_OPENED, &opened.encode()).is_err() { + if conn.try_send_bytes_with(wire::MSG_OPENED, &opened.encode(), [client_ring.into(), answered.into()]).is_err() { // The client went before it heard: nothing of the session reached // the device, and what was taken for it goes back. if let Err(why) = dev.dma_unmap(device_addr) { @@ -203,17 +215,17 @@ impl Sessions { let id = self.next_id; self.next_id += 1; let state = ServerSession::new(first, opened.blocks(), writes); - let served = Served { conn, region, device_addr, rings, state, unique, closing: false, requests: 0, posted: false }; + let served = Served { heard, ring, region, device_addr, rings, state, unique, closing: false, requests: 0, posted: false }; self.sessions.insert(id, served); let access = if writes { "" } else { ", read-only" }; println!("{}: session {id} opened {} ({blocks} blocks{access})", self.name, crate::guid_text(unique)); } - /// Watch every session's connection, under `base` plus its id. + /// Watch every session's doorbell, under `base` plus its id. pub fn watch(&self, poller: &Poller, base: u64) { for (id, s) in &self.sessions { if !s.closing { - poller.watch(&s.conn, READABLE, base + id); + poller.watch(&s.heard, READABLE, base + id); } } } @@ -319,27 +331,18 @@ impl Sessions { continue; } s.posted = false; - if s.rings.1.publish(s.region.words()) { - match s.conn.write_nonblock(&[1]) { - Ok(_) | Err(SyscallError::WouldBlock) => {} - Err(_) => s.closing = true, - } + if s.rings.1.publish(s.region.words()) && s.ring.ring().is_err() { + s.closing = true; } } } - /// Consume session `id`'s doorbell bytes; once its client has hung up, - /// the session is closing. + /// Take session `id`'s doorbell; once its client has left, the session is + /// closing. pub fn hear(&mut self, id: u64) { let s = self.sessions.get_mut(&id).expect("a session"); - let mut sink = [0u8; 64]; - while !s.closing { - match s.conn.read_nonblock(&mut sink) { - Ok(0) => s.closing = true, - Ok(_) => {} - Err(SyscallError::WouldBlock) => return, - Err(_) => s.closing = true, - } + if s.heard.try_take().is_err() { + s.closing = true; } } diff --git a/lib/logstream/src/lib.rs b/lib/logstream/src/lib.rs index 622c71a9001..3b2ffd7496c 100644 --- a/lib/logstream/src/lib.rs +++ b/lib/logstream/src/lib.rs @@ -61,15 +61,15 @@ pub const ORIGINS: &str = "log-origins"; /// `logkeeper`'s, where it puts every program's line with its head. pub const CONSOLE: &str = "log-console"; -/// The endowment label of a program's end of the pipe [`REGISTER`] hands -/// `logkeeper` the other end of: nothing is written to it, and the program's end -/// closing — which only its exit does — is how `logkeeper` knows the ring has no -/// writer left to wait for. A label and not a slot, so no child inherits it. +/// The endowment label of a program's waker of the bell [`REGISTER`] hands +/// `logkeeper`: nothing rings it, and the program's waker closing — which only +/// its exit does — is how `logkeeper` knows the ring has no writer left to wait +/// for. A label and not a slot, so no child inherits it. pub const ALIVE: &str = "log-alive"; /// The supervisor → `logkeeper`: a program's log ring. The payload is [`Registration`]; the -/// frame carries two handles — the ring, and the read end of a pipe whose -/// only writer is the program, so its end is the ring's. +/// frame carries two handles — the ring, and the bell whose only waker is the +/// program's, so its end is the ring's. pub const REGISTER: u32 = 1; /// `logkeeper`'s answer to a reader on [`SERVICE`]: one handle, the read end of the diff --git a/lib/quiesce/src/lib.rs b/lib/quiesce/src/lib.rs index 1e83dd25284..b54d10084b1 100644 --- a/lib/quiesce/src/lib.rs +++ b/lib/quiesce/src/lib.rs @@ -84,9 +84,10 @@ pub const SYNC_MS: u64 = 5_000; /// (`toyos_xhci::hc::STOP_NS`), so a driver that keeps its bound is waited out. pub const DRIVERS_MS: u64 = 3_000; -/// The endowment a disk driver is told the machine stops by: a pipe the supervisor -/// closes once every file server has synced. The driver hands its devices back, -/// leaves its controller as firmware would find it, and ends. +/// The endowment a disk driver is told the machine stops by: a bell whose +/// waker the supervisor closes once every file server has synced; it is +/// readable from then. The driver hands its devices back, leaves its +/// controller as firmware would find it, and ends. pub const DRIVER_STOP: &str = "stop"; /// The thread the kernel's quiesce-last actuators hold, by the name its diff --git a/sdk/abi/src/handle.rs b/sdk/abi/src/handle.rs index 86cfb1978a8..acc471ed9a6 100644 --- a/sdk/abi/src/handle.rs +++ b/sdk/abi/src/handle.rs @@ -90,10 +90,12 @@ pub enum ObjectKind { Job = 14, Notice = 15, WakeLatency = 16, + EventSignal = 17, + EventWait = 18, } impl ObjectKind { - pub const ALL: [ObjectKind; 17] = [ + pub const ALL: [ObjectKind; 19] = [ Self::PipeRead, Self::PipeWrite, Self::Connection, @@ -111,6 +113,8 @@ impl ObjectKind { Self::Job, Self::Notice, Self::WakeLatency, + Self::EventSignal, + Self::EventWait, ]; pub const fn from_word(word: u32) -> Option { diff --git a/sdk/abi/src/syscall.rs b/sdk/abi/src/syscall.rs index 1d784f84b1e..4c95cc77890 100644 --- a/sdk/abi/src/syscall.rs +++ b/sdk/abi/src/syscall.rs @@ -340,6 +340,14 @@ pub const SYS_VM_REMAP: u64 = 146; /// /// [`Rights::WAKE_LATENCY`]: crate::handle::Rights::WAKE_LATENCY pub const SYS_WAKE_LATENCY_HOLD: u64 = 147; +/// A fresh event's two sides. See [`event_create`]. +pub const SYS_EVENT_CREATE: u64 = 148; +/// Raise an event, on its signal side's `WRITE`. See [`event_signal`]. +pub const SYS_EVENT_SIGNAL: u64 = 149; +/// Take an event's pending signal, on its wait side's `READ`. See [`event_take`]. +pub const SYS_EVENT_TAKE: u64 = 150; +/// Park on one handle's one condition, with a timeout. See [`handle_wait`]. +pub const SYS_HANDLE_WAIT: u64 = 151; /// Bins in the per-process syscall profile — one for every number this ABI /// issues, and one at the end for every number it does not. @@ -347,13 +355,13 @@ pub const SYS_WAKE_LATENCY_HOLD: u64 = 147; /// **The profile's parts sum to its total, and that is the whole requirement.** /// A bin array narrower than the ABI reaches drops calls out of the line while /// the total goes on counting them. -pub const SYSCALL_PROFILE_BINS: usize = 149; +pub const SYSCALL_PROFILE_BINS: usize = 153; /// Where a number this ABI does not issue is counted. Merging is a degradation /// a reader can see in the line; dropping is one nobody can. pub const SYSCALL_PROFILE_OTHER: usize = SYSCALL_PROFILE_BINS - 1; -const _: () = assert!(SYS_WAKE_LATENCY_HOLD < SYSCALL_PROFILE_OTHER as u64); +const _: () = assert!(SYS_HANDLE_WAIT < SYSCALL_PROFILE_OTHER as u64); pub const WNOHANG: u64 = 1; /// Every bit `SYS_PROCESS_WAIT`'s flag word defines; a wait carrying another @@ -1049,6 +1057,44 @@ pub fn pipe() -> Result { }) } +/// An event's two sides: [`signal`](Self::signal) carries `WRITE|DUP|TRANSFER`, +/// [`wait`](Self::wait) `READ|WAIT|DUP|TRANSFER`. +pub struct EventEnds { + pub signal: RawHandle, + pub wait: RawHandle, +} + +/// A fresh event: one coalescing pending signal, raised by its signal side +/// and taken by its wait side. Packed as [`pipe`] is. +pub fn event_create() -> Result { + let raw = check(syscall(SYS_EVENT_CREATE, 0, 0, 0, 0))?; + Ok(EventEnds { signal: RawHandle((raw >> 32) as u32), wait: RawHandle((raw & 0xFFFF_FFFF) as u32) }) +} + +/// Raise the event; signals before a take are one. Never blocks and +/// allocates nothing. `Gone` once the wait side's last handle has gone. +pub fn event_signal(signal: RawHandle) -> Result<(), SyscallError> { + check(syscall(SYS_EVENT_SIGNAL, signal.0 as u64, 0, 0, 0)).map(drop) +} + +/// Take the pending signal. `WouldBlock` with none pending; `Gone` with none +/// pending and the signal side's last handle gone, which is never a signal. +/// A wait side is readable while either holds, and a readable watch takes +/// nothing. +pub fn event_take(wait: RawHandle) -> Result<(), SyscallError> { + check(syscall(SYS_EVENT_TAKE, wait.0 as u64, 0, 0, 0)).map(drop) +} + +/// Park until `handle` answers the one condition `interest` names +/// ([`READABLE`](crate::inbox::READABLE), [`WRITABLE`](crate::inbox::WRITABLE) +/// or [`OTHER_END_GONE`](crate::inbox::OTHER_END_GONE)), as an inbox watch +/// would, or `timeout_nanos` passes; `u64::MAX` waits for good. Answers the +/// condition's bit, or `0` at the timeout. Needs `WAIT`; an object with no +/// such condition, or a device claim, is `NotSupported`. +pub fn handle_wait(handle: RawHandle, interest: u32, timeout_nanos: u64) -> Result { + check(syscall(SYS_HANDLE_WAIT, handle.0 as u64, interest as u64, timeout_nanos, 0)).map(|bit| bit as u32) +} + /// Read the inherited environment variables into `buf`. /// Returns the number of bytes written, or the required size if buf is too small. pub fn get_env(buf: &mut [u8]) -> usize { diff --git a/sdk/libc/src/posix_io.rs b/sdk/libc/src/posix_io.rs index ce5b958b554..227a3bd710f 100644 --- a/sdk/libc/src/posix_io.rs +++ b/sdk/libc/src/posix_io.rs @@ -1002,12 +1002,12 @@ pub struct pollfd { struct Waited { /// Entries ready, their `revents` written. ready: i32, - /// Whether the wake pipe beside them was readable. + /// Whether the wake event beside them was readable. woken: bool, } /// The most entries and doorbells one poll watches: a poller's handles, one of -/// them the wake pipe a handler interrupts it by. +/// them the wake event a handler interrupts it by. const MAX_POLLED: usize = toyos::poller::Poller::MAX_HANDLES as usize - 1; /// `poll`'s wait over the `n` entries at `fds` until `deadline`, in the diff --git a/sdk/libc/src/signal.rs b/sdk/libc/src/signal.rs index c6235fb1c94..0067b09261b 100644 --- a/sdk/libc/src/signal.rs +++ b/sdk/libc/src/signal.rs @@ -1,12 +1,12 @@ //! The signals libc imitates, delivered: libc's own thread, which raises them -//! from the events they stand for, the wake pipe a blocking call watches +//! from the events they stand for, the wake event a blocking call watches //! beside what it waits for, and the handlers' runs. Where each goes is //! `sigreq`'s. //! //! **libc's own thread starts with the first thing it watches**: an alarm, a //! `SIGCHLD` handler, or the process's notice. It watches every child whose //! end has not raised `SIGCHLD`, at most [`crate::spawn::MAX_CHILDREN`], -//! beside its control pipe and the notice, which fills one poller. +//! beside its control event and the notice, which fills one poller. //! //! **libc listens to the notice once a quit's signal is off its default or //! blocked**, before the call that put it there returns: until then a quit @@ -23,43 +23,43 @@ use crate::errcode::ENOSYS; use crate::pthread::Lock; use crate::sigreq::{Delivery, Kept, Raised, SigAction, Signals, SIGALRM, SIGCHLD, SIGHUP, SIGINT, SIGTERM}; -/// The ends of a pipe this library made and never closes. +/// The sides of an event this library made and never closes: `wait` is +/// readable while it is knocked. #[derive(Clone, Copy)] -struct Pipe { - read: toyos::BorrowedHandle<'static>, - write: toyos::BorrowedHandle<'static>, +struct Event { + wait: toyos::BorrowedHandle<'static>, + signal: toyos::BorrowedHandle<'static>, } -impl Pipe { +impl Event { fn new() -> Self { - let ends = syscall::pipe().expect("signals: no pipe for libc's own use"); + let ends = syscall::event_create().expect("signals: no event for libc's own use"); // SAFETY: made here and never closed: libc's own for the process's life. - unsafe { Self { read: toyos::BorrowedHandle::borrow_raw(ends.read), write: toyos::BorrowedHandle::borrow_raw(ends.write) } } + unsafe { Self { wait: toyos::BorrowedHandle::borrow_raw(ends.wait), signal: toyos::BorrowedHandle::borrow_raw(ends.signal) } } } - /// One byte in, or none if one is already there to be read. fn knock(&self) { - match syscall::write_nonblock(self.write.raw(), &[0]) { - Ok(_) | Err(SyscallError::WouldBlock) => {} - Err(e) => panic!("signals: a pipe of libc's own refused a byte: {e:?}"), + if let Err(e) = syscall::event_signal(self.signal.raw()) { + panic!("signals: an event of libc's own refused a signal: {e:?}"); } } - /// Every byte out. fn drain(&self) { - let mut buf = [0u8; 64]; - while matches!(syscall::read_nonblock(self.read.raw(), &mut buf), Ok(n) if n > 0) {} + match syscall::event_take(self.wait.raw()) { + Ok(()) | Err(SyscallError::WouldBlock) => {} + Err(e) => panic!("signals: an event of libc's own refused a take: {e:?}"), + } } } struct State { signals: Signals, - /// The pipe each interruptible blocking call watches, made by the first. - wake: Option, - /// Whether `wake` holds its byte. + /// The event each interruptible blocking call watches, made by the first. + wake: Option, + /// Whether `wake` is knocked. woken: bool, - /// libc's own thread's control pipe, once it runs. - keeper: Option, + /// libc's own thread's control event, once it runs. + keeper: Option, /// When the alarm is due, in the monotonic clock's nanoseconds. alarm: Option, /// Whether this process listens to its notice, which it does for good. @@ -103,7 +103,7 @@ fn on_keeper(tid: u32) -> bool { static HANDLERS: AtomicBool = AtomicBool::new(false); impl State { - /// The wake pipe's byte made to agree with `Signals::wake_wanted`. + /// The wake event made to agree with `Signals::wake_wanted`. fn settle(&mut self) { let Some(wake) = self.wake else { return }; match (self.signals.wake_wanted(), self.woken) { @@ -119,7 +119,7 @@ impl State { fn kick(&mut self) { let keeper = *self.keeper.get_or_insert_with(|| { crate::pthread::spawn_own(keep); - Pipe::new() + Event::new() }); keeper.knock(); } @@ -371,7 +371,7 @@ impl Call { true => crate::sigreq::KEEPER_MASK, false => mask.unwrap_or_else(|| state.signals.mask(tid)), }; - let wake = state.wake.get_or_insert_with(Pipe::new).read; + let wake = state.wake.get_or_insert_with(Event::new).wait; state.signals.enter(tid, mask, children); state.settle(); Some(Self { tid, mask, children, wake, inside: true }) @@ -450,9 +450,8 @@ pub(crate) fn sleep(nanos: u64) -> Result<(), u64> { if left == 0 { return Ok(()); } - let poller = Poller::new(1); - poller.watch(&call.wake(), READABLE, 0); - poller.wait(1, left, |_| {}); + // Ready or the deadline: the take at the loop's top says which. + toyos::poller::wait_one(&call.wake(), READABLE, left).expect("signals: libc's own wake event refused a wait"); } } @@ -480,7 +479,7 @@ unsafe extern "C" fn keep(_: *mut u8) -> *mut u8 { loop { let (control, listening, due) = { let state = STATE.lock(); - let control = state.keeper.expect("libc's own thread runs only once its pipe is made").read; + let control = state.keeper.expect("libc's own thread runs only once its event is made").wait; let watched = match state.signals.kept(SIGCHLD) { Kept::Handle(_) => crate::spawn::unraised(), _ => alloc::vec::Vec::new(), diff --git a/sdk/libc/src/sigreq.rs b/sdk/libc/src/sigreq.rs index 577ce177c4d..6bbfd8a7bb9 100644 --- a/sdk/libc/src/sigreq.rs +++ b/sdk/libc/src/sigreq.rs @@ -395,7 +395,7 @@ impl Signals { /// Whether some thread in a blocking call has something to look at: a /// signal it leaves unblocked, or children moved since it last looked. - /// libc's wake pipe holds its byte exactly while this holds. + /// libc's wake event is pending exactly while this holds. pub(crate) fn wake_wanted(&self) -> bool { self.waiters .iter() diff --git a/sdk/libc/src/spawn.rs b/sdk/libc/src/spawn.rs index 25f62146ef3..7ffc3f6bf32 100644 --- a/sdk/libc/src/spawn.rs +++ b/sdk/libc/src/spawn.rs @@ -47,7 +47,7 @@ const POSIX_SPAWN_SETSIGMASK: i16 = 0x08; const FLAGS: i16 = POSIX_SPAWN_RESETIDS | POSIX_SPAWN_SETPGROUP | POSIX_SPAWN_SETSIGDEF | POSIX_SPAWN_SETSIGMASK; /// The most children unwaited at once: libc's own thread watches them all on -/// one poller, beside its own control pipe and the process's notice. +/// one poller, beside its own control event and the process's notice. pub(crate) const MAX_CHILDREN: usize = toyos::poller::Poller::MAX_HANDLES as usize - 2; /// What the shell `system` and `popen` run a command with. @@ -469,7 +469,7 @@ fn wait_child(pid: i32, block: bool) -> Result, - signal: crate::Pipe, + signal: crate::wake::Bell, period_frames: u32, device_sample_rate: u32, device_channels: u16, @@ -273,7 +273,7 @@ impl AudioStream { let [shm, signal] = control .recv_handles_exact::(&header) .ok_or(AudioError::MissingHandles)?; - let signal = crate::Pipe(signal); + let signal = crate::wake::Bell::from(signal); let slot_count = resp.slot_count as u32; let shm_size = AudioSlotHeader::SIZE + slot_count as usize * resp.client_period_bytes as usize; @@ -297,14 +297,12 @@ impl AudioStream { /// Block until soundserver signals, then fill all available ring slots via the /// callback. Each callback invocation receives one period-sized buffer. /// - /// Returns `Err(Disconnected)` on signal-pipe EOF (soundserver is gone or - /// removed this client) — the caller must stop the stream, not retry. + /// Returns `Err(Disconnected)` once soundserver has let go of the signal + /// (it is gone or removed this client) — the caller must stop the stream, + /// not retry. pub fn wait_and_fill(&mut self, mut callback: impl FnMut(&mut [u8])) -> Result<(), AudioError> { - let mut buf = [0u8; 64]; - match self.signal.read(&mut buf) { - Ok(0) => return Err(AudioError::Disconnected), - Ok(_) => {} - Err(e) => return Err(AudioError::Ipc(IpcError::Syscall(e))), + if self.signal.wait_for(u64::MAX).is_err() { + return Err(AudioError::Disconnected); } while let Some(slot) = self.slot_writer.begin_fill() { callback(&mut self.period); diff --git a/sdk/toyos/src/endow.rs b/sdk/toyos/src/endow.rs index 89d1466a13d..37c8e4da636 100644 --- a/sdk/toyos/src/endow.rs +++ b/sdk/toyos/src/endow.rs @@ -71,6 +71,8 @@ from_handle! { Device => Device, crate::Console => crate::Console, crate::Pipe => crate::Pipe, + crate::wake::Waker => crate::wake::Waker::from, + crate::wake::Bell => crate::wake::Bell::from, crate::ipc::Connection => crate::ipc::Connection, crate::FramebufferDev => |h| crate::FramebufferDev(Device(h)), crate::PciDev => |h| crate::PciDev(Device(h)), diff --git a/sdk/toyos/src/lib.rs b/sdk/toyos/src/lib.rs index 6456a16465f..34c65d58356 100644 --- a/sdk/toyos/src/lib.rs +++ b/sdk/toyos/src/lib.rs @@ -377,6 +377,8 @@ mod std_io { std_owned_from!( super::Pipe, + super::wake::Waker, + super::wake::Bell, super::Device, super::Console, super::ipc::Connection, diff --git a/sdk/toyos/src/net.rs b/sdk/toyos/src/net.rs index d0383eb3799..4fa74c521b7 100644 --- a/sdk/toyos/src/net.rs +++ b/sdk/toyos/src/net.rs @@ -184,7 +184,7 @@ ipc_payload! { pub timeout_ms: u32, } - /// The answer to [`MsgType::TcpConnect`], with the stream's two bells + /// The answer to [`MsgType::TcpConnect`], with the stream's bells /// (`net::stream::BELLS`). pub struct TcpConnectResponse { pub local_port: u16, @@ -214,7 +214,7 @@ ipc_payload! { } /// The answer to a listener holder's [`socket::ACCEPT`], with the stream's - /// connection and its two bells: the peer, and whether Nagle's algorithm + /// connection and its bells: the peer, and whether Nagle's algorithm /// is off for the stream as its listener had it, which netstack has /// written into the region's header too. pub struct TcpAcceptResponse { diff --git a/sdk/toyos/src/net/ends.rs b/sdk/toyos/src/net/ends.rs index c1a20361e2c..ab56cb7d4cb 100644 --- a/sdk/toyos/src/net/ends.rs +++ b/sdk/toyos/src/net/ends.rs @@ -20,6 +20,8 @@ use super::NetError; pub(super) const HELD_CONNECTION: Rights = Rights::DUP.union(Rights::WAIT).union(Rights::READ).union(Rights::WRITE); /// What a holder keeps of a bell netstack rings: it reads and waits on it, and duplicates it. pub(super) const HELD_BELL: Rights = Rights::DUP.union(Rights::WAIT).union(Rights::READ); +/// What a holder keeps of the bell it rings netstack on: it signals it, and duplicates it. +pub(super) const HELD_RUNG: Rights = Rights::DUP.union(Rights::WRITE); /// `handle` held with `rights` alone, the original closed, as a number the table keeps from /// here: refused, the original is closed all the same. diff --git a/sdk/toyos/src/net/socket.rs b/sdk/toyos/src/net/socket.rs index a5c7495476b..2fb8ce2c99b 100644 --- a/sdk/toyos/src/net/socket.rs +++ b/sdk/toyos/src/net/socket.rs @@ -23,8 +23,7 @@ //! rung makes what its request needs and reads the bell empty just before the request goes, //! and what another holder took first is answered `WouldBlock` too. A call that fails before //! its request goes leaves the ring in the bell, where a poller still sees it. Nothing else -//! reads the bell. The look is made with a poller of the end's own, made with the end, so a -//! call that does not wait asks the kernel for nothing it can refuse. +//! takes the bell; a look observes it and takes nothing. //! //! **A handle with no end here is refused** [`NetError::NotASocket`]. A descriptor's handle is //! a copy of its end's bell, so a poller watches it for reading; the end keeps its own @@ -38,7 +37,6 @@ use toyos_abi::syscall; use toyos_abi::RawHandle; use crate::ipc::{self, Connection}; -use crate::poller::Poller; use crate::sync::Lock; use crate::{AsHandle, BorrowedHandle, OwnedHandle}; @@ -166,9 +164,8 @@ struct End { static TABLE: Lock> = Lock::new(Ends::new()); -/// Each end's turn on its connection, one request and its answer at a time, and the poller its -/// look at its bell is made with: `Some` from the end's making until [`free`]. -static TURNS: [Lock>; MAX_SOCKETS] = [const { Lock::new(None) }; MAX_SOCKETS]; +/// Each end's turn on its connection, one request and its answer at a time. +static TURNS: [Lock<()>; MAX_SOCKETS] = [const { Lock::new(()) }; MAX_SOCKETS]; /// Lets go of an end no handle or call reaches: its connection, which is its holder's leaving, @@ -179,16 +176,9 @@ fn free(slot: usize, end: End) { close_kept(end.connection.raw()); close_kept(end.bell.raw()); } - *TURNS[slot].lock() = None; TABLE.lock().unreserve(slot); } -/// The poller a reserved slot's end looks at its bell with. Refused, nothing is made. -fn make_poller(slot: usize) -> Result<(), NetError> { - *TURNS[slot].lock() = Some(Poller::try_new(1).map_err(|_| NetError::ResourceExhausted)?); - Ok(()) -} - /// Runs `f` as a call through `handle`, its end kept until it returns. fn call(handle: BorrowedHandle<'_>, f: impl FnOnce(usize, End) -> Result) -> Result { let (slot, end) = TABLE.lock().call(handle.raw()).ok_or(NetError::NotASocket)?; @@ -204,13 +194,13 @@ fn call_on(handle: BorrowedHandle<'_>, kind: Kind, f: impl FnOnce(usize, End) call(handle, |slot, end| if end.kind == kind { f(slot, end) } else { Err(NetError::InvalidInput) }) } -/// Whether `bell` is readable, waiting for it at most `nanos` on `poller`: rung, or hung up -/// with netstack. Nothing is read. -fn rung(poller: &Poller, bell: BorrowedHandle<'_>, nanos: u64) -> bool { - poller.watch(&bell, crate::poller::READABLE, 0); - let mut answered = false; - poller.wait(1, nanos, |_| answered = true); - answered +/// Whether `bell` is readable, waiting for it at most `nanos`: rung, or hung up with netstack. +/// Nothing is taken. +fn rung(bell: BorrowedHandle<'_>, nanos: u64) -> Result { + match syscall::handle_wait(bell.raw(), crate::poller::READABLE, nanos) { + Ok(ready) => Ok(ready != 0), + Err(_) => Err(NetError::Io), + } } /// One request's read of the bell, handed out by [`look`] and spent once: spending it gives the @@ -245,36 +235,32 @@ impl Drained { /// A holder's bell, as [`look`] reads it. trait Bell { /// `f` under the end's turn, told whether the bell is rung now. - fn look_now(&self, f: impl FnOnce(bool) -> R) -> R; + fn look_now(&self, f: impl FnOnce(bool) -> R) -> Result; /// Until the bell is rung, or for at most `nanos`, out of the turn. fn wait(&self, nanos: u64) -> Result<(), NetError>; fn drain(&self); } -/// An end's own bell: looked at with the end's poller under its turn, waited on with a poller -/// of the wait's own, which the end's other calls do not share. +/// An end's own bell: looked at under the end's turn, waited on out of it. struct EndBell { slot: usize, bell: BorrowedHandle<'static>, } impl Bell for EndBell { - fn look_now(&self, f: impl FnOnce(bool) -> R) -> R { - let turn = TURNS[self.slot].lock(); - let Some(poller) = turn.as_ref() else { unreachable!("a live end has its poller") }; - f(rung(poller, self.bell, 0)) + fn look_now(&self, f: impl FnOnce(bool) -> R) -> Result { + let _turn = TURNS[self.slot].lock(); + rung(self.bell, 0).map(f) } fn wait(&self, nanos: u64) -> Result<(), NetError> { - // Netstack gone hangs the bell up, which is readable, and the next look says so. - let poller = Poller::try_new(1).map_err(|_| NetError::ResourceExhausted)?; - rung(&poller, self.bell, nanos); - Ok(()) + // Netstack gone leaves the bell readable for good, and the next look says so. + rung(self.bell, nanos).map(drop) } fn drain(&self) { - let mut bytes = [0u8; 64]; - while let Ok(1..) = syscall::read_nonblock(self.bell.raw(), &mut bytes) {} + // A bell netstack let go of answers `Gone`, which the request it was read for answers. + let _ = syscall::event_take(self.bell.raw()); } } @@ -283,7 +269,7 @@ impl Bell for EndBell { fn look(bell: &impl Bell, wait: Wait, mut ask: impl FnMut(Drain<'_>) -> Result) -> Result { let drain = || bell.drain(); loop { - if let Some(answer) = bell.look_now(|rung| rung.then(|| ask(Drain(&drain)))) { + if let Some(answer) = bell.look_now(|rung| rung.then(|| ask(Drain(&drain))))? { match answer { Err(NetError::WouldBlock) => {} answer => return answer, @@ -331,7 +317,7 @@ pub fn bind(addr: [u8; 4], port: u16) -> Result { /// [`MAX_SOCKETS`]. fn open(kind: Kind, addr: [u8; 4], ask: impl FnOnce(NetstackConn) -> Result<(BoundResponse, [OwnedHandle; 1], Connection), NetError>) -> Result { let slot = TABLE.lock().reserve().ok_or(NetError::ResourceExhausted)?; - let made = make_poller(slot).and_then(|()| NetstackConn::connect()).and_then(ask).and_then(|(bound, [bell], connection)| { + let made = NetstackConn::connect().and_then(ask).and_then(|(bound, [bell], connection)| { let (descriptor, connection, bell) = take_up(kind, connection.into_owned(), bell)?; Ok((descriptor, End { kind, connection, bell, addr, port: bound.port })) }); @@ -342,7 +328,6 @@ fn open(kind: Kind, addr: [u8; 4], ask: impl FnOnce(NetstackConn) -> Result<(Bou Ok(Bound { handle, port: end.port }) } Err(refused) => { - *TURNS[slot].lock() = None; table.unreserve(slot); Err(refused) } @@ -541,7 +526,7 @@ fn bare(connection: BorrowedHandle<'_>) -> Result<(), NetError> { pub fn export(handle: BorrowedHandle<'_>) -> Result { call(handle, |slot, end| { let _turn = TURNS[slot].lock(); - stream::ask_export(end.connection) + stream::ask_export(end.connection, None) }) } @@ -555,7 +540,7 @@ pub(super) fn hold(handle: BorrowedHandle<'_>) -> Option { pub(super) fn export_held(slot: usize) -> Result { let end = TABLE.lock().at(slot).expect("a held end is live"); let _turn = TURNS[slot].lock(); - stream::ask_export(end.connection) + stream::ask_export(end.connection, None) } /// The call a [`hold`] began, ended. @@ -568,14 +553,13 @@ pub(super) fn release(slot: usize) { /// An exported end taken up as this process's holder of its socket, `payload` and `handles` the /// [`ADOPT`] netstack queued on it (`super::adopt_exported`): the end and its bell narrowed to /// keep no `TRANSFER`. The descriptor's handle; `None`, with everything closed, where the frame -/// is malformed, the process holds [`MAX_SOCKETS`] or the kernel makes no poller for the end or -/// will not narrow them. +/// is malformed, the process holds [`MAX_SOCKETS`] or the kernel will not narrow them. pub(super) fn adopted(connection: Connection, payload: &[u8], handles: ipc::Batch) -> Option { let adopt: Adopt = ipc::decode_payload(payload).ok()?; let [bell] = handles.exact::<1>()?; let kind = Kind::from_u32(adopt.kind)?; let slot = TABLE.lock().reserve()?; - let made = make_poller(slot).and_then(|()| take_up(kind, connection.into_owned(), bell)); + let made = take_up(kind, connection.into_owned(), bell); let mut table = TABLE.lock(); match made { Ok((descriptor, connection, bell)) => { @@ -583,7 +567,6 @@ pub(super) fn adopted(connection: Connection, payload: &[u8], handles: ipc::Batc Some(descriptor) } Err(_) => { - *TURNS[slot].lock() = None; table.unreserve(slot); None } @@ -676,9 +659,9 @@ mod tests { } impl Bell for Held<'_> { - fn look_now(&self, f: impl FnOnce(bool) -> R) -> R { + fn look_now(&self, f: impl FnOnce(bool) -> R) -> Result { let rung = self.server.borrow().bells[self.holder] > 0; - f(rung) + Ok(f(rung)) } /// Runs what the scenario says happens meanwhile; a holder still unrung when nothing is diff --git a/sdk/toyos/src/net/stream.rs b/sdk/toyos/src/net/stream.rs index 16705864a1a..94db3a7b9b8 100644 --- a/sdk/toyos/src/net/stream.rs +++ b/sdk/toyos/src/net/stream.rs @@ -10,19 +10,21 @@ //! receiving half's shutdown and the hang-up any holder saw. //! //! **A stream has holders, each a process's session**: its connection to -//! netstack and two bells netstack writes, a reader's and a writer's, so -//! neither takes the other's wake. A doorbell says "look", never "there are -//! bytes" (virtio 1.2 §2.9). The process that connected or accepted is the -//! first holder; [`export`] asks netstack for another, whose connection end a -//! spawn endows and the child takes up (`super::adopt_exported`). An accept is a -//! listener holder's request (`super::socket`), answered with the stream's -//! connection. Every holder -//! shares the one region, and the session's gates make their calls take turns -//! (`crate::transport::session`). +//! netstack, two bells netstack rings, a reader's and a writer's, so neither +//! takes the other's wake, and the waker of the bell netstack watches it on. +//! A doorbell says "look", never "there are bytes" (virtio 1.2 §2.9). The +//! process that connected or accepted is the first holder; [`export`] asks +//! netstack for another, whose connection end a spawn endows and the child +//! takes up (`super::adopt_exported`). An accept is a listener holder's +//! request (`super::socket`), answered with the stream's connection. Every +//! holder shares the one region, and the session's gates make their calls +//! take turns (`crate::transport::session`). //! //! **A holder's connection carries frames**: the request and its answer, then -//! [`RING`] and [`EXPORT`] from the holder, and netstack's answers to an -//! export. Its hang-up is the holder's leaving. +//! [`EXPORT`] from the holder and netstack's answers to it. A holder rings +//! its waker for anything netstack is to look at, and its waker's last handle +//! going is the holder's leaving; before a connect is answered, the first +//! holder holds no waker yet, and its connection's hang-up is its leaving. //! //! **A holder stays in its process.** Its connection, bells and region are //! held without `TRANSFER` once taken up, so no spawn or message can move one; @@ -46,9 +48,9 @@ use toyos_abi::handle::Rights; use toyos_abi::syscall::{self, SyscallError}; use toyos_abi::RawHandle; -use super::ends::{close_kept, kept, narrowed, Ends, HELD_BELL, HELD_CONNECTION}; +use super::ends::{close_kept, kept, narrowed, Ends, HELD_BELL, HELD_CONNECTION, HELD_RUNG}; -use crate::ipc::{self, Connection, TrySendError}; +use crate::ipc::{self, Connection}; use crate::shm::{Bytes, SharedMemory}; use crate::sync::Lock; use crate::transport::session::{self, Bells, Doorbell, Holder, HungUp, Layout, Mode, Polled, Receiving, Refused as Answer, Sending, Took, Waited}; @@ -102,11 +104,6 @@ pub type Words = [AtomicU32; WORDS]; /// netstack's end of a stream. pub type Server = session::Server; -/// What netstack watches a stream's connection for: the client's rings, and -/// its hang-up, which the connection reads as its end. A connection is no pipe -/// end, so the kernel refuses it `OTHER_END_GONE`. -pub const SERVER_WATCH: u32 = crate::poller::READABLE; - /// The receive ring's `end`: the peer's reset, or netstack's own. pub const RESET: u32 = 2; /// The receive ring's `end`: [tcp] gave the connection up, by its user timeout @@ -117,10 +114,8 @@ pub const UNREACHABLE: u32 = 4; // ---- a holder's connection ---- -/// A holder rang: it moved a cursor or armed a want netstack is to look at. -pub const RING: u32 = 1; -/// A holder asks for another holder of its stream: answered [`EXPORTED`] or -/// [`REFUSED`]. One at a time on a holder's connection. +/// A holder asks for another holder of its stream, and rings: answered +/// [`EXPORTED`] or [`REFUSED`]. One at a time on a holder's connection. pub const EXPORT: u32 = 2; /// The answer to an [`EXPORT`], carrying the new holder's connection end, /// with `TRANSFER`, on which [`ADOPT`] waits. @@ -129,12 +124,15 @@ pub const EXPORTED: u32 = 0x81; /// error code. pub const REFUSED: u32 = 0x82; /// The first frame on an exported end, carrying [`Adopt`] and, in order, the -/// region and the holder's receive and send bells. +/// region and the holder's [`BELLS`]. pub const ADOPT: u32 = 0x83; /// The bells a connect's or an accept's answer and an [`ADOPT`] carry: the -/// receive bell, then the send bell. -pub const BELLS: usize = 2; +/// receive bell and the send bell netstack rings, then the waker of the bell +/// netstack watches the holder on. A holder rings it when it moved a cursor, +/// armed a want or asked an [`EXPORT`]; its last handle going is the holder's +/// leaving. +pub const BELLS: usize = 3; crate::ipc_payload! { /// Who an exported end's holder is, and the stream's endpoints, as netstack @@ -197,14 +195,15 @@ struct End { local_port: u16, } -/// An end's region, its handle and mapping, and the bells netstack rings for -/// it, kept as [`kept`] says. +/// An end's region, its handle and mapping, the bells netstack rings for it +/// and the waker it rings netstack with, kept as [`kept`] says. #[derive(Clone, Copy, Debug, PartialEq, Eq)] struct Mapped { handle: RawHandle, base: usize, receive_bell: BorrowedHandle<'static>, send_bell: BorrowedHandle<'static>, + rung: BorrowedHandle<'static>, } /// Lets go of an end no handle or call reaches: its halves, its connection, @@ -216,7 +215,7 @@ fn free(slot: usize, end: End) { // SAFETY: the halves that read the mapping are gone, and nothing else // reaches it. let _ = unsafe { syscall::vm_release(mapped.base as *mut u8, REGION_BYTES) }; - [mapped.receive_bell.raw(), mapped.send_bell.raw(), mapped.handle] + [mapped.receive_bell.raw(), mapped.send_bell.raw(), mapped.rung.raw(), mapped.handle] }); for handle in core::iter::once(end.connection.raw()).chain(region.into_iter().flatten()) { // SAFETY: no handle and no call reaches the end, whose own these are. @@ -348,7 +347,7 @@ impl Call { fn bells(&self) -> Doorbells { let mapped = self.end.region.expect("a call is made only on an end with its region"); - Doorbells { connection: self.end.connection, receive: mapped.receive_bell, send: mapped.send_bell } + Doorbells { receive: mapped.receive_bell, send: mapped.send_bell, rung: mapped.rung } } } @@ -373,12 +372,11 @@ impl session::Lock for Half { } } -/// A call's doorbells, and the end's own handle of the connection it rings -/// netstack on. +/// A call's doorbells, and the waker it rings netstack with. struct Doorbells { - connection: BorrowedHandle<'static>, receive: BorrowedHandle<'static>, send: BorrowedHandle<'static>, + rung: BorrowedHandle<'static>, } impl Doorbells { @@ -395,38 +393,32 @@ impl Bells for Doorbells { type Until = u64; fn drain(&self, bell: Doorbell) -> Took { - let mut took = Took::Nothing; - let mut bytes = [0u8; 64]; - loop { - match syscall::read_nonblock(self.handle(bell).raw(), &mut bytes) { - Ok(0) => return Took::Gone, - Ok(_) => took = Took::Rung, - Err(SyscallError::WouldBlock) => return took, - Err(_) => return Took::Gone, - } + match syscall::event_take(self.handle(bell).raw()) { + Ok(()) => Took::Rung, + Err(SyscallError::WouldBlock) => Took::Nothing, + Err(_) => Took::Gone, } } fn wait(&self, bell: Doorbell, until: Option) -> Waited { let handle = self.handle(bell); - let mut bytes = [0u8; 64]; - let Some(until) = until else { - return match syscall::read(handle.raw(), &mut bytes) { - Ok(0) | Err(_) => Waited::Gone, - Ok(_) => Waited::Rung, + loop { + match self.drain(bell) { + Took::Rung => return Waited::Rung, + Took::Gone => return Waited::Gone, + Took::Nothing => {} + } + let left = match until { + None => u64::MAX, + Some(until) => match until.saturating_sub(toyos_abi::clock::nanos_since_boot()) { + 0 => return Waited::TimedOut, + left => left, + }, }; - }; - let left = until.saturating_sub(toyos_abi::clock::nanos_since_boot()); - if left > 0 { - let poller = crate::poller::Poller::new(1); - poller.watch(&handle, crate::poller::READABLE, 0); - poller.wait(1, left, |_| {}); - } - match syscall::read_nonblock(handle.raw(), &mut bytes) { - Ok(0) => Waited::Gone, - Ok(_) => Waited::Rung, - Err(SyscallError::WouldBlock) => Waited::TimedOut, - Err(_) => Waited::Gone, + // Ready or the deadline: the take above says which. + if syscall::handle_wait(handle.raw(), crate::poller::READABLE, left).is_err() { + return Waited::Gone; + } } } @@ -438,11 +430,9 @@ impl Bells for Doorbells { } fn ring(&self) -> Result<(), HungUp> { - match ipc::try_signal(self.connection, RING) { - Err(TrySendError::Syscall(SyscallError::Gone)) => Err(HungUp), - // Frames netstack has not read yet are a ring already, and - // anything else is no hang-up: the next call's wait answers it. - _ => Ok(()), + match syscall::event_signal(self.rung.raw()) { + Ok(()) => Ok(()), + Err(_) => Err(HungUp), } } } @@ -761,8 +751,8 @@ pub(super) fn accept(listener: BorrowedHandle<'_>, drain: super::socket::Drain<' let header = answer(listener)?; let accepted: TcpAcceptResponse = ipc::recv_payload(listener, &header).map_err(hangup)?; let handles = ipc::recv_handles(listener, &header).map_err(hangup)?; - let [connection, receive, send] = handles.exact::<3>().ok_or(NetError::Protocol(header.msg_type))?; - Ok((accepted, [receive, send], Connection::received(connection))) + let [connection, receive, send, rung] = handles.exact::<{ BELLS + 1 }>().ok_or(NetError::Protocol(header.msg_type))?; + Ok((accepted, [receive, send, rung], Connection::received(connection))) })?; let answer = opened.answer; Ok(Accepted { @@ -855,6 +845,7 @@ struct Held { connection: RawHandle, receive_bell: RawHandle, send_bell: RawHandle, + rung: RawHandle, region: RawHandle, } @@ -862,8 +853,9 @@ impl Held { /// The end, its region mapped at `base`. fn end(&self, base: usize, peer: ([u8; 4], u16), local_port: u16) -> End { // SAFETY: the handles `take_up` narrowed for this end, which keeps them until its `free`. - let (receive_bell, send_bell, connection) = unsafe { (kept(self.receive_bell), kept(self.send_bell), kept(self.connection)) }; - let region = Mapped { handle: self.region, base, receive_bell, send_bell }; + let (receive_bell, send_bell, rung, connection) = + unsafe { (kept(self.receive_bell), kept(self.send_bell), kept(self.rung), kept(self.connection)) }; + let region = Mapped { handle: self.region, base, receive_bell, send_bell, rung }; End { region: Some(region), connection, peer, local_port } } } @@ -881,17 +873,18 @@ pub(super) fn hold_connection(connection: OwnedHandle) -> Result<(OwnedHandle, R /// as [`Held`]. Refused, the region is unmapped and every handle closed. fn take_up(connection: OwnedHandle, bells: [OwnedHandle; BELLS], region: OwnedHandle, base: *mut u8) -> Result { let connection = hold_connection(connection); - let [receive, send] = bells.map(|bell| narrowed(bell, HELD_BELL)); + let [receive, send, rung] = bells; + let (receive, send, rung) = (narrowed(receive, HELD_BELL), narrowed(send, HELD_BELL), narrowed(rung, HELD_RUNG)); let region = narrowed(region, HELD_REGION); - match (connection, receive, send, region) { - (Ok((descriptor, connection)), Ok(receive_bell), Ok(send_bell), Ok(region)) => { - Ok(Held { descriptor, connection, receive_bell, send_bell, region }) + match (connection, receive, send, rung, region) { + (Ok((descriptor, connection)), Ok(receive_bell), Ok(send_bell), Ok(rung), Ok(region)) => { + Ok(Held { descriptor, connection, receive_bell, send_bell, rung, region }) } - (connection, receive, send, region) => { + (connection, receive, send, rung, region) => { // SAFETY: nothing has read the mapping through this end yet, and // nothing else reaches it. let _ = unsafe { syscall::vm_release(base, REGION_BYTES) }; - let kept = connection.map(|(_, own)| own).into_iter().chain(receive).chain(send).chain(region); + let kept = connection.map(|(_, own)| own).into_iter().chain(receive).chain(send).chain(rung).chain(region); // SAFETY: each narrowed here and kept by nothing yet. kept.for_each(|handle| unsafe { close_kept(handle) }); Err(NetError::Io) @@ -909,7 +902,8 @@ fn take_up(connection: OwnedHandle, bells: [OwnedHandle; BELLS], region: OwnedHa pub fn export(handle: BorrowedHandle<'_>) -> Result { let asked = call(handle, |call| { let _turn = call.halves().control.lock(); - ask_export(call.end.connection) + let mapped = call.end.region.expect("a call is made only on an end with its region"); + ask_export(call.end.connection, Some(mapped.rung)) }); match asked { Ok(answer) => answer, @@ -931,7 +925,7 @@ pub(super) fn export_held(slot: usize) -> Result { return Err(NetError::NetstackNotFound); } let _turn = HALVES[slot].control.lock(); - ask_export(end.connection) + ask_export(end.connection, end.region.map(|mapped| mapped.rung)) } /// The call a [`hold`] began, ended. @@ -940,9 +934,13 @@ pub(super) fn release(slot: usize) { } /// [`EXPORT`] on a holder's connection, a stream's or a socket's, and -/// netstack's answer to it. -pub(super) fn ask_export(connection: BorrowedHandle<'_>) -> Result { +/// netstack's answer to it. A stream's holder rings its `rung` after the +/// frame, since netstack watches that and not the connection. +pub(super) fn ask_export(connection: BorrowedHandle<'_>, rung: Option>) -> Result { ipc::signal(connection, EXPORT).map_err(hangup)?; + if let Some(rung) = rung { + syscall::event_signal(rung.raw()).map_err(|_| NetError::NetstackNotFound)?; + } let header = ipc::recv_header(connection).map_err(hangup)?; match header.msg_type { EXPORTED => { @@ -969,12 +967,12 @@ pub(super) fn ask_export(connection: BorrowedHandle<'_>) -> Result) -> Option { let slot = TABLE.lock().reserve()?; - let taken = first.and_then(adoption).and_then(|(holder, adopt, [region, receive, send])| { + let taken = first.and_then(adoption).and_then(|(holder, adopt, [region, receive, send, rung])| { let region = SharedMemory::lent(region, REGION_BYTES).ok()?; let (region, base) = region.into_raw(); // SAFETY: `into_raw` gave this the region's handle, which nothing else answers for. let region = unsafe { OwnedHandle::from_raw(region) }; - Some((holder, adopt, base, [receive, send], region)) + Some((holder, adopt, base, [receive, send, rung], region)) }); let made = match taken { Some((holder, adopt, base, bells, region)) => take_up(connection.into_owned(), bells, region, base).map(|held| { diff --git a/sdk/toyos/src/poller.rs b/sdk/toyos/src/poller.rs index 6f8a7edada4..fc278ad4b76 100644 --- a/sdk/toyos/src/poller.rs +++ b/sdk/toyos/src/poller.rs @@ -11,6 +11,15 @@ use crate::AsHandle; pub use toyos_abi::inbox::{OTHER_END_GONE, READABLE, WRITABLE}; +/// Park until `handle` answers the one condition `interest` names +/// ([`READABLE`], [`WRITABLE`] or [`OTHER_END_GONE`]), or `timeout_nanos` +/// passes (`u64::MAX`: for good): `true` for the condition, `false` at the +/// timeout. The answer is the one a poller's watch gets, with no inbox made +/// for it ([`syscall::handle_wait`]). +pub fn wait_one(handle: &impl AsHandle, interest: u32, timeout_nanos: u64) -> Result { + syscall::handle_wait(handle.as_handle().raw(), interest, timeout_nanos).map(|bit| bit != 0) +} + /// The inbox page, and the only thing in this crate that touches it. /// /// **The kernel is the second writer of every byte below, so no Rust reference diff --git a/sdk/toyos/src/wake.rs b/sdk/toyos/src/wake.rs index 36cfcbe9ad0..ce4078ddd18 100644 --- a/sdk/toyos/src/wake.rs +++ b/sdk/toyos/src/wake.rs @@ -1,79 +1,142 @@ -//! A wake from any thread to the one thread that waits for it. +//! A wake from any thread or process to whoever holds the bell. //! -//! A pipe whose bytes say nothing but "woken": a [`Waker`] writes one without -//! waiting, and the [`Bell`] is watched in a [`Poller`](crate::poller::Poller) -//! beside the thread's other handles, or waited on alone. Wakes coalesce, so -//! any number raised before the bell is taken are one. +//! An event object: a [`Waker`] raises it without waiting, and the [`Bell`] is +//! watched in a [`Poller`](crate::poller::Poller) beside the holder's other +//! handles, or waited on alone. Wakes coalesce, so any number raised before +//! the bell is taken are one. A readable watch takes nothing. //! //! Take the bell before reading whatever the wakers announce: a wake raised //! after the take is then still pending at the next wait, and one raised //! before it is never lost. +//! +//! Each side sees the other's last handle go: a bell whose every waker is gone +//! is readable for good and its take answers [`Hungup`], never a wake; a +//! waker whose bell is gone rings into [`Hungup`]. + +use toyos_abi::syscall::{self, SyscallError}; -use toyos_abi::syscall::SyscallError; +use crate::{AsHandle, BorrowedHandle, OwnedHandle}; -use crate::{pipe_pair, AsHandle, Pipe}; +/// The other side's last handle has gone. +#[derive(Clone, Copy, PartialEq, Eq, Debug)] +pub struct Hungup; -/// A fresh bell and the waker that rings it. One wake is written and taken -/// here, so the pipe has held a byte and no later wake into an empty bell is -/// refused for memory (`toyos_abi::syscall::PIPE_CAPACITY`). +/// A fresh bell and the waker that rings it. pub fn pair() -> Result<(Waker, Bell), SyscallError> { - let (read, write) = pipe_pair()?; - write.write_nonblock(&[1])?; - let bell = Bell(read); - bell.take(); - Ok((Waker(write), bell)) + let ends = syscall::event_create()?; + Ok((Waker(OwnedHandle(ends.signal)), Bell(OwnedHandle(ends.wait)))) } /// Rings a [`Bell`] from any thread, and never waits. -pub struct Waker(Pipe); +pub struct Waker(OwnedHandle); impl Waker { + /// Ring, whether or not anyone still holds the bell. pub fn wake(&self) { - match self.0.write_nonblock(&[1]) { - // A full pipe is a wake already pending, and so is one refused - // memory, since a bell `pair` primed takes none while it is empty; - // a bell nobody holds any more has nobody left to wake. - Ok(_) | Err(SyscallError::WouldBlock) | Err(SyscallError::ResourceExhausted) | Err(SyscallError::Gone) => {} - Err(e) => panic!("Waker::wake: the wake pipe refused a write: {e:?}"), + let _ = self.ring(); + } + + /// Ring; [`Hungup`] once nobody holds the bell. + pub fn ring(&self) -> Result<(), Hungup> { + match syscall::event_signal(self.0.raw()) { + Ok(()) => Ok(()), + Err(SyscallError::Gone) => Err(Hungup), + Err(e) => panic!("Waker::ring: the event refused a signal: {e:?}"), } } + + /// Another waker of the same bell, to hand on. + pub fn try_clone(&self) -> Result { + self.0.try_clone().map(Waker) + } } -/// The receiving end: readable while a wake is pending. -pub struct Bell(Pipe); +/// The receiving end: readable while a wake is pending or every waker is gone. +pub struct Bell(OwnedHandle); impl Bell { /// Take every wake raised so far, and say whether there was one. /// - /// A bell whose every [`Waker`] is gone reads as ended, readable for good, - /// and a wait on it would spin: that is refused here by name. + /// For a bell whose wakers are this process's own: one whose every waker + /// is gone is readable for good, and a wait on it would spin, so that is + /// refused here by name. pub fn take(&self) -> bool { - let mut buf = [0u8; 64]; - let mut woken = false; - loop { - match self.0.read_nonblock(&mut buf) { - Ok(0) => panic!("Bell::take: every waker is gone"), - Ok(_) => woken = true, - Err(SyscallError::WouldBlock) => return woken, - Err(e) => panic!("Bell::take: the wake pipe refused a read: {e:?}"), - } + self.try_take().expect("Bell::take: every waker is gone") + } + + /// [`take`](Self::take), answering a bell whose every waker is gone. + pub fn try_take(&self) -> Result { + match syscall::event_take(self.0.raw()) { + Ok(()) => Ok(true), + Err(SyscallError::WouldBlock) => Ok(false), + Err(SyscallError::Gone) => Err(Hungup), + Err(e) => panic!("Bell::take: the event refused a take: {e:?}"), } } - /// Block until a wake is pending, and take it with every other one. + /// Block until a wake is pending, and take it. pub fn wait(&self) { - let mut buf = [0u8; 64]; - match self.0.read(&mut buf) { - Ok(0) => panic!("Bell::wait: every waker is gone"), - Ok(_) => {} - Err(e) => panic!("Bell::wait: the wake pipe refused a read: {e:?}"), + let woken = self.wait_for(u64::MAX).expect("Bell::wait: every waker is gone"); + assert!(woken, "Bell::wait: a wait with no timeout returned with no wake"); + } + + /// Wait at most `timeout_nanos` (`u64::MAX`: for good) for a wake and take + /// it: `true` for one taken, `false` at the timeout. + pub fn wait_for(&self, timeout_nanos: u64) -> Result { + let until = toyos_abi::clock::nanos_since_boot().saturating_add(timeout_nanos); + loop { + if self.try_take()? { + return Ok(true); + } + let left = match timeout_nanos { + u64::MAX => u64::MAX, + _ => until.saturating_sub(toyos_abi::clock::nanos_since_boot()), + }; + if left == 0 { + return Ok(false); + } + match syscall::handle_wait(self.0.raw(), crate::poller::READABLE, left) { + // Ready, or the timeout: the take above says which. + Ok(_) => {} + Err(e) => panic!("Bell::wait_for: the event refused a wait: {e:?}"), + } } - self.take(); + } +} + +impl AsHandle for Waker { + fn as_handle(&self) -> BorrowedHandle<'_> { + self.0.as_handle() } } impl AsHandle for Bell { - fn as_handle(&self) -> crate::BorrowedHandle<'_> { + fn as_handle(&self) -> BorrowedHandle<'_> { self.0.as_handle() } } + +/// A side a peer sent: its type is the sender's word, as a connection's is. +impl From for Waker { + fn from(handle: OwnedHandle) -> Self { + Self(handle) + } +} + +impl From for Bell { + fn from(handle: OwnedHandle) -> Self { + Self(handle) + } +} + +impl From for OwnedHandle { + fn from(waker: Waker) -> Self { + waker.0 + } +} + +impl From for OwnedHandle { + fn from(bell: Bell) -> Self { + bell.0 + } +} diff --git a/sdk/toyos/tests/transport_loom.rs b/sdk/toyos/tests/transport_loom.rs index 005e1e65712..ae82d0d8e71 100644 --- a/sdk/toyos/tests/transport_loom.rs +++ b/sdk/toyos/tests/transport_loom.rs @@ -340,7 +340,7 @@ type Rx = Receiving; type Tx = Sending; type End = Server; -/// A one-byte doorbell pipe. +/// A doorbell that coalesces, as an event does. struct Pipe { level: Mutex, posted: Condvar, @@ -366,9 +366,8 @@ impl Pipe { self.posted.notify_all(); } - /// A holder's ring of the server: a byte that says it is a holder's, as a - /// `RING` frame does, so the server that takes it looks at the words after - /// the holder wrote them. + /// A holder's ring of the server, as its rung bell is, so the server that + /// takes it looks at the words after the holder wrote them. fn ring_as_holder(&self) { self.level.lock().unwrap().rang = true; self.ring(); diff --git a/sdk/window/src/wait.rs b/sdk/window/src/wait.rs index c1cabf5d79b..2a7febe7286 100644 --- a/sdk/window/src/wait.rs +++ b/sdk/window/src/wait.rs @@ -48,9 +48,9 @@ impl Waiter { /// A waiter with room for a few windows; it grows when asked to wait on /// more. pub fn new() -> Self { - // As `Poller::new` does with its inbox: a kernel out of pipes is a + // As `Poller::new` does with its inbox: a kernel out of events is a // machine this process cannot wait on at all. - let (waker, bell) = wake::pair().expect("Waiter::new: no pipe for the wake"); + let (waker, bell) = wake::pair().expect("Waiter::new: no event for the wake"); let capacity = 4; Self { poller: Poller::new(capacity), capacity, bell, waker: Arc::new(waker) } } diff --git a/system/compositor/src/signout.rs b/system/compositor/src/signout.rs index aa526ca0b30..849548d3aca 100644 --- a/system/compositor/src/signout.rs +++ b/system/compositor/src/signout.rs @@ -25,7 +25,8 @@ use toyos::ipc::{self, Connection, RxStep}; use toyos::job::JOB_LABEL; use toyos::namespace::Namespace; use toyos::syscap::SysCap; -use toyos::{AsHandle, Pipe}; +use toyos::wake::{self, Bell, Waker}; +use toyos::AsHandle; use toyos_abi::syscall::SYSCAP_LABEL; use toyos_desktop::Opening; use toyos_session::{self as wire, Names, SignOut}; @@ -56,7 +57,7 @@ pub struct Desktop { /// The answers to the thread's opens: a control connection, or why none. opened: mpsc::Receiver>, /// Readable once per answer in `opened`. - wake: Pipe, + wake: Bell, opening: Opening, } @@ -66,7 +67,7 @@ impl Desktop { let endowed = Endowments::get(); let (asks, queue) = mpsc::channel(); let (sent, opened) = mpsc::channel(); - let (wake, woken) = toyos::pipe_pair().expect("compositor: no pipe for its session's opens"); + let (woken, wake) = wake::pair().expect("compositor: no event for its session's opens"); let login = endowed.take::(wire::LOGIN_PORT); std::thread::Builder::new() .name("launcher".into()) @@ -98,7 +99,7 @@ impl Desktop { } /// What the loop watches for an open made. - pub fn wake(&self) -> &Pipe { + pub fn wake(&self) -> &Bell { &self.wake } @@ -119,8 +120,8 @@ impl Desktop { /// A session the thread opened, now the one this compositor signs out: /// `true` where there is one. pub fn opened(&mut self) -> bool { - let mut drained = [0u8; 16]; - while self.wake.read_nonblock(&mut drained).is_ok_and(|n| n > 0) {} + // Its one waker is the launcher thread's, which ends only with this. + self.wake.try_take().expect("compositor: its launcher thread has ended"); let mut took = false; while let Ok(answer) = self.opened.try_recv() { self.opening.answered(); @@ -219,7 +220,7 @@ impl Desktop { /// **Everything the desktop starts is a job of its own**, made with the /// `jobs` right this row's system capability carries: a program started from /// the taskbar is no helper of the compositor's, and must not share its CPU. -fn launcher(queue: mpsc::Receiver, login: Option, opened: mpsc::Sender>, wake: Pipe) { +fn launcher(queue: mpsc::Receiver, login: Option, opened: mpsc::Sender>, wake: Waker) { let jobs = endow::Endowments::get().take::(SYSCAP_LABEL); // A session this thread opened, whose launcher replaces the endowed one. let mut session: Option = None; @@ -235,7 +236,7 @@ fn launcher(queue: mpsc::Receiver, login: Option, opened: mpsc:: None => Err("this row holds no `login` connector".to_string()), }; if opened.send(answer).is_ok() { - let _ = wake.write_nonblock(&[1]); + wake.wake(); } continue; } diff --git a/system/diskserver/src/session.rs b/system/diskserver/src/session.rs index 587715fd4a2..e4eb518ce54 100644 --- a/system/diskserver/src/session.rs +++ b/system/diskserver/src/session.rs @@ -2,7 +2,7 @@ //! `toyos-blockring`. //! //! **What a request means is `toyos_blockring::client::Client`'s to decide**; -//! this is the mapping, the arena, the doorbell and the connection around it. +//! this is the mapping, the arena and the doorbells around it. //! A write's data is copied into the arena and stays there until a flush //! covers it, because after a loss it is the only copy there is; a read's is //! copied out when it is answered. @@ -19,7 +19,8 @@ use std::collections::BTreeMap; use toyos::ipc::Connection; use toyos::namespace::Namespace; -use toyos::poller::{Poller, READABLE}; +use toyos::poller::READABLE; +use toyos::wake::{Bell, Waker}; use toyos_abi::syscall::SyscallError; use toyos_blockring::client::{Client, Outcome, Ticket}; use toyos_blockring::entry::{Completion, Op}; @@ -119,14 +120,14 @@ pub struct Session { service: String, guid: [u8; wire::GUID_BYTES], region: Region, - conn: Option, + /// The open session's doorbells, `None` once it ended. + bells: Option, rings: ClientRings, client: Client, opened: Opened, arena: Arena, next_ticket: Ticket, pending: BTreeMap, - poller: Poller, /// The most requests this session has had on the wire at once. peak: usize, } @@ -137,7 +138,7 @@ impl Session { pub fn open(names: Namespace, service: &str, guid: [u8; wire::GUID_BYTES]) -> Result { let region = Region::create().map_err(Error::Kernel)?; let rings = layout::client(region.words()); - let (conn, opened) = handshake(&names, service, guid, ®ion)?; + let (bells, opened) = handshake(&names, service, guid, ®ion)?; let mut client = Client::new(); client.session_started(); Ok(Self { @@ -145,14 +146,13 @@ impl Session { service: service.to_string(), guid, region, - conn: Some(conn), + bells: Some(bells), rings, client, opened, arena: Arena::new(), next_ticket: 0, pending: BTreeMap::new(), - poller: Poller::new(1), peak: 0, }) } @@ -181,7 +181,7 @@ impl Session { /// Ask for `data` (whole blocks) written at `lba`. pub fn submit_write(&mut self, lba: u64, data: &[u8]) -> Result { let blocks = whole_blocks(data.len()); - if self.conn.is_none() { + if self.bells.is_none() { return Err(Unsent::Ended); } let run = self.arena.alloc(blocks).ok_or(Unsent::ArenaFull)?; @@ -195,7 +195,7 @@ impl Session { /// Ask for `blocks` read from `lba`; the answer carries the data. pub fn submit_read(&mut self, lba: u64, blocks: u32) -> Result { assert!((1..=MAX_REQUEST_BLOCKS).contains(&blocks), "a read of {blocks} blocks"); - if self.conn.is_none() { + if self.bells.is_none() { return Err(Unsent::Ended); } let run = self.arena.alloc(blocks).ok_or(Unsent::ArenaFull)?; @@ -207,7 +207,7 @@ impl Session { /// Ask for every write acknowledged so far to be made durable. pub fn submit_flush(&mut self) -> Result { - if self.conn.is_none() { + if self.bells.is_none() { return Err(Unsent::Ended); } let ticket = self.ticket(); @@ -219,15 +219,14 @@ impl Session { /// Put what the client will send on the ring, and ring the doorbell once /// for all of it. fn pump(&mut self) { - let Some(conn) = &self.conn else { return }; + let Some(bells) = &self.bells else { return }; let page = self.region.words(); loop { match self.rings.0.space(page) { Ok(0) => break, Ok(_) => {} // The server wrote an index no server writes: the session is - // over, and the next read of the connection is where that is - // acted on. + // over, and the next drain is where that is acted on. Err(_) => break, } let Some(request) = self.client.next_request() else { break }; @@ -236,9 +235,8 @@ impl Session { } self.peak = self.peak.max(self.client.on_the_wire().count()); if self.rings.0.publish(page) { - // A full pipe is a doorbell already rung; a gone one is a server - // that has ended, which the wait finds. - let _ = conn.write_nonblock(&[1]); + // A server that has ended is found by the wait. + bells.ring.wake(); } } @@ -246,7 +244,7 @@ impl Session { /// or the service has ended. pub fn wait(&mut self, block: bool) -> Waited { let mut waited = Waited::default(); - if self.conn.is_none() { + if self.bells.is_none() { waited.ended = true; return waited; } @@ -265,13 +263,11 @@ impl Session { if !waited.answers.is_empty() || !block { return waited; } - let Some(conn) = &self.conn else { + let Some(bells) = &self.bells else { waited.ended = true; return waited; }; - self.poller.watch(conn, READABLE, 0); - self.poller.wait(1, u64::MAX, |_| {}); - if !self.take_doorbells() { + if bells.answered.wait_for(u64::MAX).is_err() { // What the server posted before it went is real: read it // before the rest is answered refused. self.drain(); @@ -306,20 +302,6 @@ impl Session { violated } - /// Consume the doorbell bytes; `false` if the server has hung up. - fn take_doorbells(&mut self) -> bool { - let Some(conn) = &self.conn else { return false }; - let mut sink = [0u8; 64]; - loop { - match conn.read_nonblock(&mut sink) { - Ok(0) => return false, - Ok(_) => continue, - Err(SyscallError::WouldBlock) => return true, - Err(_) => return false, - } - } - } - fn collect(&mut self, answers: &mut Vec) { let decided: Vec<_> = self.client.take_answers().collect(); for (ticket, outcome) in decided { @@ -341,7 +323,7 @@ impl Session { } fn end(&mut self, waited: &mut Waited) { - self.conn = None; + self.bells = None; self.client.session_ended(); self.collect(&mut waited.answers); waited.ended = true; @@ -354,15 +336,15 @@ impl Session { /// name whose server is being restarted queues the connection until the /// new one accepts it. pub fn reconnect(&mut self) -> Result<(), Error> { - assert!(self.conn.is_none(), "diskserver: reconnect while a session is open"); + assert!(self.bells.is_none(), "diskserver: reconnect while a session is open"); // Before the region goes to the new server: it must find this end's // two indices at zero, as it will set its own. self.rings = layout::client(self.region.words()); - let (conn, opened) = handshake(&self.names, &self.service, self.guid, &self.region)?; + let (bells, opened) = handshake(&self.names, &self.service, self.guid, &self.region)?; if opened != self.opened { return Err(Error::Protocol); } - self.conn = Some(conn); + self.bells = Some(bells); self.client.session_started(); self.pump(); Ok(()) @@ -386,30 +368,40 @@ pub(crate) fn connect(names: &Namespace, service: &str) -> Result Result { - let poller = Poller::new(1); - poller.watch(conn, READABLE, 0); - let mut ready = false; - poller.wait(1, wire::ANSWER_BOUND_NS, |_| ready = true); - if !ready { + if !toyos::poller::wait_one(conn, READABLE, wire::ANSWER_BOUND_NS).map_err(Error::Kernel)? { return Err(Error::Unanswered); } conn.recv_header().map_err(|_| Error::Ended) } -/// Connect, send the region with an open, and wait for the answer. +/// An open session's two doorbells, which the server made and sent with its +/// answer: the one this end rings for requests, and the one it rings for +/// answers, whose waker going is the server's end. +struct Bells { + ring: Waker, + answered: Bell, +} + +/// Connect, send the region with an open, and wait for the answer, which +/// carries the session's doorbells: the session is theirs from here, and the +/// connection goes. fn handshake( names: &Namespace, service: &str, guid: [u8; wire::GUID_BYTES], region: &Region, -) -> Result<(Connection, Opened), Error> { +) -> Result<(Bells, Opened), Error> { let conn = connect(names, service)?; conn.send_bytes_with(wire::MSG_OPEN, &guid, [region.share().map_err(Error::Kernel)?]).map_err(|_| Error::Ended)?; let header = answer(&conn)?; let mut payload = [0u8; Opened::BYTES]; let len = conn.recv_bytes(&header, &mut payload).map_err(|_| Error::Ended)?; match header.msg_type { - wire::MSG_OPENED => Opened::decode(&payload[..len]).map(|o| (conn, o)).ok_or(Error::Protocol), + wire::MSG_OPENED => { + let [ring, answered] = conn.recv_handles_exact::<{ wire::OPENED_HANDLES }>(&header).ok_or(Error::Protocol)?; + let opened = Opened::decode(&payload[..len]).ok_or(Error::Protocol)?; + Ok((Bells { ring: Waker::from(ring), answered: Bell::from(answered) }, opened)) + } wire::MSG_REFUSED => Err(Refusal::decode(&payload[..len]).map_or(Error::Protocol, Error::Refused)), _ => Err(Error::Protocol), } diff --git a/system/hda/src/main.rs b/system/hda/src/main.rs index 88986e75558..67df408d48f 100644 --- a/system/hda/src/main.rs +++ b/system/hda/src/main.rs @@ -439,10 +439,8 @@ const OFFER_ANSWER: Duration = Duration::from_secs(30); fn offered(control: &Connection, offer: &Offer) -> Result { let refused = |why: String| Refusal::Soundserver(why); control.send_bytes(MSG_OFFER, &offer.encode()).map_err(|why| refused(format!("the offer was refused ({why:?})")))?; - let poller = Poller::new(1); - poller.watch(control, READABLE, TOKEN_CONTROL); - let mut answered = false; - poller.wait(1, OFFER_ANSWER.as_nanos() as u64, |_| answered = true); + let answered = toyos::poller::wait_one(control, READABLE, OFFER_ANSWER.as_nanos() as u64) + .map_err(|why| refused(format!("its answer could not be waited for ({why:?})")))?; if !answered { return Err(refused(format!("it did not answer the offer within {OFFER_ANSWER:?}"))); } diff --git a/system/inspect/src/lib.rs b/system/inspect/src/lib.rs index dbdf1019212..26b6a915dab 100644 --- a/system/inspect/src/lib.rs +++ b/system/inspect/src/lib.rs @@ -6,7 +6,7 @@ use std::time::{Duration, Instant}; use toyos::endow::{self, EndowError}; use toyos::ipc::{self, FrameRx, RxStep}; -use toyos::poller::{Poller, READABLE}; +use toyos::poller::{wait_one, READABLE}; use toyos_inspect::{Owner, Value, MAX_SNAPSHOT_BYTES, MSG_INSPECT, MSG_SNAPSHOT}; /// How long one owner has to answer. @@ -34,7 +34,6 @@ pub fn ask(owner: Owner) -> Result, String> { conn.signal(MSG_INSPECT) .map_err(|e| format!("`{}` would not take the request ({e:?})", owner.port))?; - let poller = Poller::new(1); let mut rx: Box> = Box::new(FrameRx::new()); let deadline = Instant::now() + ANSWER_BOUND; loop { @@ -61,7 +60,8 @@ pub fn ask(owner: Owner) -> Result, String> { if left.is_zero() { return Err(format!("`{}` did not answer within {ANSWER_BOUND:?}", owner.port)); } - poller.watch(&conn, READABLE, 0); - poller.wait(1, left.as_nanos() as u64, |_| {}); + // Ready or the deadline: the next pump and this check say which. + wait_one(&conn, READABLE, left.as_nanos() as u64) + .map_err(|e| format!("the kernel refused a wait on `{}`'s connection ({e:?})", owner.port))?; } } diff --git a/system/logkeeper/src/main.rs b/system/logkeeper/src/main.rs index 6f451ce9012..b079826efbd 100644 --- a/system/logkeeper/src/main.rs +++ b/system/logkeeper/src/main.rs @@ -87,7 +87,8 @@ use toyos::poller::{Poller, READABLE, WRITABLE}; use toyos::port::Acceptor; use toyos::say; use toyos::syscap::SysCap; -use toyos::{Console, Pipe}; +use toyos::wake::Bell; +use toyos::Console; use toyos_abi::syscall::SyscallError; use toyos_logstream::{ ProgramLine, Registration, Tag, CONSOLE, FLUSH, FLUSHED, LOG_OPENED, MAX_TAG, ORIGINS, REGISTER, RESUME, @@ -323,7 +324,7 @@ impl Log { let Some([ring, alive]) = self.supervisor_rx.take_handles::<2>() else { panic!("logkeeper: the supervisor registered {name:?} with no ring"); }; - self.take(registration.tag, registration.pid, ring, Pipe::from(alive)); + self.take(registration.tag, registration.pid, ring, Bell::from(alive)); } // **The flush runs before the next frame is read**: the supervisor sends // RESUME when the stop it flushed for was refused, which can be @@ -338,7 +339,7 @@ impl Log { } /// A ring the supervisor registered, read from the next round on. - fn take(&mut self, tag: Tag<'_>, pid: u32, ring: toyos::OwnedHandle, alive: Pipe) { + fn take(&mut self, tag: Tag<'_>, pid: u32, ring: toyos::OwnedHandle, alive: Bell) { if self.origins.len() >= MAX_ORIGINS { toyos::warn!( "logkeeper: refusing {}'s ring: {MAX_ORIGINS} programs' rings are already read", @@ -401,19 +402,16 @@ impl Log { let oldest = read.iter().map(|said| said.at_ns).min(); // A program's end, after its reads: what its writers left is swept - // whole. A byte on a pipe nothing writes is the program's own doing - // and not its end. + // whole. A ring of a bell nothing rings is the program's own doing and + // not its end. ended.sort_unstable(); ended.dedup(); let mut gone: Vec = Vec::new(); let count = self.origins.len(); for &i in ended.iter().filter(|&&i| i < count) { let origin = &mut self.origins[i]; - let mut probe = [0u8; 1]; - match origin.alive.read_nonblock(&mut probe) { - Ok(0) => {} - Ok(_) | Err(SyscallError::WouldBlock) => continue, - Err(e) => panic!("logkeeper: {}'s end refused a read: {e:?}", origin.tag), + if origin.alive.try_take().is_ok() { + continue; } let (abandoned, ahead) = origin.sweep(i, &mut read); if ahead > 0 { diff --git a/system/logkeeper/src/origin.rs b/system/logkeeper/src/origin.rs index 37d8b6cfa2f..0a33d4e1996 100644 --- a/system/logkeeper/src/origin.rs +++ b/system/logkeeper/src/origin.rs @@ -23,7 +23,7 @@ use toyos::log::region::{Body, Ring, LANES, RING_BYTES}; use toyos::log::ring::Reader; use toyos::log::Severity; use toyos::shm::SharedMemory; -use toyos::Pipe; +use toyos::wake::Bell; use toyos_elide::limit::{Admit, Limit}; use toyos_logstream::Tag; @@ -152,7 +152,7 @@ pub struct Origin { /// The mapping; the view below is over it. _region: SharedMemory, ring: Ring, - pub alive: Pipe, + pub alive: Bell, shared: Reader, lanes: [Reader; LANES], /// The allowance of the process the supervisor started, and of every other writer. @@ -163,7 +163,7 @@ pub struct Origin { impl Origin { /// Map a ring the supervisor sent, under the name and pid it sent with it. - pub fn open(tag: Tag<'_>, pid: u32, ring: toyos::OwnedHandle, alive: Pipe) -> Result { + pub fn open(tag: Tag<'_>, pid: u32, ring: toyos::OwnedHandle, alive: Bell) -> Result { let region = SharedMemory::lent(ring, RING_BYTES) .map_err(|why| format!("{}'s ring is refused: {why}", tag.as_str()))?; let base = core::ptr::NonNull::new(region.as_ptr()).ok_or("a mapped ring at null")?; diff --git a/system/netstack/src/bells.rs b/system/netstack/src/bells.rs new file mode 100644 index 00000000000..5b7e866c994 --- /dev/null +++ b/system/netstack/src/bells.rs @@ -0,0 +1,23 @@ +//! A holder's bell of a listener or a datagram socket, as the node's [`Bell`]: netstack's +//! waker of an event it made, whose bell only that holder is handed. +//! +//! **A ring never waits and is never retried.** A holder whose bell is gone has a connection +//! whose hang-up reports its leaving. + +use toyos::wake::{self, Bell as Handed, Waker}; +use toyos_abi::syscall::SyscallError; +use toyos_net_node::Bell; + +pub struct Rung(Waker); + +impl Bell for Rung { + fn ring(&mut self) { + self.0.wake(); + } +} + +/// A new bell: the side to hand its holder, and the side the node rings. +pub fn bell() -> Result<(Handed, Rung), SyscallError> { + let (waker, bell) = wake::pair()?; + Ok((bell, Rung(waker))) +} diff --git a/system/netstack/src/lent.rs b/system/netstack/src/lent.rs index c2e88e94e52..d3c35e3fb55 100644 --- a/system/netstack/src/lent.rs +++ b/system/netstack/src/lent.rs @@ -22,7 +22,8 @@ use toyos::ipc::FrameRx; use toyos::net::stream::{Words, RECEIVE_BYTES, RECEIVE_RING, REGION_BYTES, SEND_BYTES, SEND_RING}; use toyos::shm::{self, SharedMemory}; use toyos::transport::session::{Holder, Rings}; -use toyos::{Connection, OwnedHandle, Pipe}; +use toyos::wake::{Bell, Waker}; +use toyos::{Connection, OwnedHandle}; use toyos_abi::syscall::SyscallError; use toyos_net_node::Session; use toyos_net_tcp::Lent; @@ -74,14 +75,15 @@ impl Lent for Ring { /// One holder of a stream, as netstack keeps it. pub struct Holding { - /// What it rings and asks on, and whose hang-up is its leaving. + /// What it asks on. pub connection: Connection, - /// Its frames, each bare: rings and exports. + /// Its frames, each a bare export. pub frames: FrameRx<0>, - /// The two bells' write ends, rung for a reader and for a writer: the holder holds the read - /// ends. - pub receive: Pipe, - pub send: Pipe, + /// The wakers of the two bells the holder holds, rung for a reader and for a writer. + pub receive: Waker, + pub send: Waker, + /// The bell the holder rings, whose wakers going is its leaving. + pub rung: Bell, } /// A stream's holders by number, and a handle of its region for the holders netstack makes. @@ -134,15 +136,15 @@ impl Session for Region { Box::new(Ring { mapping: self.mapping.clone(), base: SEND_RING, len: SEND_BYTES as usize }) } - /// A doorbell that is full is rung already, and one whose reader is gone has a holder its - /// connection's hang-up reports. + /// A ring into a bell its holder no longer holds is dropped: that + /// holder's leaving is its rung bell's end. fn ring(&self, rings: Rings) { for holding in self.holders.by_number.borrow().values() { if rings.receive { - let _ = holding.receive.write_nonblock(&[1]); + holding.receive.wake(); } if rings.send { - let _ = holding.send.write_nonblock(&[1]); + holding.send.wake(); } } } diff --git a/system/netstack/src/main.rs b/system/netstack/src/main.rs index f334489fada..ea9b945cbc5 100644 --- a/system/netstack/src/main.rs +++ b/system/netstack/src/main.rs @@ -7,7 +7,7 @@ //! an address, a socket or a segment is the node's. What is here is what the //! node cannot do and stay pure: the link (`link`), the clock, the kernel's //! random source, the regions clients lend (`lent`), the bells a socket's -//! holders are rung on (`pipes`), the clients' connections (`client`) and +//! holders are rung on (`bells`), the clients' connections (`client`) and //! their requests (`serve`). //! //! **One link, until netstack runs several** @@ -51,10 +51,10 @@ use toyos_net_shard::{Config, Secrets}; use toyos_net_wire::ethernet::IndividualMac; use toyos_net_wire::Instant; +mod bells; mod client; mod lent; mod link; -mod pipes; mod serve; mod watches; @@ -98,12 +98,12 @@ const TCP_BUFFER: u32 = toyos::net::stream::RECEIVE_BYTES; const PIPE_BYTES: u64 = PIPE_CAPACITY as u64; /// What one holder of a stream can make this machine hold: the two pipes of -/// its connection and its two bells, each full. -const HOLDER_BYTES: u64 = 4 * PIPE_BYTES; +/// its connection, each full. Its bells are events, which hold no bytes. +const HOLDER_BYTES: u64 = 2 * PIPE_BYTES; /// What one holder of a listener or a datagram socket can make it hold: the -/// two pipes of its connection and its bell. -const SOCKET_HOLDER_BYTES: u64 = 3 * PIPE_BYTES; +/// two pipes of its connection, its bell being an event. +const SOCKET_HOLDER_BYTES: u64 = 2 * PIPE_BYTES; /// What one place can make this machine hold, at the largest of the things a /// place is (`toyos-net-node`'s `places`): a stream, its region and its first diff --git a/system/netstack/src/pipes.rs b/system/netstack/src/pipes.rs deleted file mode 100644 index 59ff6f9d212..00000000000 --- a/system/netstack/src/pipes.rs +++ /dev/null @@ -1,23 +0,0 @@ -//! A holder's bell of a listener or a datagram socket, as the node's [`Bell`]: netstack's write -//! end of a pipe it made, whose read end only that holder is handed. -//! -//! **A ring is one non-blocking write that is never retried.** A full pipe is rung already, and -//! one whose reader is gone has a holder whose connection's hang-up reports its leaving. - -use toyos::Pipe; -use toyos_abi::syscall::SyscallError; -use toyos_net_node::Bell; - -pub struct Rung(Pipe); - -impl Bell for Rung { - fn ring(&mut self) { - let _ = self.0.write_nonblock(&[1]); - } -} - -/// A new bell: the read end to hand its holder, and the end the node rings. -pub fn bell() -> Result<(Pipe, Rung), SyscallError> { - let (read, write) = toyos::pipe_pair()?; - Ok((read, Rung(write))) -} diff --git a/system/netstack/src/serve.rs b/system/netstack/src/serve.rs index 65912526107..fed88b0a697 100644 --- a/system/netstack/src/serve.rs +++ b/system/netstack/src/serve.rs @@ -20,9 +20,11 @@ //! **A stream is its region and its holders** (`toyos::net::stream`): the //! connection its connect or accept came on, and one more for each export. //! netstack judges the region before it maps it (`lent`), makes the first -//! holder's two bells and answers once with them, and from then watches each -//! holder's connection for its frames, rings and exports, and its hang-up, -//! which is that holder's leaving and, for the last, the stream's close. A +//! holder's bells and answers once with them, and from then watches each +//! holder's rung bell for its rings and exports, and its wakers' end, which is +//! that holder's leaving and, for the last, the stream's close. Until a +//! connect is answered netstack itself holds the rung's waker, so it watches +//! the first holder's connection instead, whose hang-up is that leaving. A //! holder netstack makes is a connection of its private port's, its `ADOPT` //! queued on it before the asker is answered. A client names no stream by a //! number, and a frame no holder of the protocol sends drops that holder. @@ -49,12 +51,13 @@ use std::rc::{Rc, Weak}; use toyos::net::*; use toyos::net::socket::{self, Kind, ACCEPT, CONNECT, OPTION, PEER, RECEIVE, SEND, SEND_PEER, SET_OPTION}; -use toyos::net::stream::{self, Adopt, ADOPT, BELLS, EXPORT, EXPORTED, REFUSED, RING}; +use toyos::net::stream::{Adopt, ADOPT, BELLS, EXPORT, EXPORTED, REFUSED}; use toyos::transport::session::Holder; use toyos::poller::READABLE; use toyos::port::Acceptor; use toyos::ipc::RxStep; -use toyos::{ipc, say, AsHandle, Connection, OwnedHandle, Pipe}; +use toyos::wake::{self, Bell, Hungup, Waker}; +use toyos::{ipc, say, AsHandle, Connection, OwnedHandle}; use toyos_abi::syscall::{SyscallError, MAX_BADGE}; use toyos_inspect::Snapshot; use toyos_net_node::{ @@ -67,7 +70,7 @@ use toyos_net_wire::{Instant, Port}; use crate::client::{Client, Request}; use crate::lent::{Holders, Holding, Region}; -use crate::pipes; +use crate::bells; use crate::watches::Watches; use crate::HOSTNAME; @@ -113,10 +116,9 @@ struct SocketHolder { struct Stream { id: StreamId, holders: Weak, - /// The connect not yet answered: the bells' read ends its answer hands - /// over, and the holders, kept for the answer past a session the node let - /// go of. - answer: Option<([Pipe; BELLS], Rc)>, + /// The connect not yet answered: the bells its answer hands over, and the + /// holders, kept for the answer past a session the node let go of. + answer: Option<([OwnedHandle; BELLS], Rc)>, /// The number the next holder made is given. next: u16, } @@ -180,20 +182,36 @@ pub struct Sockets { rung: Vec, } -/// Two bells: the read ends, which a holder is handed, and the write ends -/// netstack rings. -fn bells() -> Result<([Pipe; BELLS], [Pipe; BELLS]), SyscallError> { - let (receive_read, receive) = toyos::pipe_pair()?; - let (send_read, send) = toyos::pipe_pair()?; - Ok(([receive_read, send_read], [receive, send])) +/// What netstack keeps of a holder's three bells: the wakers of its receive +/// and send bells, and the bell it rings netstack on. +struct Kept { + receive: Waker, + send: Waker, + rung: Bell, } -/// What a lent region makes: the session, the bells' read ends and the holders. -type Lent = (Region, [Pipe; BELLS], Rc); +/// A holder's three bells: what it is handed, in [`BELLS`]'s order, and what +/// netstack keeps. +fn stream_bells() -> Result<([OwnedHandle; BELLS], Kept), SyscallError> { + let (receive, receive_bell) = wake::pair()?; + let (send, send_bell) = wake::pair()?; + let (rung_waker, rung) = wake::pair()?; + Ok(([receive_bell.into(), send_bell.into(), rung_waker.into()], Kept { receive, send, rung })) +} + +/// A holder on `connection` with the bells netstack keeps of it. +fn holding(connection: Connection, kept: Kept) -> Holding { + let Kept { receive, send, rung } = kept; + Holding { connection, frames: ipc::FrameRx::new(), receive, send, rung } +} + +/// What a lent region makes: the session, the bells its holder is handed and +/// the holders. +type Lent = (Region, [OwnedHandle; BELLS], Rc); /// A stream's region, judged, and its first holder's bells made, the holder's -/// connection being `connection`: the session the node is lent, the bells' read -/// ends the answer hands over, and the holders. Refused by name, nothing is +/// connection being `connection`: the session the node is lent, the bells the +/// answer hands over, and the holders. Refused by name, nothing is /// mapped, and the connection is handed back with the refusal's word. fn lend(connection: Connection, region: toyos::OwnedHandle) -> Result { let mapping = match Region::judged(region) { @@ -203,8 +221,8 @@ fn lend(connection: Connection, region: toyos::OwnedHandle) -> Result Ok((session, theirs, holders)), Err(why) => panic!("netstack: the kernel would not duplicate a region it mapped: {why:?}"), @@ -213,10 +231,10 @@ fn lend(connection: Connection, region: toyos::OwnedHandle) -> Result(holders: &Holders, bells: [Pipe; BELLS], answer: &T) { +fn hand_over(holders: &Holders, bells: [OwnedHandle; BELLS], answer: &T) { let by_number = holders.by_number.borrow(); let Some(first) = by_number.get(&Holder::FIRST) else { return }; - if let Err(why) = first.connection.try_send_with(RespType::Result as u32, answer, bells.map(OwnedHandle::from)) { + if let Err(why) = first.connection.try_send_with(RespType::Result as u32, answer, bells) { say!("netstack: a stream's client left before its answer: {why:?}"); } } @@ -361,7 +379,7 @@ impl Sockets { } } - fn hold_stream(&mut self, id: StreamId, holders: &Rc, answer: Option<[Pipe; BELLS]>) { + fn hold_stream(&mut self, id: StreamId, holders: &Rc, answer: Option<[OwnedHandle; BELLS]>) { let key = self.alloc_id(); let answer = answer.map(|bells| (bells, holders.clone())); self.streams.insert(key, Stream { id, holders: Rc::downgrade(holders), answer, next: 2 }); @@ -429,7 +447,7 @@ impl Sockets { msg.client.error(ERR_INVALID_INPUT); return; }; - let Ok((theirs, bell)) = pipes::bell() else { return msg.client.error(ERR_RESOURCE_EXHAUSTED) }; + let Ok((theirs, bell)) = bells::bell() else { return msg.client.error(ERR_RESOURCE_EXHAUSTED) }; match node.listen(Ipv4Addr::from(req.addr), Port::new(req.port), req.options.nodelay(), Box::new(bell), draw) { Ok((id, port)) => self.bound(node, now, Held::Listener(id), req.addr, msg.client, port, theirs), Err(refusal) => msg.client.error(listen_refused(refusal)), @@ -443,7 +461,7 @@ impl Sockets { msg.client.error(ERR_INVALID_INPUT); return; }; - let Ok((theirs, bell)) = pipes::bell() else { return msg.client.error(ERR_RESOURCE_EXHAUSTED) }; + let Ok((theirs, bell)) = bells::bell() else { return msg.client.error(ERR_RESOURCE_EXHAUSTED) }; match node.udp_bind(Ipv4Addr::from(req.addr), Port::new(req.port), Box::new(bell), draw) { Ok((id, port)) => self.bound(node, now, Held::Datagram(id), req.addr, msg.client, port, theirs), Err(refusal) => msg.client.error(refused(refusal)), @@ -451,7 +469,7 @@ impl Sockets { } /// A socket the node made, answered to its first holder with its bell. - fn bound(&mut self, node: &mut Node, now: Instant, socket: Held, addr: [u8; 4], client: Client, port: Port, bell: Pipe) { + fn bound(&mut self, node: &mut Node, now: Instant, socket: Held, addr: [u8; 4], client: Client, port: Port, bell: Bell) { let answer = BoundResponse { port: port.get(), _pad: 0 }; if let Err(why) = client.conn.try_send_with(RespType::Result as u32, &answer, [OwnedHandle::from(bell)]) { say!("netstack: a socket's client left before its answer: {why:?}"); @@ -558,14 +576,20 @@ impl Sockets { } /// Asks the kernel about everything a pass could be owed for: each - /// holder's connection, a stream's for its client's rings and a socket's - /// for its requests, and either's hang-up, and the connection of each - /// client that waits for a lookup, which hangs up by closing it. + /// stream holder's bell, for its rings and its leaving, or for a connect + /// not yet answered its connection, for its hang-up; each socket + /// holder's connection, for its requests and its hang-up, and the + /// connection of each client that waits for a lookup, which hangs up by + /// closing it. pub fn watch(&self, poller: &mut Watches) { for (key, (stream, number)) in &self.holders { - let Some(holders) = self.streams.get(stream).and_then(|stream| stream.holders.upgrade()) else { continue }; - if let Some(holding) = holders.by_number.borrow().get(number) { - poller.watch(&holding.connection, stream::SERVER_WATCH, TOKEN_STREAM | u64::from(*key)); + let Some(stream) = self.streams.get(stream) else { continue }; + let Some(holders) = stream.holders.upgrade() else { continue }; + let token = TOKEN_STREAM | u64::from(*key); + match holders.by_number.borrow().get(number) { + Some(holding) if stream.answer.is_some() => poller.watch(&holding.connection, READABLE, token), + Some(holding) => poller.watch(&holding.rung, READABLE, token), + None => {} } } for (key, holder) in &self.sockets { @@ -601,9 +625,10 @@ impl Sockets { true } - /// What a holder's connection had, frame by frame: its rings for the pass, - /// its exports answered, and its hang-up, or a frame no holder sends, its - /// leaving. + /// What a holder's bell, or its connection while its connect is + /// unanswered, said: a ring for the pass and its exports answered, read + /// frame by frame off its connection, or its leaving, which a hang-up and + /// a frame no holder sends are too. fn holder_answered(&mut self, node: &mut Node, now: Instant, key: u32, answer: Result) { let Some(&(stream_key, number)) = self.holders.get(&key) else { return }; let Some(stream) = self.streams.get(&stream_key) else { return }; @@ -611,13 +636,19 @@ impl Sockets { let Some(holders) = stream.holders.upgrade() else { return }; let (mut rang, mut exports, mut left) = (false, 0usize, answer.is_err()); if let Err(why) = answer { - say!("netstack: dropping a stream's holder — the kernel refused the watch of its connection: {why:?}"); + say!("netstack: dropping a stream's holder — the kernel refused the watch of its bell or connection: {why:?}"); } if let Some(holding) = holders.by_number.borrow_mut().get_mut(&number).filter(|_| !left) { + match holding.rung.try_take() { + Ok(woken) => rang = woken, + Err(Hungup) => left = true, + } for _ in 0..FRAMES_A_WAKE { + if left { + break; + } match heard(holding.frames.pump(&holding.connection)) { Heard::Nothing => break, - Heard::Ring => rang = true, Heard::Export => exports += 1, Heard::Leaving(why) => { if let Some(step) = why { @@ -668,10 +699,10 @@ impl Sockets { }; let made = (|| { let (theirs, ours) = self.exports.connect()?; - let (handed, rung) = bells()?; - Ok::<_, SyscallError>((theirs, ours, handed, rung, holders.region()?)) + let (handed, kept) = stream_bells()?; + Ok::<_, SyscallError>((theirs, ours, handed, kept, holders.region()?)) })(); - let Ok((theirs, ours, [receive_read, send_read], [receive, send], region)) = made else { + let Ok((theirs, ours, [receive_bell, send_bell, rung_waker], kept, region)) = made else { say!("netstack: refusing an export — the kernel made no connection, bells or region for it: {made:?}", made = made.err()); node.departed(now, id, number); return refuse(ERR_RESOURCE_EXHAUSTED); @@ -683,13 +714,13 @@ impl Sockets { local_port: local.get(), _pad: 0, }; - let handles = [region, OwnedHandle::from(receive_read), OwnedHandle::from(send_read)]; + let handles = [region, receive_bell, send_bell, rung_waker]; if let Err(why) = ours.try_send_with(ADOPT, &adopt, handles) { say!("netstack: refusing an export — its ADOPT would not queue: {why:?}"); node.departed(now, id, number); return refuse(ERR_RESOURCE_EXHAUSTED); } - holders.by_number.borrow_mut().insert(number, Holding { connection: ours, frames: ipc::FrameRx::new(), receive, send }); + holders.by_number.borrow_mut().insert(number, holding(ours, kept)); if let Some(stream) = self.streams.get_mut(&stream_key) { stream.next = next.wrapping_add(1); } @@ -793,7 +824,7 @@ impl Sockets { }); drop(region); let (answer, handles) = match (accepted, made) { - (Ok(accepted), Some((theirs, [receive, send], holders))) => { + (Ok(accepted), Some((theirs, [receive, send, rung], holders))) => { let answer = TcpAcceptResponse { remote_addr: accepted.remote.addr.octets(), remote_port: accepted.remote.port.get(), @@ -801,7 +832,7 @@ impl Sockets { nodelay: u32::from(accepted.nodelay), }; self.hold_stream(accepted.id, &holders, None); - (Ok(answer), Some([theirs.into_owned(), OwnedHandle::from(receive), OwnedHandle::from(send)])) + (Ok(answer), Some([theirs.into_owned(), receive, send, rung])) } (Ok(_), None) => unreachable!("the node took a stream it was given no session for"), (Err(AcceptRefused::NoRegion), _) => (Err(code), None), @@ -827,7 +858,7 @@ impl Sockets { let Some(&SocketHolder { socket, addr, .. }) = self.sockets.get(&key) else { return true }; let made = (|| { let (theirs, ours) = self.exports.connect()?; - let (handed, bell) = pipes::bell()?; + let (handed, bell) = bells::bell()?; Ok::<_, SyscallError>((theirs, ours, handed, bell)) })(); let reply = match made { @@ -904,18 +935,16 @@ fn tcp_counters(counters: &toyos_net_tcp::Counters, snap: &mut Snapshot) { #[derive(Debug)] enum Heard { Nothing, - Ring, Export, /// It hung up, or, with what it sent, wrote what no holder writes. Leaving(Option), } -/// A holder's frame as netstack takes it: [`RING`] and [`EXPORT`], bare, -/// and anything else its leaving. +/// A holder's frame as netstack takes it: [`EXPORT`], bare, and anything else +/// its leaving. fn heard(step: RxStep) -> Heard { match step { RxStep::Idle => Heard::Nothing, - RxStep::Frame { msg_type: RING, payload_len: 0, handles: 0 } => Heard::Ring, RxStep::Frame { msg_type: EXPORT, payload_len: 0, handles: 0 } => Heard::Export, RxStep::Eof => Heard::Leaving(None), step @ (RxStep::Malformed | RxStep::Frame { .. }) => Heard::Leaving(Some(step)), @@ -1071,7 +1100,7 @@ mod tests { /// there, so a connection that holds nothing reaches nothing. #[test] fn a_new_connection_names_no_socket() { - for holders in [ACCEPT, OPTION, SET_OPTION, SEND, RECEIVE, CONNECT, PEER, SEND_PEER, EXPORT, EXPORTED, REFUSED, RING, ADOPT, socket::ADOPT] { + for holders in [ACCEPT, OPTION, SET_OPTION, SEND, RECEIVE, CONNECT, PEER, SEND_PEER, EXPORT, EXPORTED, REFUSED, ADOPT, socket::ADOPT] { assert_eq!(MsgType::from_u32(holders), None, "{holders}"); } } @@ -1110,7 +1139,6 @@ mod tests { (Kind::Listener, EXPORT, &[], 1), (Kind::Datagram, SET_OPTION, &option[..7], 0), (Kind::Datagram, OPTION, &option[..], 0), - (Kind::Listener, RING, &[], 0), (Kind::Listener, CONNECT, &datagram[..8], 0), (Kind::Listener, PEER, &[], 0), (Kind::Listener, SEND_PEER, b"hi", 0), @@ -1125,20 +1153,17 @@ mod tests { } } - /// Only a bare ring and a bare export are a holder's: any payload, any - /// handle, any other type and broken framing are its leaving, and none - /// panics. + /// Only a bare export is a holder's frame: any payload, any handle, any + /// other type and broken framing are its leaving, and none panics. #[test] fn a_holder_frame_no_holder_sends_is_its_leaving() { let frame = |msg_type, payload_len, handles| RxStep::Frame { msg_type, payload_len, handles }; assert!(matches!(heard(RxStep::Idle), Heard::Nothing)); - assert!(matches!(heard(frame(RING, 0, 0)), Heard::Ring)); assert!(matches!(heard(frame(EXPORT, 0, 0)), Heard::Export)); assert!(matches!(heard(RxStep::Eof), Heard::Leaving(None))); for step in [ RxStep::Malformed, - frame(RING, 1, 0), - frame(RING, 0, 1), + frame(1, 0, 0), frame(EXPORT, 4, 0), frame(EXPORT, 0, 1), frame(EXPORTED, 0, 1), diff --git a/system/nvme/src/main.rs b/system/nvme/src/main.rs index cc692f1b068..2883d508a7d 100644 --- a/system/nvme/src/main.rs +++ b/system/nvme/src/main.rs @@ -114,7 +114,7 @@ fn main() { let dev = claim(); let info = dev.describe().unwrap_or_else(|why| panic!("nvme: the claim would not describe itself: {why:?}")); let name = format!("nvme {:02x}:{:02x}.{}", info.bus, info.dev, info.func); - let stop = Endowments::get().take::(toyos_quiesce::DRIVER_STOP); + let stop = Endowments::get().take::(toyos_quiesce::DRIVER_STOP); let mut link = Link::new(name.clone()) .unwrap_or_else(|| panic!("{name}: started with no `{}` port: a disk driver's row receives it", toyos_blockring::disk::PORT)); let mut ctrl = match Controller::open(dev, &info) { @@ -142,7 +142,7 @@ fn main() { /// A driver whose controller cannot be used: it keeps its link, so the /// service goes on saying so, and opens nothing. -fn idle(link: &mut Link, stop: Option<&toyos::Pipe>) -> ! { +fn idle(link: &mut Link, stop: Option<&toyos::wake::Bell>) -> ! { let poller = Poller::new(2); loop { link.watch(&poller, TOKEN_LINK); @@ -158,7 +158,7 @@ fn idle(link: &mut Link, stop: Option<&toyos::Pipe>) -> ! { } } -fn serve(name: &str, ctrl: &mut Controller, disk: &mut Sessions, link: &mut Link, stop: Option<&toyos::Pipe>) -> ! { +fn serve(name: &str, ctrl: &mut Controller, disk: &mut Sessions, link: &mut Link, stop: Option<&toyos::wake::Bell>) -> ! { let poller = Poller::new(3 + MAX_SESSIONS as u32); let mut ready: Vec = Vec::new(); let mut done: Vec = Vec::new(); diff --git a/system/soundserver/src/client.rs b/system/soundserver/src/client.rs index a7fb8e18579..e2943aa351e 100644 --- a/system/soundserver/src/client.rs +++ b/system/soundserver/src/client.rs @@ -1,7 +1,7 @@ //! One stream: its ring, its ramp, and how it ended. //! //! A client is a control connection, a shared-memory ring soundserver made for it, a -//! signal pipe soundserver kept the write end of, and a gain. This module is where +//! signal soundserver kept the signal side of, and a gain. This module is where //! all four are assembled ([`open_stream`]) and where one period of the result //! reaches the bus ([`mix_client`]). //! @@ -38,7 +38,7 @@ pub(crate) struct ClientResampler { /// How a stream ended, as far as soundserver can honestly tell. /// /// Two things witness a client leaving and they race: the control thread reads -/// the peer, and the mix loop finds the signal pipe gone on its next write. +/// the peer, and the mix loop finds the client's signal gone on its next ring. /// Both start the same ramp, so no audio differs — but only the first of them /// *knows* anything. A clean exit and a crash tear down the same descriptors the /// same way; the kernel's `exit:` line carries the code and nothing on this side @@ -55,10 +55,10 @@ pub(crate) enum Departure { /// The control connection ended without a close. The client's process is /// gone; whether it exited or crashed is not knowable here. Disconnected, - /// The signal pipe broke, which says the client's descriptor table is gone + /// The client's signal has no wait side left, which says its handle table is gone /// and nothing about why. The weakest of the four, and the only one the /// others replace. - SignalPipeGone, + SignalGone, } impl Departure { @@ -79,7 +79,7 @@ impl Departure { Departure::Closed => 0, Departure::Refused => 1, Departure::Disconnected => 2, - Departure::SignalPipeGone => 3, + Departure::SignalGone => 3, } } } @@ -90,7 +90,7 @@ impl core::fmt::Display for Departure { Departure::Closed => "closed", Departure::Refused => "refused", Departure::Disconnected => "disconnected", - Departure::SignalPipeGone => "signal pipe gone", + Departure::SignalGone => "its signal is gone", }) } } @@ -100,11 +100,11 @@ pub(crate) struct ClientStream { pub(crate) slot_reader: AudioSlotReader, /// One period of the ring, copied out of the slot it is decoded from. period: Vec, - /// The write end of the signal pipe. soundserver makes both ends and sends the - /// read end to the client, so crash detection is by construction: - /// the moment the client's table goes, the read end goes with it and the - /// next signal breaks. - pub(crate) signal_write: toyos::Pipe, + /// The signal side of the client's event. soundserver makes both sides and + /// sends the wait side to the client, so crash detection is by + /// construction: the moment the client's table goes, the wait side goes + /// with it and the next ring answers `Hungup`. + pub(crate) signal: toyos::wake::Waker, pub(crate) gain: GainRamp, pub(crate) client_channels: u16, pub(crate) client_period_frames: u32, @@ -191,15 +191,13 @@ pub(crate) fn open_stream( hdr.read_idx.store(0, core::sync::atomic::Ordering::Relaxed); } - // **soundserver makes the pipe and keeps the write end.** That is what makes a - // dead client detectable without bookkeeping: the read end it is sent goes - // when its table does, and the next signal answers `Gone`. The client used - // to make the pipe and name it by an id, because an id was only openable - // by a peer of its creator and the peer relation ran one way. - let (signal_read, signal_write) = match toyos::pipe_pair() { + // **soundserver makes the event and keeps the signal side.** That is what + // makes a dead client detectable without bookkeeping: the wait side it is + // sent goes when its table does, and the next ring answers `Hungup`. + let (signal, signal_bell) = match toyos::wake::pair() { Ok(ends) => ends, Err(e) => { - say!("soundserver: no signal pipe for client {client_id} ({e:?})"); + say!("soundserver: no signal event for client {client_id} ({e:?})"); return None; } }; @@ -215,7 +213,7 @@ pub(crate) fn open_stream( slot_count: slot_count as u16, fade_frames: ramp_frames, }; - if control.send_with(MSG_STREAM_OPENED, &opened, [client_shm, signal_read.into()]).is_err() { + if control.send_with(MSG_STREAM_OPENED, &opened, [client_shm, signal_bell.into()]).is_err() { // Client died mid-open; the dropped control connection removes it. say!("soundserver: client {client_id} vanished during stream open"); } @@ -246,7 +244,7 @@ pub(crate) fn open_stream( client_id, slot_reader, period: vec![0; client_period_bytes as usize], - signal_write, + signal, gain, client_channels: req.channels, client_period_frames, @@ -338,11 +336,11 @@ mod tests { /// The control thread read the peer; the mix loop only found a descriptor /// missing. Whichever arrives first, the removal must be reported with what /// was actually established — and a clean exit must never be reported as a - /// death, which is what `SignalPipeGone` refuses to claim. + /// death, which is what `SignalGone` refuses to claim. #[test] fn the_stronger_witness_wins_in_either_order() { use Departure::*; - for (a, b) in [(Closed, SignalPipeGone), (Refused, SignalPipeGone), (Disconnected, SignalPipeGone)] { + for (a, b) in [(Closed, SignalGone), (Refused, SignalGone), (Disconnected, SignalGone)] { assert_eq!(a.refine(b), a, "{b} must not replace {a}"); assert_eq!(b.refine(a), a, "{a} must replace {b}"); } @@ -350,9 +348,9 @@ mod tests { // then dropped. assert_eq!(Closed.refine(Disconnected), Closed); assert_eq!(Disconnected.refine(Closed), Closed); - // Idempotent, so a repeated witness — the mix loop writes a broken pipe + // Idempotent, so a repeated witness — the mix loop rings a gone signal // every period until the ramp finishes — changes nothing. - for how in [Closed, Refused, Disconnected, SignalPipeGone] { + for how in [Closed, Refused, Disconnected, SignalGone] { assert_eq!(how.refine(how), how); } } diff --git a/system/soundserver/src/command.rs b/system/soundserver/src/command.rs index cf321af3ada..c47365dc06b 100644 --- a/system/soundserver/src/command.rs +++ b/system/soundserver/src/command.rs @@ -12,6 +12,7 @@ //! is the one that can afford it. use toyos_abi::syscall; +use toyos::wake::Waker; use toyos_mixer::Gain; use crate::client::{ClientStream, Departure}; @@ -75,11 +76,11 @@ impl CommandRing { /// forever) or asserting (soundserver dead at a client's choosing). The retry is /// unbounded because the mix thread is the process's main thread — if it has /// stopped, soundserver is already gone. -pub(crate) fn submit(cmd_ring: &CommandRing, cmd_pipe_write: toyos::BorrowedHandle<'_>, cmd: MixCommand, period_nanos: u64) { +pub(crate) fn submit(cmd_ring: &CommandRing, cmd_wake: &Waker, cmd: MixCommand, period_nanos: u64) { let mut cmd = cmd; loop { let full = cmd_ring.try_push(cmd); - let _ = syscall::write_nonblock(cmd_pipe_write.raw(), &[1]); + cmd_wake.wake(); match full { Ok(()) => return, Err(returned) => { @@ -97,14 +98,14 @@ pub(crate) fn submit(cmd_ring: &CommandRing, cmd_pipe_write: toyos::BorrowedHand /// instead of being reconstructed from a flag on the other side. pub(crate) fn remove( cmd_ring: &CommandRing, - cmd_pipe_write: toyos::BorrowedHandle<'_>, + cmd_wake: &Waker, client_id: usize, departure: Departure, period_nanos: u64, ) { submit( cmd_ring, - cmd_pipe_write, + cmd_wake, MixCommand::RemoveClient { client_id, departure }, period_nanos, ); diff --git a/system/soundserver/src/control.rs b/system/soundserver/src/control.rs index d225b064967..75eac14885c 100644 --- a/system/soundserver/src/control.rs +++ b/system/soundserver/src/control.rs @@ -18,8 +18,8 @@ use toyos::audio::{ use toyos::ipc::{self, RxStep}; use toyos::poller::{Poller, READABLE}; use toyos::port::Acceptor; +use toyos::wake::{Bell, Waker}; use toyos::Connection; -use toyos_abi::syscall; use toyos_mixer::{period_nanos, Gain, MAX_CLIENT_RATE, MIN_CLIENT_RATE}; use crate::client::{open_stream, Departure}; @@ -35,7 +35,7 @@ use crate::volume::{self, VolumeClient}; /// size minus one. 64 costs the same 2 MiB page as 32; 63 simultaneous streams /// is already past what the mixer renders inside one 2.9 ms period, and costs /// 189 of the kernel's 4096 handle slots (a control connection plus both signal -/// pipe ends per client). +/// event sides per client). pub(crate) const MAX_CONTROL_CLIENTS: usize = 63; /// The widest control payload soundserver decodes: `StreamOpenRequest` is @@ -132,8 +132,8 @@ pub(crate) fn control_thread( volume_acceptor: Acceptor, mut outputs: Outputs, cmd_ring: &CommandRing, - cmd_pipe_write: toyos::BorrowedHandle<'_>, - ended_read: toyos::BorrowedHandle<'_>, + cmd_wake: &Waker, + ended_bell: &Bell, mut device: Device, published: &Published, ) { @@ -183,11 +183,11 @@ pub(crate) fn control_thread( device = crate::output_device(&attached.offer, attached.period_frames); period_nanos = toyos_mixer::period_nanos(device.period_frames as u64, device.rate as u64); ramp_frames = toyos_mixer::ramp_frames(device.rate); - submit(cmd_ring, cmd_pipe_write, MixCommand::Attach(attached), period_nanos); + submit(cmd_ring, cmd_wake, MixCommand::Attach(attached), period_nanos); } poller.watch(&acceptor, READABLE, TOKEN_ACCEPT); - poller.watch(&ended_read, READABLE, TOKEN_ENDED); + poller.watch(&ended_bell, READABLE, TOKEN_ENDED); outputs.watch(&poller, TOKEN_OUTPUTS); poller.watch(&volume_acceptor, READABLE, TOKEN_VOLUME_ACCEPT); for (i, client) in clients.iter().enumerate() { @@ -233,8 +233,7 @@ pub(crate) fn control_thread( } } if ready.contains(&TOKEN_ENDED) { - let mut drain = [0u8; 8]; - while matches!(syscall::read_nonblock(ended_read.raw(), &mut drain), Ok(n) if n == drain.len()) {} + ended_bell.take(); outputs.ended(&mut events); } outputs.serve(&ready, TOKEN_OUTPUTS, &mut events); @@ -247,7 +246,7 @@ pub(crate) fn control_thread( device = crate::NULL_SINK; period_nanos = toyos_mixer::period_nanos(device.period_frames as u64, device.rate as u64); ramp_frames = toyos_mixer::ramp_frames(device.rate); - submit(cmd_ring, cmd_pipe_write, MixCommand::Detach, period_nanos); + submit(cmd_ring, cmd_wake, MixCommand::Detach, period_nanos); if !relay.idle() { volume::answer(&mut volume_clients, &mut relay, None); } @@ -278,7 +277,7 @@ pub(crate) fn control_thread( // crashed is not knowable here. RxStep::Eof => { if let Some(idx) = clients[i].stream_idx { - remove(cmd_ring, cmd_pipe_write, idx, Departure::Disconnected, period_nanos); + remove(cmd_ring, cmd_wake, idx, Departure::Disconnected, period_nanos); } dead.push(i); disconnected = true; @@ -290,7 +289,7 @@ pub(crate) fn control_thread( RxStep::Malformed => { say!("soundserver: frame this protocol cannot describe, disconnecting client"); if let Some(idx) = clients[i].stream_idx { - remove(cmd_ring, cmd_pipe_write, idx, Departure::Refused, period_nanos); + remove(cmd_ring, cmd_wake, idx, Departure::Refused, period_nanos); } dead.push(i); disconnected = true; @@ -334,14 +333,14 @@ pub(crate) fn control_thread( disconnected = true; break 'msgs; }; - submit(cmd_ring, cmd_pipe_write, MixCommand::AddClient(Box::new(client), generation), period_nanos); + submit(cmd_ring, cmd_wake, MixCommand::AddClient(Box::new(client), generation), period_nanos); clients[i].stream_idx = Some(idx); } Control::SetVolume { idx } => { let raw = f32::from_le_bytes(payload[0..4].try_into().unwrap()); let Some(gain) = Gain::from_wire(raw) else { say!("soundserver: volume is not a number, disconnecting client"); - remove(cmd_ring, cmd_pipe_write, idx, Departure::Refused, period_nanos); + remove(cmd_ring, cmd_wake, idx, Departure::Refused, period_nanos); dead.push(i); disconnected = true; break 'msgs; @@ -349,7 +348,7 @@ pub(crate) fn control_thread( clients[i].pending_volume = Some(gain); } Control::Close { idx } => { - remove(cmd_ring, cmd_pipe_write, idx, Departure::Closed, period_nanos); + remove(cmd_ring, cmd_wake, idx, Departure::Closed, period_nanos); dead.push(i); disconnected = true; break 'msgs; @@ -381,7 +380,7 @@ pub(crate) fn control_thread( Control::Violation => { say!("soundserver: protocol violation (msg {msg_type}), disconnecting client"); if let Some(idx) = clients[i].stream_idx { - remove(cmd_ring, cmd_pipe_write, idx, Departure::Refused, period_nanos); + remove(cmd_ring, cmd_wake, idx, Departure::Refused, period_nanos); } dead.push(i); disconnected = true; @@ -395,7 +394,7 @@ pub(crate) fn control_thread( // volume instead of at silence and strand the stream unremoved. if !disconnected { if let (Some(client_id), Some(target)) = (clients[i].stream_idx, clients[i].pending_volume.take()) { - submit(cmd_ring, cmd_pipe_write, MixCommand::SetVolume { client_id, target }, period_nanos); + submit(cmd_ring, cmd_wake, MixCommand::SetVolume { client_id, target }, period_nanos); } } } diff --git a/system/soundserver/src/main.rs b/system/soundserver/src/main.rs index 203127b8dc6..7588094b5ad 100644 --- a/system/soundserver/src/main.rs +++ b/system/soundserver/src/main.rs @@ -23,7 +23,8 @@ use toyos::endow; use toyos::port::Acceptor; -use toyos::{AsHandle, PciDev}; +use toyos::wake::{self, Bell, Waker}; +use toyos::PciDev; use toyos_mixer::{period_frames, ramp_frames, DEVICE_RATES}; use toyos_sound_output as sound_output; @@ -121,9 +122,9 @@ fn run(ports: Ports, own: Option<(VirtioBackend, Device)>) { let device = own.as_ref().map_or(NULL_SINK, |(_, device)| *device); let outputs = Outputs::new(ports.output, own.is_some()); let cmd_ring = Arc::new(CommandRing::new()); - let (cmd_read, cmd_write) = toyos::pipe_pair().expect("soundserver: failed to create the command pipe"); + let (cmd_wake, cmd_bell) = wake::pair().expect("soundserver: no event for its commands"); // The mix thread's word that it ended a device loop on its own finding. - let (ended_read, ended_write) = toyos::pipe_pair().expect("soundserver: failed to create the ended pipe"); + let (ended_wake, ended_bell) = wake::pair().expect("soundserver: no event for a device loop's end"); let initial = if own.is_some() { State::Suspended } else { State::Idle }; let published = Arc::new(Published::new(initial)); @@ -137,8 +138,8 @@ fn run(ports: Ports, own: Option<(VirtioBackend, Device)>) { ports.volume, outputs, &cmd_ring2, - cmd_write.as_handle(), - ended_read.as_handle(), + &cmd_wake, + &ended_bell, device, &published2, ); @@ -147,7 +148,7 @@ fn run(ports: Ports, own: Option<(VirtioBackend, Device)>) { if let Some((mut backend, device)) = own { let period_bytes = toyos_virtio_sound::PERIOD_BYTES; - let why = serve_device(&mut backend, &cmd_ring, &published, cmd_read.as_handle(), &device, period_bytes, 0, ended_write.as_handle()); + let why = serve_device(&mut backend, &cmd_ring, &published, &cmd_bell, &device, period_bytes, 0, &ended_wake); panic!("soundserver: its own virtio-sound device loop ended ({why})"); } let mut generation = 0; @@ -156,7 +157,7 @@ fn run(ports: Ports, own: Option<(VirtioBackend, Device)>) { "soundserver: no output yet, presenting a null sink ({}Hz {}ch, {} frames/period, streams discarded)", device.rate, device.channels, device.period_frames ); - let attached = null_sink_thread(&cmd_ring, &published, cmd_read.as_handle(), &device, generation); + let attached = null_sink_thread(&cmd_ring, &published, &cmd_bell, &device, generation); let Attached { mut backend, offer, period_frames, generation: playing } = *attached; let output = output_device(&offer, period_frames); say!( @@ -164,7 +165,7 @@ fn run(ports: Ports, own: Option<(VirtioBackend, Device)>) { output.rate, output.channels, output.period_frames ); let period_bytes = offer.period_bytes as usize; - let why = serve_device(&mut backend, &cmd_ring, &published, cmd_read.as_handle(), &output, period_bytes, playing, ended_write.as_handle()); + let why = serve_device(&mut backend, &cmd_ring, &published, &cmd_bell, &output, period_bytes, playing, &ended_wake); say!("soundserver: the output is gone ({why})"); generation = playing + 1; } @@ -185,11 +186,11 @@ fn serve_device( backend: &mut dyn Backend, cmd_ring: &CommandRing, published: &Published, - cmd_pipe_read: toyos::BorrowedHandle<'_>, + cmd_bell: &Bell, device: &Device, period_bytes: usize, generation: u32, - ended_write: toyos::BorrowedHandle<'_>, + ended_wake: &Waker, ) -> String { say!( "soundserver: ready, {} buffers, {}Hz {}ch, {} bytes/period, {} frames/period", @@ -199,7 +200,7 @@ fn serve_device( backend, cmd_ring, published, - cmd_pipe_read, + cmd_bell, device.buffers as usize, device.rate, device.channels, @@ -207,6 +208,6 @@ fn serve_device( device.period_frames as usize, ramp_frames(device.rate), generation, - ended_write, + ended_wake, ) } diff --git a/system/soundserver/src/mix.rs b/system/soundserver/src/mix.rs index 7fd5dedceb7..bd18f40f70f 100644 --- a/system/soundserver/src/mix.rs +++ b/system/soundserver/src/mix.rs @@ -16,7 +16,7 @@ use std::time::Duration; use toyos::endow::Endowments; use toyos::poller::{Poller, READABLE}; use toyos::syscap::{SysCap, WakeLatencyHold}; -use toyos_abi::syscall; +use toyos::wake::{Bell, Waker}; use toyos_sound_output::{run, Run}; use toyos_mixer::{ deferral_floor_nanos, period_nanos, quantize_period, scratch_frames, wake_left_idle, Dll, @@ -74,25 +74,19 @@ fn flush(stats: &mut MixStats, totals: &mut Totals, clients: usize) -> Window { /// Signal every client before the wait so priority inheritance can fill their /// rings while soundserver blocks, and reap the ones that died doing it. /// -/// A broken pipe here is the client's departure, caught here rather +/// A ring that answers `Hungup` is the client's departure, caught here rather /// than left to the control connection — a client that goes mid-stream would /// otherwise stay `is_streaming()` and keep the loop deferring buffers for a -/// producer that no longer exists. Departure is exactly `Err(Gone)`, the -/// kernel's broken-pipe error; a full pipe is `Err(WouldBlock)` and means the -/// client is merely behind on consuming signals, which must leave it untouched -/// — a paused client stops reading its pipe indefinitely and is alive. +/// producer that no longer exists. A signal still pending only coalesces: a +/// paused client stops taking its signals indefinitely and is alive. /// -/// It says nothing on its own: [`Departure::SignalPipeGone`] is the weakest of +/// It says nothing on its own: [`Departure::SignalGone`] is the weakest of /// the four witnesses and the control thread's is on the way, so the word waits /// for `retain_active` and the strongest witness by then wins. fn signal_clients(streams: &mut [ClientStream], ramp_frames: u32) { for stream in streams.iter_mut() { - let gone = matches!( - stream.signal_write.write_nonblock(&[1]), - Err(syscall::SyscallError::Gone) - ); - if gone { - stream.depart(Departure::SignalPipeGone, ramp_frames); + if stream.signal.ring().is_err() { + stream.depart(Departure::SignalGone, ramp_frames); } } } @@ -118,7 +112,7 @@ fn apply_commands( } MixCommand::AddClient(client, _) => { say!("soundserver: client {} connected (id={})", streams.len(), client.client_id); - let _ = client.signal_write.write_nonblock(&[1]); + client.signal.wake(); streams.push(*client); } MixCommand::RemoveClient { client_id, departure } => { @@ -159,7 +153,7 @@ enum Change { Detach, } -/// Every stream the device loop held ends with it: its signal pipe closes, so +/// Every stream the device loop held ends with it: its signal side closes, so /// its client sees the stream end, and nothing plays it on. fn end_streams(streams: &mut Vec) { for stream in streams.drain(..) { @@ -220,7 +214,7 @@ pub(crate) fn mix_thread( backend: &mut dyn Backend, cmd_ring: &CommandRing, published: &Published, - cmd_pipe_read: toyos::BorrowedHandle<'_>, + cmd_bell: &Bell, num_buffers: usize, device_sample_rate: u32, device_channels: u16, @@ -228,7 +222,7 @@ pub(crate) fn mix_thread( device_period_frames: usize, ramp_frames: u32, generation: u32, - ended_write: toyos::BorrowedHandle<'_>, + ended_wake: &Waker, ) -> String { let device_period_samples = device_period_frames * device_channels as usize; let period_nanos = period_nanos(device_period_frames as u64, device_sample_rate as u64); @@ -364,7 +358,7 @@ pub(crate) fn mix_thread( armed(!streams.is_empty(), &dll, period_nanos, toyos_abi::clock::nanos_since_boot() as f64); poller.watch(&backend.handle(), READABLE, TOKEN_AUDIO); - poller.watch(&cmd_pipe_read, READABLE, TOKEN_CMD); + poller.watch(&cmd_bell, READABLE, TOKEN_CMD); let mut cmd_ready = false; poller.wait(1, timeout, |token| match token { @@ -379,8 +373,7 @@ pub(crate) fn mix_thread( let started_at_wake = started; if cmd_ready { - let mut drain = [0u8; 64]; - while matches!(syscall::read_nonblock(cmd_pipe_read.raw(), &mut drain), Ok(n) if n == drain.len()) {} + cmd_bell.take(); } match apply_commands(cmd_ring, &mut streams, ramp_frames, generation) { None => {} @@ -400,7 +393,7 @@ pub(crate) fn mix_thread( let completed = backend.completion(); if let Some(why) = backend.ended() { end_streams(&mut streams); - let _ = syscall::write_nonblock(ended_write.raw(), &[1]); + ended_wake.wake(); return why.to_string(); } if let Some(rec) = completed { @@ -719,13 +712,13 @@ pub(crate) fn mix_thread( /// After mixing one period it throws the samples away. /// /// Idle discipline: with no streams it holds no timer and takes no -/// wakes, blocking on the command pipe alone, so an audience of zero costs +/// wakes, blocking on the command bell alone, so an audience of zero costs /// exactly zero CPU. It does not request the RT band — it protects no audible /// output, so there is nothing for the band to protect. pub(crate) fn null_sink_thread( cmd_ring: &CommandRing, published: &Published, - cmd_pipe_read: toyos::BorrowedHandle<'_>, + cmd_bell: &Bell, device: &Device, generation: u32, ) -> Box { @@ -771,7 +764,7 @@ pub(crate) fn null_sink_thread( signal_clients(&mut streams, ramp_frames); // Idle discipline: no streams → no timer, no wakes. A connect - // arrives as a command-pipe byte, the only wake source the null sink + // arrives as a ring of the command bell, the only wake source the null sink // has. While streaming, wake at the next grid point. let timeout = if streams.is_empty() { u64::MAX @@ -779,7 +772,7 @@ pub(crate) fn null_sink_thread( next_period_ns.saturating_sub(toyos_abi::clock::nanos_since_boot()).max(1) }; - poller.watch(&cmd_pipe_read, READABLE, TOKEN_CMD); + poller.watch(&cmd_bell, READABLE, TOKEN_CMD); let mut cmd_ready = false; poller.wait(1, timeout, |token| match token { TOKEN_CMD => cmd_ready = true, @@ -791,8 +784,7 @@ pub(crate) fn null_sink_thread( } if cmd_ready { - let mut drain = [0u8; 64]; - while matches!(syscall::read_nonblock(cmd_pipe_read.raw(), &mut drain), Ok(n) if n == drain.len()) {} + cmd_bell.take(); } // Nothing is drained past an output to play to: what is behind it is // the output's. diff --git a/system/supervisor/src/main.rs b/system/supervisor/src/main.rs index 8e2057269f4..e3a2246b37a 100644 --- a/system/supervisor/src/main.rs +++ b/system/supervisor/src/main.rs @@ -104,6 +104,8 @@ use std::os::toyos::process::CommandExt; use std::path::Path; use std::process::{Child, Command, Stdio}; use std::rc::Rc; +use std::sync::atomic::{AtomicU32, Ordering}; +use std::sync::Arc; use std::time::{Duration, Instant}; use toyos_swap::{Refusal, Request as SwapRequest, Word}; @@ -127,9 +129,10 @@ use toyos::port::{self, Acceptor, Connector}; use toyos::log::region::{Ring, RING_BYTES}; use toyos::power; use toyos::say; +use toyos::wake::{self, Bell, Waker}; use toyos::shm::SharedMemory; use toyos::syscap::SysCap; -use toyos::{AsHandle, Pipe}; +use toyos::AsHandle; use toyos_abi::Rights; use toyos_blockring::wire::{Grant as BlockGrant, Scope}; use toyos_logstream::{ @@ -378,7 +381,7 @@ impl Log { } // SAFETY: the descriptor installed above, named nowhere else. unsafe { toyos::fd::close(held) }.expect("supervisor: the ring's descriptor it just made"); - let (alive, keep) = toyos::pipe_pair().expect("supervisor: no pipe to say it lives"); + let (keep, alive) = wake::pair().expect("supervisor: no event to say it lives"); // Held for the machine's life: the supervisor's end is the machine's. core::mem::forget(keep); log.register("supervisor", toyos_abi::syscall::getpid().0, ring, alive); @@ -386,9 +389,9 @@ impl Log { } /// Name program `pid` the owner of `ring`, its log, and move the ring to - /// `logkeeper` under `name`, with the read end of the pipe whose only writer is - /// that program. - fn register(&self, name: &str, pid: u32, ring: SharedMemory, alive: Pipe) { + /// `logkeeper` under `name`, with the bell whose only waker is that + /// program's. + fn register(&self, name: &str, pid: u32, ring: SharedMemory, alive: Bell) { let Some(tag) = Tag::new(name) else { panic!("supervisor: `{name}` is not a name a line of the log can carry"); }; @@ -415,7 +418,6 @@ impl Log { return; } self.unanswered.set(self.unanswered.get() + 1); - let poller = Poller::new(1); let began = Instant::now(); loop { let step = self.rx.borrow_mut().pump(&self.conn); @@ -446,8 +448,11 @@ impl Log { ); return; } - poller.watch(&self.conn, READABLE, 0); - poller.wait(1, left.as_nanos() as u64, |_| {}); + // Ready or the bound: the next pump and the check above say which. + if let Err(e) = toyos::poller::wait_one(&self.conn, READABLE, left.as_nanos() as u64) { + toyos::warn!("supervisor: the kernel refused a wait on logkeeper's answer ({e:?}); stopping without it"); + return; + } } } } @@ -684,8 +689,10 @@ type Job = Box; /// and the loop waits on it and on the end together ([`Supervisor::files`]). struct Worker { jobs: std::sync::mpsc::Sender, - /// Readable once per job finished. - done: Pipe, + /// Rung as a job finishes. + done: Bell, + /// Jobs finished since last counted. + finished: Arc, /// Jobs handed over and not seen finished: one that outlived its bound is /// still here, and the worker takes no other until it finishes. owed: u32, @@ -698,28 +705,28 @@ struct Worker { impl Worker { fn start() -> Self { let (jobs, queue) = std::sync::mpsc::channel::(); - let (done, finished) = toyos::pipe_pair().expect("supervisor: no pipe for its file worker"); + let (ring, done) = wake::pair().expect("supervisor: no event for its file worker"); + let finished = Arc::new(AtomicU32::new(0)); + let count = Arc::clone(&finished); std::thread::Builder::new() .name("files".into()) .spawn(move || { for job in queue { job(); - finished.write(&[1]).expect("supervisor: its file worker could not say a job finished"); + count.fetch_add(1, Ordering::Release); + ring.wake(); } }) .expect("supervisor: its file worker could not be started"); - Self { jobs, done, owed: 0, owed_what: String::new(), poller: Poller::new((1 + MAX_CHILDREN + sessions::MAX_SESSIONS) as u32) } + let poller = Poller::new((1 + MAX_CHILDREN + sessions::MAX_SESSIONS) as u32); + Self { jobs, done, finished, owed: 0, owed_what: String::new(), poller } } - /// Count the jobs that have finished since last asked. + /// Count the jobs that have finished since last asked: the bell is taken + /// first, so a job counted after it rings it again. fn settle(&mut self) { - let mut seen = [0u8; 8]; - while self.owed > 0 { - match self.done.read_nonblock(&mut seen) { - Ok(n) if n > 0 => self.owed -= n as u32, - _ => return, - } - } + self.done.try_take().expect("supervisor: its file worker has ended"); + self.owed -= self.finished.swap(0, Ordering::Acquire); } } @@ -800,9 +807,9 @@ struct Instance { child: Child, /// The job it runs in, kept while it runs: what holds its row's floor. _job: toyos::job::Job, - /// A disk driver's [`toyos_quiesce::DRIVER_STOP`] pipe: closing it tells the - /// driver the machine stops. - stop: Option, + /// The waker of a disk driver's [`toyos_quiesce::DRIVER_STOP`] bell: + /// closing it tells the driver the machine stops. + stop: Option, } /// A service's kept acceptors, shared with what [`Grants`] mints on them. @@ -1199,7 +1206,7 @@ impl<'a> Supervisor<'a> { /// One `drives` process of `row`, holding `function`'s claim under /// `entry`'s spelling. - fn start_instance(&mut self, row: &'static Program, entry: PciMatch, function: Pci) -> Result<(Child, Option, toyos::job::Job), String> { + fn start_instance(&mut self, row: &'static Program, entry: PciMatch, function: Pci) -> Result<(Child, Option, toyos::job::Job), String> { if self.services.len() + self.instances.len() >= MAX_CHILDREN { return Err(format!("the supervisor watches {MAX_CHILDREN} processes already")); } @@ -1214,9 +1221,9 @@ impl<'a> Supervisor<'a> { let mut stop = None; if is_disk_driver(row) { extra.ports.push((toyos_blockring::disk::PORT.to_string(), self.grants.disk(function.arrival)?)); - let (read, write) = toyos::pipe_pair().map_err(|e| format!("no pipe for its stop: {e:?}"))?; - extra.stop = Some(read); - stop = Some(write); + let (waker, bell) = wake::pair().map_err(|e| format!("no event for its stop: {e:?}"))?; + extra.stop = Some(bell); + stop = Some(waker); } let launcher = Minting::machine(&self.minted, row, self.sessions.machine()); let served = Served::Keep { acceptors: &[], owed: &[] }; @@ -1906,7 +1913,7 @@ impl<'a> Supervisor<'a> { let until = Instant::now() + Duration::from_millis(toyos_quiesce::DRIVERS_MS); let mut waiting: Vec = Vec::new(); for instance in &mut self.instances { - // The pipe's end closing is the word. + // The waker's closing is the word. if instance.stop.take().is_some() { waiting.push(instance.function.arrival); } @@ -3166,14 +3173,14 @@ fn start<'a>( (log, ring) } }; - // The ring's end is its writer's: a pipe whose one write end the program + // The ring's end is its writer's: an event whose one waker the program // holds under a label no child inherits. let mut alive = None; if ring.is_some() { - let (read, write) = toyos::pipe_pair() - .map_err(|e| std::io::Error::other(format!("no pipe for its log's end: {e:?}")))?; - command.endow(ALIVE, write.into()); - alive = Some(read); + let (waker, bell) = wake::pair() + .map_err(|e| std::io::Error::other(format!("no event for its log's end: {e:?}")))?; + command.endow(ALIVE, waker.into()); + alive = Some(bell); } let child = command.spawn()?; @@ -3511,8 +3518,8 @@ struct Extra { /// SYSTEM's ROOT image, a duplicate. root: Option, ports: Vec<(String, Connector)>, - /// A disk driver's end of its [`toyos_quiesce::DRIVER_STOP`] pipe. - stop: Option, + /// The bell of a disk driver's [`toyos_quiesce::DRIVER_STOP`]. + stop: Option, /// Handles endowed by label. endow: Vec<(&'static str, toyos::OwnedHandle)>, } diff --git a/system/supervisor/src/sessions.rs b/system/supervisor/src/sessions.rs index 61f960810fd..2c93014325d 100644 --- a/system/supervisor/src/sessions.rs +++ b/system/supervisor/src/sessions.rs @@ -28,7 +28,8 @@ use toyos::launch::LAUNCHER; use toyos::namespace::{self, Namespace}; use toyos::poller::{Poller, READABLE}; use toyos::port::{self, Acceptor, Connector}; -use toyos::{say, AsHandle, BorrowedHandle, OwnedHandle, Pipe}; +use toyos::wake::{self, Bell, Waker}; +use toyos::{say, AsHandle, BorrowedHandle, OwnedHandle}; use toyos_abi::syscall; use toyos_abi::Rights; use toyos_manifest::launch::{Authority, Screen, Session}; @@ -190,30 +191,30 @@ impl Logins { /// /// **One thread per [`PER_WATCHER`] apps**, each waiting on its own poller, so /// the set is bounded by the process table and not by one poller's width. An -/// end reaches the loop through one pipe, which it watches under +/// end reaches the loop as a ring of one bell, which it watches under /// `TOKEN_INSTANCES`. pub(crate) struct Watch { - /// Readable once per end. - pub ended: Pipe, - notify: Arc, + /// Rung once an end is sent. + pub ended: Bell, + notify: Arc, ends: mpsc::Receiver<(u64, Session)>, sender: mpsc::Sender<(u64, Session)>, watchers: Vec, } -/// What one watcher thread's poller holds beside its wake pipe. +/// What one watcher thread's poller holds beside its bell. const PER_WATCHER: usize = Poller::MAX_HANDLES as usize - 1; const WAKE: u64 = u64::MAX; struct Watcher { add: mpsc::Sender<(u64, Session, OwnedHandle)>, - wake: Pipe, + wake: Waker, held: Arc, } impl Watch { pub fn new() -> Self { - let (ended, notify) = toyos::pipe_pair().expect("supervisor: no pipe for its apps' ends"); + let (notify, ended) = wake::pair().expect("supervisor: no event for its apps' ends"); let (sender, ends) = mpsc::channel(); Self { ended, notify: Arc::new(notify), ends, sender, watchers: Vec::new() } } @@ -230,12 +231,12 @@ impl Watch { let watcher = &self.watchers[at]; watcher.held.fetch_add(1, Ordering::Relaxed); watcher.add.send((instance, session, handle)).expect("supervisor: a watcher of its apps has ended"); - let _ = watcher.wake.write_nonblock(&[1]); + watcher.wake.wake(); } fn watcher(&self) -> Watcher { let (add, added) = mpsc::channel::<(u64, Session, OwnedHandle)>(); - let (woken, wake) = toyos::pipe_pair().expect("supervisor: no pipe for a watcher of its apps"); + let (waker, woken) = wake::pair().expect("supervisor: no event for a watcher of its apps"); let held = Arc::new(AtomicUsize::new(0)); let (notify, sender, count) = (Arc::clone(&self.notify), self.sender.clone(), Arc::clone(&held)); std::thread::Builder::new() @@ -253,8 +254,7 @@ impl Watch { ready.clear(); poller.wait(1, u64::MAX, |token| ready.push(token)); if ready.contains(&WAKE) { - let mut drained = [0u8; 64]; - while woken.read_nonblock(&mut drained).is_ok_and(|n| n > 0) {} + woken.take(); } let mut at = apps.len(); while at > 0 { @@ -265,12 +265,12 @@ impl Watch { let (instance, session, _) = apps.swap_remove(at); count.fetch_sub(1, Ordering::Relaxed); sender.send((instance, session)).expect("supervisor: its loop has ended"); - let _ = notify.write_nonblock(&[1]); + notify.wake(); } } }) .expect("supervisor: a watcher of its apps could not be started"); - Watcher { add, wake, held } + Watcher { add, wake: waker, held } } } @@ -495,8 +495,7 @@ impl Supervisor<'_> { /// Every app end the watch saw, told to its session's owner. pub(crate) fn apps_ended(&mut self) { - let mut drained = [0u8; 64]; - while self.watch.ended.read_nonblock(&mut drained).is_ok_and(|n| n > 0) {} + self.watch.ended.take(); while let Ok((instance, session)) = self.watch.ends.try_recv() { if let Some(login) = self.logins.of(session) { if let Some(link) = login.link.conn() { diff --git a/system/terminal/src/main.rs b/system/terminal/src/main.rs index 13c234238aa..e64416b77e1 100644 --- a/system/terminal/src/main.rs +++ b/system/terminal/src/main.rs @@ -21,7 +21,7 @@ use std::process::Command; use std::time::Duration; use terminal::Console; -use toyos::poller::{Poller, READABLE}; +use toyos::poller::{wait_one, Poller, READABLE}; use toyos::port; use toyos::surface::{self, Delivery, Host, Leaving, Notice}; use window::Window; @@ -254,12 +254,10 @@ fn main() { /// Wait for the shell to end, saying every [`STILL_WAITING`] how long it has /// been outstanding. fn wait_for(child: &mut std::process::Child) { - let poller = Poller::new(1); let mut waited = Duration::ZERO; loop { - poller.watch(&*child, READABLE, 0); - let mut ended = false; - poller.wait(1, STILL_WAITING.as_nanos() as u64, |_| ended = true); + let ended = wait_one(&*child, READABLE, STILL_WAITING.as_nanos() as u64) + .expect("terminal: the kernel refused a wait on the shell it started"); if ended { return; } diff --git a/system/usbd/src/main.rs b/system/usbd/src/main.rs index 8d8fcebf554..d0b58d5cba5 100644 --- a/system/usbd/src/main.rs +++ b/system/usbd/src/main.rs @@ -160,7 +160,7 @@ fn main() { let host = Process { began: Instant::now(), name: format!("usbd {at}") }; let name = host.name.clone(); let acceptor = endow::acceptor(toyos_inspect::USB.port); - let stop = Endowments::get().take::(toyos_quiesce::DRIVER_STOP); + let stop = Endowments::get().take::(toyos_quiesce::DRIVER_STOP); let link = Link::new(name.clone()); // xHCI 1.2 §5.2.1: the capability registers are in BAR 0. let (bar, _bar) = match Bar::map(&dev, 0, info.bar_bytes[0]) { @@ -198,7 +198,7 @@ fn main() { /// A refusal this driver can name, before it serves anything: said once, the /// block service told this controller is unusable, and nothing served. -fn not_serving(name: &str, why: &str, mut link: Option, stop: Option) -> ! { +fn not_serving(name: &str, why: &str, mut link: Option, stop: Option) -> ! { println!("{name}: NOT SERVING — {why}"); if let Some(link) = &mut link { link.settle(Settled::Unusable); @@ -208,7 +208,7 @@ fn not_serving(name: &str, why: &str, mut link: Option, stop: Option, stop: Option) -> ! { +fn idle(mut link: Option, stop: Option) -> ! { let poller = Poller::new(2); loop { if let Some(link) = &mut link { @@ -245,7 +245,7 @@ fn serve( usb: &Driver, acceptor: Option, mut link: Option, - stop: Option, + stop: Option, ) -> ! { let mut tasks = usb.tasks(); let mut disks: BTreeMap = BTreeMap::new(); diff --git a/tests/guest/libc/spawn_children_cap.c b/tests/guest/libc/spawn_children_cap.c index 5fecf1464f6..0c8234805d0 100644 --- a/tests/guest/libc/spawn_children_cap.c +++ b/tests/guest/libc/spawn_children_cap.c @@ -1,7 +1,7 @@ /* At most 254 children wait unwaited, which libc's own thread watches - beside its control pipe and the notice: the 255th spawn is refused EAGAIN, + beside its control event and the notice: the 255th spawn is refused EAGAIN, and beside the 254 a poll of 255 descriptors, a poller's handles but the - wake pipe's, answers. */ + wake event's, answers. */ #include #include "check.h" diff --git a/tests/guest/rust-tests/Cargo.lock b/tests/guest/rust-tests/Cargo.lock index 5c09e74f4e2..cf129ddd534 100644 --- a/tests/guest/rust-tests/Cargo.lock +++ b/tests/guest/rust-tests/Cargo.lock @@ -436,6 +436,18 @@ dependencies = [ "libc", ] +[[package]] +name = "mio" +version = "1.2.0" +source = "git+https://github.com/ToyOSOrg/mio?branch=toyos#fe42def7bec9a1f01bfff764256fcdce77e7f7e1" +dependencies = [ + "libc", + "log", + "toyos-abi", + "wasi", + "windows-sys 0.61.2", +] + [[package]] name = "ml-kem" version = "0.3.2" @@ -971,6 +983,7 @@ dependencies = [ "libloading", "logkeeper-api", "memmap2", + "mio", "ml-kem", "ring", "toyos", diff --git a/tests/guest/rust-tests/Cargo.toml b/tests/guest/rust-tests/Cargo.toml index 4b680d9b18a..b2def872cd4 100644 --- a/tests/guest/rust-tests/Cargo.toml +++ b/tests/guest/rust-tests/Cargo.toml @@ -29,6 +29,9 @@ toyos-pci-claim = { path = "../../../lib/pci-claim" } toyos-pci = { path = "../../../kernel/pci" } libloading = { git = "https://github.com/ToyOSOrg/rust_libloading", branch = "toyos-sdk-0.12" } cpal = { git = "https://github.com/ToyOSOrg/cpal", branch = "toyos-0.18.0" } +# Its ToyOS selector and waker, which no program in the image builds, for +# `mio_waker_coalesces`. +mio = { git = "https://github.com/ToyOSOrg/mio", branch = "toyos", features = ["os-poll"] } memmap2 = "0.9" # `ring`'s primitives, for `ring_kat`'s known answers and as `crypto_parity`'s # reference. diff --git a/tests/guest/rust-tests/src/bin/cpal_drop_unreleased.rs b/tests/guest/rust-tests/src/bin/cpal_drop_unreleased.rs index 6303092b9ea..8cb9392c5af 100644 --- a/tests/guest/rust-tests/src/bin/cpal_drop_unreleased.rs +++ b/tests/guest/rust-tests/src/bin/cpal_drop_unreleased.rs @@ -2,11 +2,11 @@ //! //! This binary is the stream's server: it describes its device and answers the //! open as soundserver does, and -//! then holds the signal pipe open without writing it until the client's +//! then holds the stream's signal without raising it until the client's //! `Drop` has come back — a soundserver whose mix loop has stopped. Its child, the //! same binary with `client`, reaches it as `soundserver`, builds a stream through //! cpal and drops it unplayed. The stream thread sends the close and waits on -//! the pipe; `Drop` has to come back with the refusal by name through the error +//! the signal; `Drop` has to come back with the refusal by name through the error //! callback rather than wait for good. One signal then has to end the stream //! thread while its process lives, so the stream's connection closes as it //! would for soundserver. @@ -95,14 +95,14 @@ fn serve() { AudioSlotHeader::SIZE + OPENED.slot_count as usize * OPENED.client_period_bytes as usize, ) .expect("a ring"); - let (signal_read, signal_write) = toyos::pipe_pair().expect("a signal pipe"); - conn.send_with(MSG_STREAM_OPENED, &OPENED, [ring.share().expect("the ring to share"), signal_read.into()]) + let (signal, signal_bell) = toyos::wake::pair().expect("a signal event"); + conn.send_with(MSG_STREAM_OPENED, &OPENED, [ring.share().expect("the ring to share"), signal_bell.into()]) .expect("the open answered with its handles"); let close = conn.recv_header().expect("the client's close"); assert_eq!(close.msg_type, MSG_STREAM_CLOSE, "the dropped stream did not close"); - // The write end is held until the client has gone, so nothing but its + // The signal side is held until the client has gone, so nothing but its // own deadline can end its wait. The client says the refusal once `Drop` // has returned with it. let mut said = String::new(); @@ -112,7 +112,7 @@ fn serve() { assert!(said.starts_with("client: "), "the client said {said:?} before its drop came back"); print!("{said}"); - signal_write.write(&[1]).expect("a signal to the stream thread"); + signal.ring().expect("a signal to the stream thread"); match conn.recv_header() { Err(IpcError::Disconnected) => {} other => panic!( @@ -124,10 +124,10 @@ fn serve() { // was the stream thread's. drop(child.stdin.take()); let status = child.wait().expect("wait for the client"); - drop(signal_write); + drop(signal); assert!(status.success(), "the client exited {status:?}"); println!( - "cpal's drop came back while its server held the signal pipe, and the next signal \ + "cpal's drop came back while its server held the signal, and the next signal \ ended its stream thread" ); } diff --git a/tests/guest/rust-tests/src/bin/event_sides.rs b/tests/guest/rust-tests/src/bin/event_sides.rs new file mode 100644 index 00000000000..88fa851c4b4 --- /dev/null +++ b/tests/guest/rust-tests/src/bin/event_sides.rs @@ -0,0 +1,120 @@ +//! An event's two sides on the real kernel: what `kernel::event`'s host tests +//! cannot show, which is the wake, the watch and the end across processes. +//! +//! **A parked wait is woken.** A thread parked in `SYS_HANDLE_WAIT` on a bell +//! is woken by a ring from another process, and by that process's end, which +//! closes its waker. Each happens only once the thread is parked, as +//! `pipe_other_end_gone` orders its leavings, and a thread left parked is a +//! hang the runner's ceiling reds. +//! +//! **A watch takes nothing.** An inbox watch answered readable, and a wait +//! with no timeout to spend, leave the ring pending for the take that follows. +//! +//! **An end is never a ring.** A bell rung once and then left by its last +//! waker answers the ring, and then only `Gone`; a waker whose bell has gone +//! answers `Gone` too. + +use std::io::Read; +use std::os::toyos::process::CommandExt; +use std::process::{Command, Stdio}; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::thread; + +use toyos::endow::{Endowments, SYSCAP_LABEL}; +use toyos::poller::{wait_one, Poller, READABLE}; +use toyos::syscap::SysCap; +use toyos::wake::{self, Bell, Hungup, Waker}; + +#[path = "../roster.rs"] +mod roster; + +const SELF_PATH: &str = "/system/bin/test_rs_event_sides"; +const TOKEN: u64 = 7; +/// What the child is endowed its waker as. +const RING_LABEL: &str = "ring"; +/// What the child is endowed its bell as. +const HEAR_LABEL: &str = "hear"; + +fn main() { + match std::env::args().nth(1).as_deref() { + Some("child") => child(), + Some(other) => panic!("unknown role {other:?}"), + None => test(), + } +} + +/// Rings its waker once its standard input says so, then waits until it ends, +/// holding its waker and its bell till it exits. +fn child() { + let endowed = Endowments::get(); + let ring: Waker = endowed.take(RING_LABEL).expect("the waker this test endows"); + let _hear: Bell = endowed.take(HEAR_LABEL).expect("the bell this test endows"); + let mut stdin = std::io::stdin(); + let mut go = [0u8; 1]; + stdin.read_exact(&mut go).expect("the parent's word to ring"); + ring.ring().expect("the parent holds the bell"); + let mut rest = Vec::new(); + stdin.read_to_end(&mut rest).expect("the parent's end of standard input"); +} + +fn test() { + let cap: SysCap = Endowments::get() + .take(SYSCAP_LABEL) + .expect("test-runner endows every binary it spawns a system capability"); + + let (waker, bell) = wake::pair().expect("an event"); + assert!(!wait_one(&bell, READABLE, 1_000_000).expect("a wait on a bell"), "an unrung bell answered a wait"); + waker.wake(); + waker.wake(); + let poller = Poller::new(1); + poller.watch(&bell, READABLE, TOKEN); + let mut answers = Vec::new(); + poller.wait_answers(1, u64::MAX, |token, said| answers.push((token, said))); + assert_eq!(answers, [(TOKEN, Ok(READABLE))], "a rung bell's watch"); + assert!(wait_one(&bell, READABLE, 0).expect("a look at a bell"), "a watch took the ring"); + assert_eq!(bell.try_take(), Ok(true), "two rings before a take are one, and a look took none"); + assert_eq!(bell.try_take(), Ok(false), "a take took the ring"); + println!(" two rings are one, and a watch and a look take nothing"); + + let (parent_waker, parent_bell) = wake::pair().expect("the parent's event"); + let (child_waker, child_bell) = wake::pair().expect("the child's event"); + let mut child = Command::new(SELF_PATH) + .arg("child") + .endow(RING_LABEL, child_waker.into()) + .endow(HEAR_LABEL, parent_bell.into()) + .stdin(Stdio::piped()) + .spawn() + .expect("spawn the child"); + let mut stdin = child.stdin.take().expect("the child's standard input"); + woken(&cap, &child_bell, || std::io::Write::write_all(&mut stdin, b"g").expect("tell the child to ring")); + assert_eq!(child_bell.try_take(), Ok(true), "the child's ring, still pending after the wait that saw it"); + println!(" a parked wait was woken by another process's ring"); + + woken(&cap, &child_bell, || drop(stdin)); + assert!(child.wait().expect("wait the child").success(), "the child exited nonzero"); + assert_eq!(child_bell.try_take(), Err(Hungup), "a bell whose last waker went answered a ring nobody rang"); + assert!(wait_one(&child_bell, READABLE, 0).expect("a look at a bell"), "a bell with no waker left is not readable for good"); + assert_eq!(child_bell.try_take(), Err(Hungup), "a bell with no waker left answered a ring"); + assert_eq!(parent_waker.ring(), Err(Hungup), "a waker whose bell went answered a ring as heard"); + println!("event_sides: a ring wakes and coalesces, a watch takes nothing, and an end is never a ring"); +} + +/// A thread parked in one `SYS_HANDLE_WAIT` on `bell` is woken by `cause`, +/// which runs once the thread is parked. +fn woken(cap: &SysCap, bell: &Bell, cause: impl FnOnce()) { + let entering = AtomicBool::new(false); + let woke = thread::scope(|s| { + let waiter = s.spawn(|| { + entering.store(true, Ordering::Release); + wait_one(bell, READABLE, u64::MAX).expect("a wait on a bell") + }); + // The roster first: its syscall is the loop's preemption point. + roster::await_true(|| { + entering.load(Ordering::Acquire) + && roster::my_threads(cap).iter().any(|&(is_thread, state)| is_thread && state == roster::BLOCKED) + }); + cause(); + waiter.join().expect("the waiter thread panicked") + }); + assert!(woke, "a wait with no timeout returned unready"); +} diff --git a/tests/guest/rust-tests/src/bin/kernel_objects_refused.rs b/tests/guest/rust-tests/src/bin/kernel_objects_refused.rs index 186f54c68f7..671e56dbb37 100644 --- a/tests/guest/rust-tests/src/bin/kernel_objects_refused.rs +++ b/tests/guest/rust-tests/src/bin/kernel_objects_refused.rs @@ -63,6 +63,7 @@ fn main() { let refused = [ ("a pipe", syscall::pipe().map(|_| ())), ("a port", syscall::port_create().map(|_| ())), + ("an event", syscall::event_create().map(|_| ())), ("a directory", syscall::mkdir(b"/tmp/kernel_objects_refused_more")), ("a pipe's first write, its frame", syscall::write(unwritten.write, b"x").map(|_| ())), ("a /tmp file's page", syscall::write(file, &[0x5a; PAGE]).map(|_| ())), diff --git a/tests/guest/rust-tests/src/bin/libc_recv_beside_close.rs b/tests/guest/rust-tests/src/bin/libc_recv_beside_close.rs index b4cc8a5dd71..c82c7bef7f8 100644 --- a/tests/guest/rust-tests/src/bin/libc_recv_beside_close.rs +++ b/tests/guest/rust-tests/src/bin/libc_recv_beside_close.rs @@ -14,7 +14,7 @@ //! //! **This test is netstack for its child**, as `stream_handles` is: the //! child's `connect` reaches a port of this test's, which answers with the -//! region judged and two bells, so no NIC and no peer is needed. The order is +//! region judged and its bells, so no NIC and no peer is needed. The order is //! awaited, never timed: a look of the child's `recv` that finds the ring //! empty arms the ring's want for data, which the test cleared, and that is //! past the descriptor's lookup. Only then does the child close the @@ -31,13 +31,13 @@ use std::io::{BufRead, BufReader, Write}; use std::process::{Command, Stdio}; use std::sync::atomic::{AtomicI32, Ordering}; -use toyos::net::stream::{Server, Words, LAYOUT, RECEIVE_RING, REGION_BYTES, SERVER_WATCH}; -use toyos::poller::Poller; +use toyos::net::stream::{Server, Words, LAYOUT, RECEIVE_RING, REGION_BYTES}; +use toyos::wake::{self, Hungup}; use toyos::net::{MsgType, RespType, TcpConnectRequest, TcpConnectResponse}; use toyos::shm::SharedMemory; -use toyos::{namespace, port, AsHandle, OwnedHandle}; +use toyos::{namespace, port, OwnedHandle}; use std::os::toyos::process::CommandExt; -use toyos_abi::syscall::{self, SVC_LABEL}; +use toyos_abi::syscall::SVC_LABEL; const SELF_PATH: &str = "/system/bin/test_rs_libc_recv_beside_close"; @@ -135,9 +135,10 @@ fn test(role: &str) { // SAFETY: the mapping is `REGION_BYTES`, 2 MiB aligned, and its header is reached only as atomics. let words = unsafe { &*(region.as_ptr() as *const Words) }; let mut server = Server::new(words, LAYOUT); - let (receive_bell, receive) = toyos::pipe_pair().expect("the receive bell"); - let (send_bell, _send) = toyos::pipe_pair().expect("the send bell"); - let bells = [OwnedHandle::from(receive_bell), OwnedHandle::from(send_bell)]; + let (receive, receive_bell) = wake::pair().expect("the receive bell"); + let (_send, send_bell) = wake::pair().expect("the send bell"); + let (rung_waker, rung) = wake::pair().expect("the rung bell"); + let bells = [OwnedHandle::from(receive_bell), send_bell.into(), rung_waker.into()]; conn.send_with(RespType::Result as u32, &TcpConnectResponse { local_port: 49_152, _pad: 0 }, bells) .expect("answer the connect with the bells"); assert_eq!(line(&mut out), "connected"); @@ -158,7 +159,7 @@ fn test(role: &str) { // whose wait watches the stream looks at the second ring. for _ in 0..2 { words[RECEIVE_WANT].store(0, Ordering::Release); - syscall::write(receive.as_handle().raw(), &[1]).expect("ring the reader"); + receive.ring().expect("ring the reader"); if !looked(words) { let _ = child.kill(); panic!("the {role}'s recv never looked at its stream again once its descriptor's number named a pipe"); @@ -167,7 +168,7 @@ fn test(role: &str) { region.bytes().write(RECEIVE_RING, PEER); let _ = server.publish(words, PEER.len() as u32).expect("the client keeps the protocol"); - syscall::write(receive.as_handle().raw(), &[1]).expect("ring the reader"); + receive.ring().expect("ring the reader"); let said = line(&mut out); if said != format!("received {}", String::from_utf8_lossy(PEER)) { @@ -175,15 +176,17 @@ fn test(role: &str) { panic!("the {role} ended {status:?} after the close, having said {said:?}"); } // The recv returned, and with it the socket's last hold: libc's copy of - // `toyos`, which holds the stream's end, lets it go, so its connection - // hangs up while the child lives. - let poller = Poller::new(1); - poller.watch(&conn, SERVER_WATCH, 0); - let mut answered = false; - poller.wait(1, CEILING_NANOS, |_| answered = true); - let mut bytes = [0u8; 64]; - while syscall::read_nonblock(conn.as_handle().raw(), &mut bytes).is_ok_and(|n| n > 0) {} - let hung_up = answered && syscall::read_nonblock(conn.as_handle().raw(), &mut bytes) == Ok(0); + // `toyos`, which holds the stream's end, lets it go, so its waker of the + // bell netstack watches it on closes while the child lives. + let until = toyos_abi::clock::nanos_since_boot() + CEILING_NANOS; + let hung_up = loop { + let left = until.saturating_sub(toyos_abi::clock::nanos_since_boot()); + match rung.wait_for(left) { + Ok(true) => continue, + Ok(false) => break false, + Err(Hungup) => break true, + } + }; writeln!(tell, "exit").expect("tell the child to exit"); let status = child.wait().expect("wait the child"); assert!(hung_up, "the stream's end outlived its descriptor's close and its last call, in a child that lives on"); diff --git a/tests/guest/rust-tests/src/bin/mio_waker_coalesces.rs b/tests/guest/rust-tests/src/bin/mio_waker_coalesces.rs new file mode 100644 index 00000000000..d4b65784b27 --- /dev/null +++ b/tests/guest/rust-tests/src/bin/mio_waker_coalesces.rs @@ -0,0 +1,38 @@ +//! mio's ToyOS waker, from the `ToyOSOrg/mio` fork, which no program in the +//! image builds: any number of wakes before a poll are one event, the poll +//! that reports it takes it, and a wake after that is reported again. +//! +//! Red where the selector leaves the waker's readiness standing: a poll with +//! nothing new to say then reports the waker's token again, for good. + +use std::time::Duration; + +use mio::{Events, Poll, Token, Waker}; + +const WAKE: Token = Token(7); +const WAKES: usize = 10_000; +/// A hang's ceiling: the waker is rung before the poll that waits on it. +const CEILING: Duration = Duration::from_secs(10); + +fn main() { + let mut poll = Poll::new().expect("a poll"); + let waker = Waker::new(poll.registry(), WAKE).expect("a waker"); + let mut events = Events::with_capacity(8); + + for _ in 0..WAKES { + waker.wake().expect("a wake"); + } + poll.poll(&mut events, Some(CEILING)).expect("a poll"); + let tokens: Vec = events.iter().map(|event| event.token()).collect(); + assert_eq!(tokens, [WAKE], "{WAKES} wakes before a poll"); + + poll.poll(&mut events, Some(Duration::ZERO)).expect("a poll"); + let tokens: Vec = events.iter().map(|event| event.token()).collect(); + assert!(tokens.is_empty(), "a poll after the wake was reported reported {tokens:?}"); + + waker.wake().expect("a wake"); + poll.poll(&mut events, Some(CEILING)).expect("a poll"); + let tokens: Vec = events.iter().map(|event| event.token()).collect(); + assert_eq!(tokens, [WAKE], "a wake after the last was reported"); + println!("mio_waker_coalesces: {WAKES} wakes were one event, the poll that reported it took it, and the next wake was reported"); +} diff --git a/tests/guest/rust-tests/src/bin/netstack_connect_killed.rs b/tests/guest/rust-tests/src/bin/netstack_connect_killed.rs new file mode 100644 index 00000000000..5b756161a36 --- /dev/null +++ b/tests/guest/rust-tests/src/bin/netstack_connect_killed.rs @@ -0,0 +1,104 @@ +//! A client killed while its connect is unanswered gives its place back to +//! the real netstack at once, not when the connect gives up. +//! +//! Until netstack answers a connect it holds the waker of the bell it later +//! watches the holder on, so that bell cannot report the holder leaving; the +//! holder's connection is what does. Two children connect to an address on +//! QEMU's user network that nothing answers, and each connect stays pending +//! until netstack gives the address up, within seconds. One is killed: its +//! place, read through netstack's `inspect`, comes back while the other still +//! runs and still holds its own, so the place came back for the kill and not +//! for the connect ending, which would have ended both. The survivor is +//! killed last and gives its place back too. +//! +//! The child connects only once netstack holds its lease: before it, a +//! connect is refused at once and holds no place. netstack announces neither +//! its lease nor a change of its count, so the parent asks it again until the +//! answer it waits for or the ceiling. + +use std::collections::BTreeMap; +use std::process::{Command, Stdio}; +use std::time::{Duration, Instant}; + +use toyos::endow; +use toyos_inspect::{Value, MSG_INSPECT, MSG_SNAPSHOT, NET}; + +const SELF_PATH: &str = "/system/bin/test_rs_netstack_connect_killed"; + +/// On QEMU's user network, where no host answers for it. +const NOWHERE: &str = "10.0.2.77:80"; + +/// A hang ceiling under QEMU, nothing more. The kill has to land before +/// netstack gives the address up (`nud::BROADCAST_SOLICIT` requests +/// `nud::RETRANS` apart, about 3 s), and a run that misses it is red at "a +/// child ended before it was killed". +const CEILING: Duration = Duration::from_secs(20); + +/// Between two questions to netstack. +const ASK_AGAIN: Duration = Duration::from_millis(20); + +fn main() { + match std::env::args().nth(1).as_deref() { + Some("connect") => { + let got = std::net::TcpStream::connect(NOWHERE); + panic!("a connect to {NOWHERE} answered {got:?}"); + } + Some(other) => panic!("unknown role {other:?}"), + None => test(), + } +} + +fn test() { + until(|snapshot| snapshot.get("net.lease.held") == Some(&Value::Bool(true)), "netstack held a lease"); + let before = held(&snapshot()); + let connect = || Command::new(SELF_PATH).arg("connect").stdin(Stdio::null()).spawn().expect("a connecting child"); + let (mut killed, mut survivor) = (connect(), connect()); + until(|snapshot| held(snapshot) == before + 2, "both children's connects held a place"); + kill(&mut killed); + until(|snapshot| held(snapshot) == before + 1, "the killed child's place came back"); + assert!(survivor.try_wait().expect("the survivor's state").is_none(), "the survivor's connect ended: something answered"); + assert_eq!(held(&snapshot()), before + 1, "the survivor's place went with the other's"); + kill(&mut survivor); + until(|snapshot| held(snapshot) == before, "the survivor's place came back"); + println!("netstack_connect_killed: a client killed mid-connect gave its place back"); +} + +fn kill(child: &mut std::process::Child) { + assert!(child.try_wait().expect("the child's state").is_none(), "a child ended before it was killed"); + child.kill().expect("kill a connecting child"); + let status = child.wait().expect("a killed child"); + assert!(!status.success(), "a killed child ended {status:?}"); +} + +/// Ask netstack until `wanted` holds of its snapshot, or panic at the +/// ceiling saying `what` never happened. +fn until(wanted: impl Fn(&BTreeMap) -> bool, what: &str) { + let deadline = Instant::now() + CEILING; + loop { + let now = snapshot(); + if wanted(&now) { + return; + } + assert!(Instant::now() < deadline, "{what} not within {CEILING:?}: netstack holds {:?} place(s)", now.get("net.places.held")); + std::thread::sleep(ASK_AGAIN); + } +} + +/// netstack's snapshot now. +fn snapshot() -> BTreeMap { + let conn = endow::service(NET.port).expect("the image gives test-runner netstack"); + conn.signal(MSG_INSPECT).expect("the inspect request"); + let header = conn.recv_header().expect("netstack's answer"); + assert_eq!(header.msg_type, MSG_SNAPSHOT, "netstack answered something that is not a snapshot"); + let mut buf = vec![0u8; header.len() as usize]; + let n = conn.recv_bytes(&header, &mut buf).expect("the snapshot's bytes"); + toyos_inspect::decode(&buf[..n], NET).expect("a snapshot for `net`") +} + +/// The places a snapshot says netstack holds. +fn held(snapshot: &BTreeMap) -> u64 { + match snapshot.get("net.places.held") { + Some(Value::U64(held)) => *held, + other => panic!("netstack's snapshot holds places.held as {other:?}"), + } +} diff --git a/tests/guest/rust-tests/src/bin/stream_handles.rs b/tests/guest/rust-tests/src/bin/stream_handles.rs index b915f53d01b..1821e52ef62 100644 --- a/tests/guest/rust-tests/src/bin/stream_handles.rs +++ b/tests/guest/rust-tests/src/bin/stream_handles.rs @@ -27,11 +27,11 @@ //! reading them, and that read ends. `sshserver` runs every program so. //! //! **This test is netstack for its children**: a port of its own served to -//! each as `netstack`, which answers a connect with the region judged and two +//! each as `netstack`, which answers a connect with the region judged and its //! bells, an export with a holder made on its private port, and a departure //! with `Server::departed`, so no NIC and no peer is needed. Every step waits -//! on an event: a line a child says, a child's exit, a connection's frame or -//! its hang-up, each under a failure ceiling and never a fixed delay. It runs +//! on an event: a line a child says, a child's exit, a holder's ring, frame or +//! leaving, each under a failure ceiling and never a fixed delay. It runs //! in a guest because the claims are the kernel's: when a dead process's //! handles close, and what its fault ends. //! @@ -49,15 +49,13 @@ use std::process::{Child, ChildStdout, Command, Stdio}; use std::sync::atomic::Ordering; use toyos::ipc::{FrameRx, IpcError, RxStep}; -use toyos::net::stream::{ - Adopt, Server, Words, ADOPT, EXPORT, EXPORTED, LAYOUT, RECEIVE_RING, REGION_BYTES, RING, SEND_RING, - SERVER_WATCH, -}; +use toyos::net::stream::{Adopt, Server, Words, ADOPT, EXPORT, EXPORTED, LAYOUT, RECEIVE_RING, REGION_BYTES, SEND_RING}; use toyos::net::{MsgType, RespType, TcpConnectRequest, TcpConnectResponse}; use toyos::poller::{Poller, READABLE}; use toyos::shm::SharedMemory; use toyos::transport::session::{Holder, Rings}; -use toyos::{namespace, port, AsHandle, Connection, OwnedHandle, Pipe}; +use toyos::wake::{self, Bell, Hungup, Waker}; +use toyos::{namespace, port, AsHandle, Connection, OwnedHandle}; use toyos_abi::syscall::{self, SyscallError, SVC_LABEL}; const SELF_PATH: &str = "/system/bin/test_rs_stream_handles"; @@ -197,33 +195,33 @@ fn shares_within_a_process_and_refuses_moves() { assert_eq!(said, ["shared", "endow refused", "send refused"], "the client said otherwise"); assert!(status.success(), "the client exited {status:?}"); println!(" given out and taken back: one end; a duplicate of its handle endowed or sent: refused"); - hang_up_answers_the_server_watch(&netstack.holders[0].connection); + departure_answers_the_holders_bell(&netstack.holders[0].rung); } -/// netstack's watch of a holder's connection, made after its client left: -/// the kernel takes it and answers it, and the connection then reads its end. -fn hang_up_answers_the_server_watch(conn: &Connection) { +/// netstack's watch of a holder's bell, made after its client left: the +/// kernel takes it and answers it, and the bell then reads its wakers' end. +fn departure_answers_the_holders_bell(rung: &Bell) { let poller = Poller::new(1); - poller.watch(&conn, SERVER_WATCH, 0); + poller.watch(rung, READABLE, 0); let mut answers = Vec::new(); poller.wait_answers(1, CEILING_NANOS, |_, answer| answers.push(answer)); assert!(matches!(answers[..], [Ok(_)]), "the kernel answered netstack's watch {answers:?}"); - let mut bytes = [0u8; 64]; - while syscall::read_nonblock(conn.as_handle().raw(), &mut bytes).is_ok_and(|n| n > 0) {} - assert_eq!(syscall::read_nonblock(conn.as_handle().raw(), &mut bytes), Ok(0), "the hang-up is the connection's end"); - println!(" netstack's watch of a holder's connection: taken, and answered by the hang-up"); + while rung.try_take() == Ok(true) {} + assert_eq!(rung.try_take(), Err(Hungup), "the departure is the bell's wakers' end"); + println!(" netstack's watch of a holder's bell: taken, and answered by its departure"); } /// A bare `dup` of a stream's handle is no stream of the process: refused by -/// name, with nothing written on its connection before the process ends. +/// name, with nothing written on its connection nor rung before the process +/// ends. fn refuses_a_foreign_handle_writing_nothing() { let (mut child, netstack) = run("foreign", &[]); - let conn = &netstack.holders[0].connection; + let (conn, rung) = (&netstack.holders[0].connection, &netstack.holders[0].rung); let mut out = BufReader::new(child.stdout.take().expect("the child's stdout")); assert_eq!(line(&mut out), "duplicated"); - // What the connect may have rung, taken; from here the connection carries nothing but its end. + // What the connect may have rung, taken; from here the bell rings nothing but its end. + while rung.try_take() == Ok(true) {} let mut bytes = [0u8; 64]; - while syscall::read_nonblock(conn.as_handle().raw(), &mut bytes).is_ok_and(|n| n > 0) {} child.stdin.take().expect("the child's stdin").write_all(b"go\n").expect("tell the child"); let said = child.wait_with_output().expect("wait the child"); let stderr = String::from_utf8_lossy(&said.stderr); @@ -231,6 +229,7 @@ fn refuses_a_foreign_handle_writing_nothing() { assert!(stderr.contains("is no stream of this process's"), "refused, but not by name: {stderr}"); let read = syscall::read(conn.as_handle().raw(), &mut bytes); assert_eq!(read, Ok(0), "the refusal wrote on the stream's connection before its end"); + assert_eq!(rung.try_take(), Err(Hungup), "the refusal rang the stream before its end"); println!(" a foreign handle: refused by name, nothing written"); } @@ -368,14 +367,24 @@ fn hold() -> ! { panic!("hold's stdin ended: nothing closes it") } -/// One holder as this test keeps it: its connection and the write ends of -/// its two bells. +/// One holder as this test keeps it: its connection, the wakers of its two +/// bells, and the bell it rings. struct Holding { number: Holder, connection: Connection, frames: FrameRx<0>, - receive: Pipe, - send: Pipe, + receive: Waker, + send: Waker, + rung: Bell, +} + +/// A holder's three bells: what it is handed, in `BELLS`'s order, and the +/// wakers and bell this test keeps. +fn bells() -> ([OwnedHandle; 3], (Waker, Waker, Bell)) { + let (receive, receive_bell) = wake::pair().expect("a receive bell"); + let (send, send_bell) = wake::pair().expect("a send bell"); + let (rung_waker, rung) = wake::pair().expect("a rung bell"); + ([receive_bell.into(), send_bell.into(), rung_waker.into()], (receive, send, rung)) } /// A stream's server end as this test plays netstack, kept for the test's @@ -407,31 +416,30 @@ impl Netstack { fn ring(&self, rings: Rings) { for holding in &self.holders { if rings.receive { - let _ = holding.receive.write_nonblock(&[1]); + holding.receive.wake(); } if rings.send { - let _ = holding.send.write_nonblock(&[1]); + holding.send.wake(); } } } - /// The next frame on `number`'s connection that is no ring, or `None` for - /// its hang-up, each wait for one under the ceiling. + /// The next frame on `number`'s connection, or `None` for its leaving, + /// each wait for its bell under the ceiling. fn next_frame(&mut self, number: Holder) -> Option { let holding = self.holder(number); loop { match holding.frames.pump(&holding.connection) { - RxStep::Frame { msg_type: RING, .. } => continue, RxStep::Frame { msg_type, .. } => return Some(msg_type), RxStep::Eof => return None, RxStep::Malformed => panic!("a holder wrote a frame no holder writes"), RxStep::Idle => {} } - let poller = Poller::new(1); - poller.watch(&holding.connection, SERVER_WATCH, 0); - let mut answered = false; - poller.wait(1, CEILING_NANOS, |_| answered = true); - assert!(answered, "a holder said nothing within the ceiling"); + match holding.rung.wait_for(CEILING_NANOS) { + Ok(true) => {} + Ok(false) => panic!("a holder said nothing within the ceiling"), + Err(Hungup) => return None, + } } } @@ -442,13 +450,12 @@ impl Netstack { let number = Holder::new(self.holders.len() as u16 + 1).expect("a holder's number"); let theirs = self.exports.1.open("export").expect("a connection of the private port"); let ours = self.exports.0.accept().expect("the private port's connection"); - let (receive_read, receive) = toyos::pipe_pair().expect("a receive bell"); - let (send_read, send) = toyos::pipe_pair().expect("a send bell"); + let ([receive_bell, send_bell, rung_waker], (receive, send, rung)) = bells(); let adopt = Adopt { holder: number.number(), peer_port: 80, peer_addr: [192, 0, 2, 2], local_port: 49_152, _pad: 0 }; let region = self.region.share().expect("a handle of the region"); - let handles = [region, OwnedHandle::from(receive_read), OwnedHandle::from(send_read)]; + let handles = [region, receive_bell, send_bell, rung_waker]; ours.send_with(ADOPT, &adopt, handles).expect("queue the holder's adoption"); - self.holders.push(Holding { number, connection: ours, frames: FrameRx::new(), receive, send }); + self.holders.push(Holding { number, connection: ours, frames: FrameRx::new(), receive, send, rung }); self.holder(asker).connection.signal_with(EXPORTED, [theirs.into_owned()]).expect("answer the export"); number } @@ -505,14 +512,12 @@ fn run(role: &str, peer: &[u8]) -> (Child, Netstack) { let mut server = Server::new(words_of(®ion), LAYOUT); region.bytes().write(RECEIVE_RING, peer); let _ = server.publish(words_of(®ion), peer.len() as u32).expect("the client keeps the protocol"); - let (receive_read, receive) = toyos::pipe_pair().expect("the receive bell"); - let (send_read, send) = toyos::pipe_pair().expect("the send bell"); - let bells = [OwnedHandle::from(receive_read), OwnedHandle::from(send_read)]; - conn.send_with(RespType::Result as u32, &TcpConnectResponse { local_port: 49_152, _pad: 0 }, bells) + let (handed, (receive, send, rung)) = bells(); + conn.send_with(RespType::Result as u32, &TcpConnectResponse { local_port: 49_152, _pad: 0 }, handed) .expect("answer the connect with the bells"); let (exports, export_connector) = port::create().expect("the private port"); let names = namespace::build().add("export", &export_connector).finish().expect("the private port's namespace"); - let first = Holding { number: Holder::FIRST, connection: conn, frames: FrameRx::new(), receive, send }; + let first = Holding { number: Holder::FIRST, connection: conn, frames: FrameRx::new(), receive, send, rung }; (child, Netstack { region, server, holders: vec![first], exports: (exports, names) }) } diff --git a/tests/toyos.rs b/tests/toyos.rs index 8486ffa1c49..24c226028fc 100644 --- a/tests/toyos.rs +++ b/tests/toyos.rs @@ -265,6 +265,10 @@ const RUST_SKIP: &[&str] = &[ // tests/images/linkcase.toml leaves it to: // `a_link_ends_under_a_held_stream_and_comes_back` runs it. "link_restart", + // A connect killed while netstack has not answered it, through real + // netstack, which a shared boot runs none of: + // `a_client_killed_mid_connect_gives_its_place_back` runs it. + "netstack_connect_killed", // It needs a launcher: `a_launch_gives_each_unsent_stdio_slot_a_null` // runs it on tests/images/launchcase.toml. "launch_unsent_stdio", @@ -663,6 +667,15 @@ const MACHINE_TESTS: &[&str] = &[ // the host's loopback that QEMU's user network reaches, so no public // network. Order and completion only. "a_link_ends_under_a_held_stream_and_comes_back", + // A client killed while its connect is unanswered gives its place back to + // the real netstack of `tests/images/netcase.toml` at once, where only its + // connection can say it left: a process's death and netstack's watch of + // it, which no host test has. The peer is an address on QEMU's user + // network nothing answers, so no public network. Order and counts, QEMU's + // clock a hang ceiling only: the kill has to land within the about 3 s + // netstack takes to give the address up, or the test is red. x86-64 alone + // (`qemu::SUITE_ARCH`). + "a_client_killed_mid_connect_gives_its_place_back", // Login sessions: their owner parents, signs out and ends with each, and // the machine's reboot waits for them until forced, which only processes // in one boot show, as order, completion and counts. The reboot ends the @@ -4664,6 +4677,7 @@ fn run_machine_test(name: &str, test_config: &Path) -> Result<(), String> { "netstack_holders_exported_to_children" => netstack_holders_exported_to_children(), "udp_receive_times_out" => udp_receive_times_out(), "a_link_ends_under_a_held_stream_and_comes_back" => a_link_ends_under_a_held_stream_and_comes_back(), + "a_client_killed_mid_connect_gives_its_place_back" => a_client_killed_mid_connect_gives_its_place_back(), "session_lifecycle" => common::sessionlifecycle::session_lifecycle(), "a_launch_gives_each_unsent_stdio_slot_a_null" => common::launchstdio::a_launch_gives_each_unsent_stdio_slot_a_null(), "a_row_reaches_its_own_state_and_no_other" => common::stateview::a_row_reaches_its_own_state_and_no_other(), @@ -5822,6 +5836,28 @@ fn udp_receive_times_out() -> Result<(), String> { Ok(()) } +/// `test_rs_netstack_connect_killed` on `netcase`'s real netstack +/// (`tests/guest/rust-tests/src/bin/netstack_connect_killed.rs`). +fn a_client_killed_mid_connect_gives_its_place_back() -> Result<(), String> { + const JOB: &str = "netstack_connect_killed"; + let tests = compile::repo_root().join("tests/guest/rust-tests"); + let bin = qemu::build_toyos_bin(qemu::SUITE_ARCH, &tests, JOB); + let config = compile::repo_root().join("tests/images/netcase.toml"); + let mut qemu = QemuInstance::boot_with_options(&config, &[], &[(JOB.to_string(), bin)], BootOptions::default()); + serial::Serial::boot(&qemu).must_be_clean()?; + let result = qemu.run_test("test_rs_netstack_connect_killed", Duration::from_secs(120)); + if let Some(why) = &result.error { + return Err(format!("{why}\nthe job said:\n{}", result.stdout)); + } + if result.exit_code != Some(0) { + return Err(format!("the job ended {:?}:\n{}", result.exit_code, result.stdout)); + } + if !result.stdout.contains("a client killed mid-connect gave its place back") { + return Err(format!("the job never said its claim:\n{}", result.stdout)); + } + Ok(()) +} + /// `test_rs_stream_handles` on a boot of its own: a holder killed inside its /// gate leaves the other working, netstack's exit ends a blocked read in each, /// within a process a stream's handles are one end, one end reads and writes