From a893de7a3969c9e1571f5e644d02427ad51b8f47 Mon Sep 17 00:00:00 2001 From: japabu Date: Sat, 24 Oct 2026 09:51:30 +0200 Subject: [PATCH 1/7] console: the VM's console is a userland driver's, and the UART goes quiet while it carries the log The kernel drove QEMU's virtio console itself: a single-port `virtconsole` the kernel replayed its records into and logkeeper's program lines through, with its own DMA pool, a kernel-driver half of `DeviceSpace` and a rule for which `1af4:1043` function was its. A `virtconsole` drops what a full host pipe refuses, and the kernel driver waited on the device from its console path, the panic path included. Now `/system/bin/virtio-console` drives a named `virtserialport` (`org.toyos.console`), which QEMU throttles instead of dropping. It is logkeeper's console carrier: it asks on `log-carrier`, is fed the boot from its first line, and says how much the host has taken. Once the host has taken the first bytes, logkeeper tells the kernel (`SYS_CONSOLE_CARRIED` on its console) and stops writing program lines to it; klogd says so on the UART and advances past what it would have written. The UART comes back when the carrier goes, when the machine stops (after the carrier has the log up to the flush, bounded), and for good on a fatal path, before its first record. The kernel releases the carrier with the holder's console, so logkeeper's own crash cannot leave the UART quiet. vdagent and virtio-console are drivers of one device type, so both rows drive `pci:07:80:00/1af4:1043` and each probes the port names: a function none of whose ports is its own ends its process with exit 0, and the supervisor hands the function to the next row (`Drives::declined`). Each process holds only its row's authority; `log-carrier` is held by `virtio-console` alone. The control protocol, the port names and the probe are `toyos-virtio-serial` (pure, host-tested); the function's bring-up over its claim is `toyos-virtio-serial-claim`, shared by both drivers. Deleted with the kernel console: `drivers/virtio.rs`, `mm/dma.rs`, `DeviceSpace`, `toyos_pci::console`, the virtio backend of `serial.rs`, the backend-change replay, the untranslated-console profile and its test, and `build/kernelconsole.rs`'s firmware split (firmware writes nothing on the named port). `toyos-dma` keeps only the register bound and is `toyos-register`. The PCI capability walk's self-test moves to `drivers/pci.rs` on a plain frame, without its virtio cases (6/6). The harness reads the console port on QEMU's stdout and the UART on a socket it listens on, which test-runner's commands go to: UART lines up to the kernel's quiet line are the port's too and go to the boot's UART file only, but for a kernel's death; every one after is published, stdout drained before each UART read so the two keep their order. Co-Authored-By: Claude Opus 5.5 --- Cargo.lock | 44 +- Cargo.toml | 6 +- build/build.rs | 8 +- build/lib.rs | 2 +- build/qemu.rs | 23 +- build/sourcegate.rs | 8 +- build/vdagent.rs | 11 +- build/{kernelconsole.rs => vmconsole.rs} | 189 +--- images/system.toml | 21 +- ...ps-a-kernel-mapping-of-its-firmware-bar.md | 26 - ...-cpus-console-burst-waits-out-the-bound.md | 5 +- ...-smmuv3-does-not-route-panics-at-attach.md | 7 +- ...on-a-wedged-virtio-console-panics-again.md | 65 -- ...-cargo-run-boot-hands-over-no-pci-claim.md | 9 +- issues/every-driver-is-still-in-the-kernel.md | 3 +- issues/kernelslice-outlives-its-allocation.md | 9 +- ...nused-oracle-for-kernel-region-lifetime.md | 7 +- ...it-does-not-re-arm-across-a-guest-reset.md | 44 - ...ig-the-guest-cut-deleted-is-still-cited.md | 6 +- ...match-only-a-non-transitional-device-id.md | 13 +- kernel/Cargo.toml | 2 +- kernel/dma/Cargo.toml | 17 - kernel/dma/src/lib.rs | 371 -------- kernel/pci/src/choose.rs | 42 - kernel/pci/src/lib.rs | 2 - kernel/pure/carrier.rs | 116 +++ kernel/pure/lib.rs | 1 + kernel/register/Cargo.toml | 15 + kernel/register/src/lib.rs | 178 ++++ kernel/src/arch/aarch64/irqchip.rs | 10 +- kernel/src/arch/aarch64/smmu/domain.rs | 9 +- kernel/src/arch/aarch64/trap.rs | 4 + kernel/src/arch/x86_64/amdvi/domain.rs | 2 +- kernel/src/arch/x86_64/idt/exceptions.rs | 6 + kernel/src/arch/x86_64/idt/nmi.rs | 2 +- kernel/src/arch/x86_64/iommu_unit.rs | 18 +- kernel/src/drivers/mod.rs | 6 - kernel/src/drivers/pci.rs | 104 +++ kernel/src/drivers/serial.rs | 107 +-- kernel/src/drivers/virtio.rs | 850 ------------------ kernel/src/drivers/virtio_console.rs | 316 ------- kernel/src/iommu/mod.rs | 78 +- kernel/src/log/console.rs | 94 +- kernel/src/log/read.rs | 9 - kernel/src/main.rs | 8 +- kernel/src/mm/dma.rs | 225 ----- kernel/src/mm/mmio.rs | 5 - kernel/src/mm/mod.rs | 2 - kernel/src/mm/pmm.rs | 8 - kernel/src/object/device.rs | 20 +- kernel/src/panic.rs | 1 + kernel/src/pcidev/mod.rs | 6 +- kernel/src/syscall/dispatch.rs | 9 +- kernel/src/syscall/io.rs | 21 + lib/logkeeper-api/src/lib.rs | 29 +- lib/logstream/src/lib.rs | 16 + lib/manifest/src/lib.rs | 23 +- lib/vdagent/src/lib.rs | 25 +- lib/virtio-serial/Cargo.toml | 15 + lib/virtio-serial/claim/Cargo.toml | 18 + .../virtio-serial/claim/src/lib.rs | 249 +++-- lib/{vdagent => virtio-serial}/src/control.rs | 98 +- lib/virtio-serial/src/lib.rs | 40 + lib/virtio-serial/src/probe.rs | 250 ++++++ sdk/abi/src/syscall.rs | 19 +- sdk/toyos/src/lib.rs | 7 + system/logkeeper/src/carrier.rs | 449 +++++++++ system/logkeeper/src/main.rs | 54 +- system/logkeeper/src/serve.rs | 95 +- system/netstack/i219/Cargo.toml | 2 +- system/supervisor/lifecycle/src/drives.rs | 94 +- system/supervisor/src/main.rs | 10 +- system/vdagent/Cargo.toml | 5 +- system/vdagent/src/main.rs | 128 ++- system/virtio-console/Cargo.toml | 16 + system/virtio-console/src/main.rs | 200 +++++ tests/common/iommu.rs | 17 - tests/common/mod.rs | 2 + tests/common/qemu.rs | 310 +++++-- tests/common/serial.rs | 6 +- tests/common/vmconsole.rs | 84 ++ tests/images/acpicase.toml | 6 + tests/images/clipboardcase.toml | 10 +- tests/images/drivescase.toml | 6 + tests/images/hostedclangcase.toml | 6 + tests/images/jobcase.toml | 6 + tests/images/launchcase.toml | 6 + tests/images/libccase.toml | 6 + tests/images/netcase.toml | 6 + tests/images/panelcase.toml | 6 + tests/images/proctreecase.toml | 6 + tests/images/scalecase.toml | 6 + tests/images/sessioncase.toml | 6 + tests/images/testcases.toml | 7 +- tests/toyos.rs | 27 +- 95 files changed, 2772 insertions(+), 2769 deletions(-) rename build/{kernelconsole.rs => vmconsole.rs} (59%) delete mode 100644 issues/a-declined-console-function-keeps-a-kernel-mapping-of-its-firmware-bar.md delete mode 100644 issues/a-panic-on-a-wedged-virtio-console-panics-again.md delete mode 100644 issues/the-kernel-console-split-does-not-re-arm-across-a-guest-reset.md delete mode 100644 kernel/dma/Cargo.toml delete mode 100644 kernel/dma/src/lib.rs delete mode 100644 kernel/pci/src/choose.rs create mode 100644 kernel/pure/carrier.rs create mode 100644 kernel/register/Cargo.toml create mode 100644 kernel/register/src/lib.rs delete mode 100644 kernel/src/drivers/virtio.rs delete mode 100644 kernel/src/drivers/virtio_console.rs delete mode 100644 kernel/src/mm/dma.rs create mode 100644 lib/virtio-serial/Cargo.toml create mode 100644 lib/virtio-serial/claim/Cargo.toml rename system/vdagent/src/channel.rs => lib/virtio-serial/claim/src/lib.rs (51%) rename lib/{vdagent => virtio-serial}/src/control.rs (53%) create mode 100644 lib/virtio-serial/src/lib.rs create mode 100644 lib/virtio-serial/src/probe.rs create mode 100644 system/logkeeper/src/carrier.rs create mode 100644 system/virtio-console/Cargo.toml create mode 100644 system/virtio-console/src/main.rs create mode 100644 tests/common/vmconsole.rs diff --git a/Cargo.lock b/Cargo.lock index ef2c75d4c16..52c6ee43330 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1517,7 +1517,6 @@ dependencies = [ "toyos-blackbox", "toyos-bootmap", "toyos-devicepath", - "toyos-dma", "toyos-elf", "toyos-gicv3", "toyos-its", @@ -1526,6 +1525,7 @@ dependencies = [ "toyos-phys", "toyos-quiesce", "toyos-random", + "toyos-register", "toyos-rootimage", "toyos-smmu", "toyos-symbols", @@ -3572,6 +3572,7 @@ dependencies = [ "toyos-update", "toyos-userbound", "toyos-vdagent", + "toyos-virtio-serial", "toyos-wallclock", "ureq", "webpki-root-certs", @@ -3629,10 +3630,6 @@ dependencies = [ "toyos-net-wire", ] -[[package]] -name = "toyos-dma" -version = "0.1.0" - [[package]] name = "toyos-dns" version = "0.1.0" @@ -3904,6 +3901,10 @@ version = "0.1.0" name = "toyos-random" version = "0.1.0" +[[package]] +name = "toyos-register" +version = "0.1.0" + [[package]] name = "toyos-resample" version = "0.1.0" @@ -4063,6 +4064,22 @@ dependencies = [ "toyos-virtio", ] +[[package]] +name = "toyos-virtio-serial" +version = "0.1.0" + +[[package]] +name = "toyos-virtio-serial-claim" +version = "0.1.0" +dependencies = [ + "toyos", + "toyos-abi", + "toyos-device-memory", + "toyos-pci-claim", + "toyos-virtio", + "toyos-virtio-serial", +] + [[package]] name = "toyos-virtio-sound" version = "0.1.0" @@ -4298,12 +4315,11 @@ version = "0.0.0" dependencies = [ "toyos", "toyos-abi", - "toyos-device-memory", "toyos-manifest", - "toyos-pci-claim", "toyos-vdagent", "toyos-vdagent-bridge", - "toyos-virtio", + "toyos-virtio-serial", + "toyos-virtio-serial-claim", "toyos-window", ] @@ -4313,6 +4329,18 @@ version = "0.9.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a" +[[package]] +name = "virtio-console" +version = "0.0.0" +dependencies = [ + "logkeeper-api", + "toyos", + "toyos-abi", + "toyos-manifest", + "toyos-virtio-serial", + "toyos-virtio-serial-claim", +] + [[package]] name = "walkdir" version = "2.5.0" diff --git a/Cargo.toml b/Cargo.toml index 9627dc1ca01..cca80f550a5 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -25,12 +25,12 @@ members = [ "build/root", "kernel", "kernel/amdvi", - "kernel/dma", "kernel/gicv3", "kernel/its", "kernel/loom", "kernel/pci", "kernel/phys", + "kernel/register", "kernel/sim", "kernel/smmu", "lib/acpi", @@ -77,6 +77,8 @@ members = [ "lib/userbound", "lib/vdagent", "lib/virtio", + "lib/virtio-serial", + "lib/virtio-serial/claim", "lib/wallclock", "loader", "sdk/abi", @@ -141,6 +143,7 @@ members = [ "system/usbd", "system/usbd/xhci", "system/vdagent", + "system/virtio-console", "system/vdagent/bridge", "tests/guest/hangupholder", "tests/guest/kernelprobe", @@ -254,6 +257,7 @@ toyos-session = { path = "lib/session" } # QEMU's clipboard channel's port name, id and port count, so the launch reads # the declaration the guest's agent drives. toyos-vdagent = { path = "lib/vdagent" } +toyos-virtio-serial = { path = "lib/virtio-serial" } # The watchdog's parameter name, so the gate over `kernel/src/params.rs` reads # the same constant the kernel and the bootloader do. toyos-tco = { path = "lib/tco" } diff --git a/build/build.rs b/build/build.rs index 60d4775a1a2..bc29ba41188 100644 --- a/build/build.rs +++ b/build/build.rs @@ -653,7 +653,8 @@ const SUPERVISOR_SERVED: &[&str] = &[toyos_swap::PORT, "power", toyos_manifest:: /// clipboard port, whose holder is sent every copy made in the session, the /// display port, whose holder sets the mode and the scale every window is /// drawn at, and the session-admin port, whose holder signs its user's -/// sessions out. +/// sessions out, and the log's carrier port, for whose holder the console +/// UART goes quiet. const HELD_ALONE: &[(&str, &[&str])] = &[ (toyos_swap::PORT, &[toyos_swap::HOLDER]), (toyos_manifest::grants::PORT, &[toyos_manifest::grants::HOLDER]), @@ -662,6 +663,7 @@ const HELD_ALONE: &[(&str, &[&str])] = &[ (toyos_manifest::clipboard::PORT, &[toyos_manifest::clipboard::HOLDER]), (toyos_manifest::display::PORT, &[toyos_manifest::display::HOLDER]), (toyos_session::ADMIN_PORT, &[toyos_session::ADMIN_HOLDER]), + (toyos_logstream::CARRIER, &[toyos_logstream::CARRIER_HOLDER]), ]; /// Rows, as a refusal names them. @@ -3290,6 +3292,10 @@ mod tests { assert!(held_by_their_holders_alone(&vdagent).is_ok()); let editor: SystemConfig = toml::from_str("[programs.editor]\nreceives = [\"clipboard\"]\n").unwrap(); assert!(held_by_their_holders_alone(&editor).is_err()); + let console: SystemConfig = toml::from_str("[programs.virtio-console]\nreceives = [\"log-carrier\"]\n").unwrap(); + assert!(held_by_their_holders_alone(&console).is_ok()); + let agent: SystemConfig = toml::from_str("[programs.vdagent]\nreceives = [\"clipboard\", \"log-carrier\"]\n").unwrap(); + assert!(held_by_their_holders_alone(&agent).unwrap_err().contains("only `virtio-console`")); } /// The prompt port reaches its server alone, and the consent port no row, diff --git a/build/lib.rs b/build/lib.rs index e3dc4360e14..b771804afad 100644 --- a/build/lib.rs +++ b/build/lib.rs @@ -25,7 +25,6 @@ pub mod flags; pub mod hostws; pub mod identity; pub mod image; -pub mod kernelconsole; pub mod signing; /// Which kernel containers may be hashed, and by whose keys; read by nothing /// but its own tests. @@ -52,6 +51,7 @@ pub mod tether; pub mod toolchain; pub mod userlandhost; pub mod vdagent; +pub mod vmconsole; pub mod wallpaper; use std::path::{Path, PathBuf}; diff --git a/build/qemu.rs b/build/qemu.rs index a6f2f65b648..b0ec46d615e 100644 --- a/build/qemu.rs +++ b/build/qemu.rs @@ -52,7 +52,7 @@ use std::path::PathBuf; use std::process::{Command, Stdio}; use toyos_build::arch::{Accel, Arch}; -use toyos_build::kernelconsole; +use toyos_build::vmconsole; /// The hardware shape QEMU presents to the guest. /// @@ -61,7 +61,7 @@ use toyos_build::kernelconsole; /// virtio device, which changes the console, the network, and audio too. #[derive(Clone, Copy, PartialEq)] pub enum Profile { - /// virtio-gpu for the display, virtio-console for the console, plus + /// virtio-gpu for the display, a virtio-serial port for the console, plus /// virtio-net and virtio-sound. Virtio, /// `-vga std`, so firmware publishes a GOP and the kernel takes the @@ -256,24 +256,17 @@ pub fn launch(opts: &Options) { qemu.arg("-device") .arg(format!("virtio-sound-pci,audiodev=audio0,streams=1{platform}")); - // The clipboard's channel, just before the console's function, so it - // enumerates first and the kernel's console rule runs. qemu.args(toyos_build::vdagent::args(platform)); - // Console wiring: virtio-console on stdio is the primary I/O channel - // (the kernel switches to it once virtio-console init completes — - // see drivers/virtio_console.rs). UART stays on a file so early-boot - // logs (before virtio is up) and panic fallback are still captured. + // The console's port on stdio, where `/system/bin/virtio-console` + // carries the log once userland runs; the UART on a file, which keeps + // the loader, the early boot, the stop and a fatal path's report, and + // the whole log again if the console's carrier goes. qemu.arg("-serial") .arg("file:/tmp/toyos-uart-early.log") .arg("-chardev") .arg("stdio,id=cs0,signal=off") - .arg("-device") - .arg(format!( - "virtio-serial-pci-non-transitional,id=virtio-serial0,max_ports=1{platform}" - )) - .arg("-device") - .arg("virtconsole,chardev=cs0,id=console0"); + .args(toyos_build::vmconsole::args("cs0", platform)); } else if opts.mute { eprintln!("metal-sim: no 16550 — the framebuffer is the only channel out"); qemu.arg("-serial").arg("none"); @@ -320,7 +313,7 @@ pub fn launch(opts: &Options) { let console = child.stdout.take().expect("QEMU's stdout is piped"); // The console ends when QEMU does; a relay that fails first ends QEMU, which // would otherwise run on into a pipe nobody reads. - if let Err(relay) = kernelconsole::relay(console, &mut terminal) { + if let Err(relay) = vmconsole::relay(console, &mut terminal) { // SIGTERM, not `Child::kill`'s SIGKILL: QEMU gives the terminal back its // modes only on its own exit path. let pid = libc::pid_t::try_from(child.id()).expect("a pid is a pid_t"); diff --git a/build/sourcegate.rs b/build/sourcegate.rs index 3e4b8419252..0e5fe3383a0 100644 --- a/build/sourcegate.rs +++ b/build/sourcegate.rs @@ -88,10 +88,8 @@ const LOG_PRODUCERS: &[&str] = &["log!(", "alert!(", "boot_phase!(", "log::emit( const NMI_SILENT: &[&str] = &["kernel/src/arch/x86_64/idt/nmi.rs", "kernel/src/hardlockup/mod.rs"]; -/// Every `enable_bus_master(` site `kernel/src` holds, by file and count. -/// No kernel site leaves a device it declined mastering the bus: virtio -/// parses its capability chain first, and switches mastering off on a refusal -/// that comes after arming. +/// Every `enable_bus_master(` site `kernel/src` holds, by file and count: +/// its definition, and the AMD-Vi unit's own function. /// /// A function handed to a *process* takes the other path: `pcidev` arms bus /// mastering with `start_bus_mastering`, after the address space that bounds it @@ -99,7 +97,6 @@ const NMI_SILENT: &[&str] = /// count below stays the census of drivers this kernel has. const BUS_MASTER_SITES: &[(&str, usize)] = &[ ("kernel/src/drivers/pci.rs", 1), - ("kernel/src/drivers/virtio.rs", 1), ("kernel/src/arch/x86_64/amdvi/mod.rs", 1), ]; @@ -137,7 +134,6 @@ const SENTINEL_ALLOWED: &[(&str, usize)] = &[ /// reds beside a permitted one and a deleted one reds its own stale row. const AUTO_TRAIT_IMPLS: &[(&str, usize)] = &[ ("kernel/src/drivers/panic_console/mod.rs", 3), - ("kernel/src/drivers/virtio_console.rs", 1), ("kernel/src/arch/x86_64/hw.rs", 1), ("kernel/src/mm/mmio.rs", 2), ("kernel/src/mm/region.rs", 2), diff --git a/build/vdagent.rs b/build/vdagent.rs index 5b0e2a09f0a..3b91a3f9528 100644 --- a/build/vdagent.rs +++ b/build/vdagent.rs @@ -1,11 +1,7 @@ //! QEMU's clipboard channel at the launch: its `qemu-vdagent` chardev, the //! virtio-serial function of its own it is on, and the agent's port, from the -//! one declaration the guest's agent reads (`toyos-vdagent`). -//! -//! **A function of its own**, created just before the console's: the console's -//! function is the kernel's log channel from early boot, and the agent's -//! enumerates first, so every virtio boot runs the kernel's choice of console -//! (`toyos_pci::console`). +//! declarations the guest's agent reads (`toyos-vdagent`, +//! `toyos-virtio-serial`). //! //! **The chardev exists only in a QEMU built with spice-protocol** (QEMU's //! `qapi/char.json`, `'if': 'CONFIG_SPICE_PROTOCOL'`), so a launch asks QEMU @@ -14,7 +10,8 @@ use std::process::Command; use std::sync::OnceLock; -use toyos_vdagent::{MAX_PORTS, PORT_ID, PORT_NAME}; +use toyos_vdagent::PORT_NAME; +use toyos_virtio_serial::{MAX_PORTS, PORT_ID}; use crate::arch::Arch; diff --git a/build/kernelconsole.rs b/build/vmconsole.rs similarity index 59% rename from build/kernelconsole.rs rename to build/vmconsole.rs index 12e33763bfa..a035a692ded 100644 --- a/build/kernelconsole.rs +++ b/build/vmconsole.rs @@ -1,69 +1,40 @@ -//! The kernel's console as the host reads it off the virtio port: every byte -//! from the kernel's first record on, and none before it. +//! The VM's console: at the launch, a virtio-serial function of its own whose +//! one port is named [`CONSOLE_PORT`] and carries what a chardev is given +//! ([`args`]); and on the host's terminal, every line in colour ([`relay`]). +//! The bytes the host reads are never coloured. //! -//! A firmware whose UEFI console drives that port writes its own lines and the -//! loader's there first, the last of them cut off where boot services end and -//! the kernel's first record written onto its tail. Every one of them is on the -//! 16550 as well, and a loader line is read there. Where the kernel begins is -//! the whole rule (`toyos_logstream::kernel_opening`), so a firmware that -//! writes nothing on the port reads the same. A terminal is shown that whole, -//! and what is the kernel's in colour ([`Painter`], [`relay`]); the bytes the -//! host reads are never coloured. +//! **A `virtserialport`, never a `virtconsole`**: QEMU 11.1.1 holds a +//! `virtserialport`'s transmit until its chardev takes it, and drops what a +//! `virtconsole`'s chardev refuses (`hw/char/virtio-console.c`, `flush_buf`). +//! Port 0 is a `virtconsole`'s alone, so the port is [`PORT_ID`] of a +//! multiport function. Firmware writes nothing on it, so its first byte is +//! the boot log's. -use std::borrow::Cow; use std::io::{self, Read, Write}; use std::os::fd::{AsFd, AsRawFd, BorrowedFd}; -use toyos_logstream::{kernel_opening, Opening, Showing}; +use toyos_logstream::Showing; +use toyos_virtio_serial::{CONSOLE_PORT, MAX_PORTS, PORT_ID}; -/// A console's bytes as they arrive, withheld until the kernel's first record. -/// -/// Of a QEMU process's first boot only: a guest reset does not re-arm it -/// (`issues/the-kernel-console-split-does-not-re-arm-across-a-guest-reset.md`). -pub struct KernelConsole { - /// The withheld tail that could still open the kernel's first record; - /// `None` once the kernel has begun. - held: Option>, -} - -impl Default for KernelConsole { - fn default() -> Self { - Self { held: Some(Vec::new()) } - } -} +/// The port's QEMU device id, which a test unplugs it by. +pub const PORT_DEVICE: &str = "console-port"; -impl KernelConsole { - /// What of the next `chunk` is the kernel's. - pub fn pass<'a>(&mut self, chunk: &'a [u8]) -> Cow<'a, [u8]> { - let Some(held) = &mut self.held else { return Cow::Borrowed(chunk) }; - held.extend_from_slice(chunk); - match kernel_opening(held) { - Opening::At(at) => { - let from = held.split_off(at); - self.held = None; - Cow::Owned(from) - } - Opening::From(at) => { - held.drain(..at); - Cow::Borrowed(&[]) - } - Opening::Nowhere => { - held.clear(); - Cow::Borrowed(&[]) - } - } - } +/// The arguments that put the console's port on `chardev`. `platform` is the +/// virtio function's `iommu_platform` property, as every other virtio function +/// on the machine is given it. +pub fn args(chardev: &str, platform: &str) -> [String; 4] { + [ + "-device".into(), + format!("virtio-serial-pci-non-transitional,id=console-serial,max_ports={MAX_PORTS}{platform}"), + "-device".into(), + format!("virtserialport,bus=console-serial.0,nr={PORT_ID},chardev={chardev},name={CONSOLE_PORT},id={PORT_DEVICE}"), + ] } -/// The console as a terminal shows it: what comes before the kernel's first -/// record as it came, and every line from it on as [`Showing`] shows it. -/// -/// From the kernel's first record on the console carries whole lines only, so -/// a line is held until it ends; before it, only what could still open that -/// record is. +/// The console as a terminal shows it: every line as [`Showing`] shows it, +/// each held until it ends. #[derive(Default)] pub struct Painter { - begun: bool, held: Vec, showing: Showing, } @@ -73,22 +44,6 @@ impl Painter { pub fn pass(&mut self, chunk: &[u8]) -> Vec { let mut out = Vec::new(); self.held.extend_from_slice(chunk); - if !self.begun { - match kernel_opening(&self.held) { - Opening::At(at) => { - out.extend(self.held.drain(..at)); - self.begun = true; - } - Opening::From(at) => { - out.extend(self.held.drain(..at)); - return out; - } - Opening::Nowhere => { - out.append(&mut self.held); - return out; - } - } - } while let Some(end) = self.held.iter().position(|&b| b == b'\n') { let line: Vec = self.held.drain(..=end).collect(); self.show(&line[..end], &mut out); @@ -97,16 +52,14 @@ impl Painter { out } - /// What the terminal is shown of what is still held once the console has - /// ended: a line a machine that stopped mid-line never finished, or the - /// bytes that could have opened the kernel's first record. + /// What the terminal is shown of a line a machine that stopped mid-line + /// never finished, once the console has ended. pub fn finish(mut self) -> Vec { let held = std::mem::take(&mut self.held); - if !self.begun || held.is_empty() { - return held; - } let mut out = Vec::new(); - self.show(&held, &mut out); + if !held.is_empty() { + self.show(&held, &mut out); + } out } @@ -176,53 +129,10 @@ mod tests { use super::*; use std::os::fd::FromRawFd; - /// QEMU 11.1.1's own edk2 on the virtio port, as a boot put it there: the - /// screen clears, `BdsDxe` and the loader, and the loader's last line cut - /// off by the handoff. - const FIRMWARE: &str = "\x1b[2J\x1b[01;01H\x1b[=3h\x1b[2J\x1b[01;01HBdsDxe: loading Boot0001 \ - \"UEFI QEMU QEMU USB HARDDRIVE TOYOS0BOOTSTICK1\" from PciRoot(0x0)/Pci(0x1,0x0)/USB(0x0,0x0)\n\ - [--.--- cpu0 loader] ToyOS Bootloader 1.0\n\ - [--.--- cpu0 loader] ROOT: read into memory at 0x7c894000+0x800000 from LBA 212992+16384, 1048576 bytes a request \ - (optimal granularity: not reported), in 22519000 counter ticks\n\ - [--.--- cpu0 loader] Loader log: the kernel handoff begins, so "; - const KERNEL: &str = "[--.--- cpu0 kernel] black box: 0x8000000 is this boot's, 16344 bytes \ for the next boot's loader\n\ [--.--- cpu0 kernel] pmm: the firmware map calls 4288393216 bytes usable\n"; - /// Everything `stream` passes, fed in pieces cut at each of `cuts`. - fn passed(stream: &str, cuts: &[usize]) -> String { - let mut console = KernelConsole::default(); - let mut out = Vec::new(); - let mut from = 0; - for &to in cuts.iter().chain([stream.len()].iter()) { - out.extend_from_slice(&console.pass(&stream.as_bytes()[from..to])); - from = to; - } - String::from_utf8(out).expect("the kernel's bytes are UTF-8") - } - - #[test] - fn the_fused_line_is_the_kernels_first_record_wherever_the_chunks_fall() { - let stream = format!("{FIRMWARE}{KERNEL}"); - for cut in 0..=stream.len() { - assert_eq!(passed(&stream, &[cut]), KERNEL, "cut at byte {cut}"); - } - let every_byte: Vec = (1..stream.len()).collect(); - let kernel = passed(&stream, &every_byte); - let first = kernel.lines().next().expect("a first line"); - assert_eq!( - crate::bootlog::message(first), - Some("black box: 0x8000000 is this boot's, 16344 bytes for the next boot's loader") - ); - } - - #[test] - fn a_port_the_firmware_left_alone_passes_whole() { - let later = format!("{KERNEL}[ 1.002 supervisor] supervisor: a program's line\n"); - assert_eq!(passed(&later, &[3, 40]), later); - } - /// What the terminal is shown of `stream` fed in pieces cut at each of `cuts`. fn painted(stream: &str, cuts: &[usize]) -> String { let mut painter = Painter::default(); @@ -235,31 +145,26 @@ mod tests { String::from_utf8(out).expect("the painter writes UTF-8") } - /// The firmware's bytes reach the terminal as they came, the kernel's - /// lines as a screen shows them, a continuation in its record's severity — - /// and the same however the chunks fall. + /// Every line as a screen shows it, a continuation in its record's + /// severity, and the same however the chunks fall. #[test] - fn a_terminal_is_shown_the_firmware_as_it_came_and_the_kernel_in_colour() { + fn a_terminal_is_shown_each_line_in_colour() { let alert = "[ 0.002 cpu1 kernel alert tid=4] PANIC: panicked at kernel/src/main.rs:1:\n oops\n"; let program = "[ 0.003 supervisor warn] supervisor: a program's line\n"; - let stream = format!("{FIRMWARE}{KERNEL}{alert}{program}"); - let mut want = String::from(FIRMWARE); + let stream = format!("{KERNEL}{alert}{program}"); + let mut want = String::new(); let mut showing = Showing::default(); - for line in format!("{KERNEL}{alert}{program}").lines() { + for line in stream.lines() { want.push_str(&format!("{}\n", showing.line(line))); } assert!(want.contains("\x1b[91m oops\x1b[0m\n"), "{want:?}"); assert_eq!(painted(&stream, &[]), want); let every_byte: Vec = (1..stream.len()).collect(); assert_eq!(painted(&stream, &every_byte), want); - // What is held before the kernel is only what could still open its first record. - assert_eq!(painted(&FIRMWARE[..FIRMWARE.len() - 3], &[]), FIRMWARE[..FIRMWARE.len() - 3]); - assert_eq!(painted("so [--.--- cpu0 ker", &[]), "so "); } /// A console that ends mid-line — a machine that stopped while it spoke — - /// still shows its last line, and one that ends on what could have begun - /// the kernel's head still shows those bytes. + /// still shows its last line. #[test] fn a_console_that_ends_mid_line_shows_its_last_line() { let cut = "[ 0.004 cpu0 kernel alert tid=1] PANIC: triple fa"; @@ -271,13 +176,6 @@ mod tests { let want = format!("{want}{}", showing.line(cut)); assert!(want.ends_with("\x1b[91mPANIC: triple fa\x1b[0m"), "{want:?}"); assert_eq!(String::from_utf8(out).expect("UTF-8"), want); - - for early in ["so [--.--- cpu0 ker", "loading\r"] { - let mut painter = Painter::default(); - let mut out = painter.pass(early.as_bytes()); - out.extend(painter.finish()); - assert_eq!(out, early.as_bytes()); - } } /// A descriptor that refuses a write as `WouldBlock` once it is full, and says so @@ -310,7 +208,7 @@ mod tests { /// A burst of kernel lines far past a terminal's capacity, and what a /// [`Painter`] shows of it. fn burst() -> (Vec, Vec) { - let mut stream = String::from(FIRMWARE); + let mut stream = String::new(); for n in 0..20_000 { stream.push_str(&format!("[ 1.{:03} cpu{} kernel alert tid=3] frame {n}: kernel::panic\n", n % 1000, n % 8)); stream.push_str(" continued\n"); @@ -381,13 +279,4 @@ mod tests { .expect("the relay's output"); assert!(shown == want, "the terminal was shown other bytes than the {length} painted"); } - - #[test] - fn nothing_before_the_kernel_passes() { - assert_eq!(passed(FIRMWARE, &[5, 200]), ""); - let mut console = KernelConsole::default(); - assert!(console.pass(FIRMWARE.as_bytes()).is_empty()); - let held = console.held.as_ref().expect("the kernel has not begun").len(); - assert!(held < toyos_logstream::MAX_HEAD, "{held} bytes withheld, more than could open a record"); - } } diff --git a/images/system.toml b/images/system.toml index f2f8018bda5..b8cf58a6733 100644 --- a/images/system.toml +++ b/images/system.toml @@ -51,11 +51,12 @@ pkg = { state = true } # image does. The kernel keeps the record ring and the console and writes no # file at all, so a boot config without `logkeeper` is a boot whose `/log` is empty — # `every_boot_config_runs_logkeeper` is what refuses one. It claims no device, and -# `log` is the port it answers `inspect` on. +# `log` is the port it answers `inspect` on. `log-carrier` is where the VM's +# console asks to carry the log, which quiets the UART while it does. [programs.logkeeper] service = true syscap = ["logread"] -serves = ["log"] +serves = ["log", "log-carrier"] # The machine's ACPI fixed hardware: the kernel puts the machine in ACPI mode # when it mints the `acpi` claim and back when the claim goes, so the SCI, @@ -128,15 +129,21 @@ service = true serves = ["netstack"] devices = ["pci:02:00:00/1af4:1041"] -# The shared clipboard with QEMU's host: one process per virtio-serial -# function the kernel does not drive (the kernel's console is `1af4:1043` too), -# holding the function its vdagent chardev is on, and the compositor's -# clipboard port. A machine with no such function, the T14 among them, starts -# none. +# The shared clipboard with QEMU's host, and the VM's console: two drivers of +# one device, `1af4:1043`, each started on every virtio-serial function and +# keeping the one whose port carries its name; on another, it ends with exit 0 +# and the function goes to the next row. Each holds only its own authority: +# vdagent the compositor's clipboard port, virtio-console the log's carrier +# port, which no other row may receive. A machine with no such function, the +# T14 among them, starts neither. [programs.vdagent] drives = ["pci:07:80:00/1af4:1043"] receives = ["clipboard"] +[programs.virtio-console] +drives = ["pci:07:80:00/1af4:1043"] +receives = ["log-carrier"] + # A login authority: each authenticated connection opens a login session, and # its shell, and a command as `shell -c`, is launched in it, the one `update` # and `swap` below start in. diff --git a/issues/a-declined-console-function-keeps-a-kernel-mapping-of-its-firmware-bar.md b/issues/a-declined-console-function-keeps-a-kernel-mapping-of-its-firmware-bar.md deleted file mode 100644 index bd6fc5e3ea8..00000000000 --- a/issues/a-declined-console-function-keeps-a-kernel-mapping-of-its-firmware-bar.md +++ /dev/null @@ -1,26 +0,0 @@ ---- -status: assigned -kind: defect -opened: 2026-10-13 ---- - -# A declined console function keeps a kernel mapping of its firmware BAR - -Held by the orchestrator. - -The kernel's console reads every virtio console function's device features -before it picks one (`kernel/src/drivers/virtio_console.rs`, -`VirtioDevice::offer` in `kernel/src/drivers/virtio.rs`). That read maps the -function's capability windows through `map_mmio`, at the BAR addresses -firmware assigned. A function the console declines is reset and left to a -claim, and its windows stay mapped in the kernel's direct map and listed in -x86_64's driven windows (`kernel/src/arch/x86_64/paging.rs`), which nothing -removes. `pcidev` then moves that BAR when a process claims the function, so -the kernel keeps a mapping of an address range the function no longer -decodes. Nothing reads through it today. - -Virtio 1.2 §4.1.4.9's PCI configuration access capability reaches a BAR's -registers through configuration space with nothing mapped. - -**Exit condition.** A function the console declines leaves no kernel mapping -and no driven window behind. diff --git a/issues/a-fatal-path-inside-its-own-cpus-console-burst-waits-out-the-bound.md b/issues/a-fatal-path-inside-its-own-cpus-console-burst-waits-out-the-bound.md index 6b7650b3b9a..115939836e4 100644 --- a/issues/a-fatal-path-inside-its-own-cpus-console-burst-waits-out-the-bound.md +++ b/issues/a-fatal-path-inside-its-own-cpus-console-burst-waits-out-the-bound.md @@ -9,9 +9,8 @@ opened: 2026-10-01 A fatal path takes the console registers by `BackendLock::try_seize` (`kernel/src/drivers/serial_lock.rs`), which finds a hold its own CPU's fatal path left, and `serial::panic_registers` gives up on any other after -`PANIC_LOCK_WAIT`. A burst holder — `uart_write_fifo`, virtio-console's -publish and its looks, a byte read — holds them as `LIVE`, which names no -CPU. A fault, a panic or a nested NMI that lands inside one waits the whole +`PANIC_LOCK_WAIT`. A burst holder — `write_wire`'s burst, a byte read — +holds them as `LIVE`, which names no CPU. A fault, a panic or a nested NMI that lands inside one waits the whole bound for a holder that never runs again, says so, and writes over it; every later `panic_registers` on that path waits it again. diff --git a/issues/a-function-the-smmuv3-does-not-route-panics-at-attach.md b/issues/a-function-the-smmuv3-does-not-route-panics-at-attach.md index 85a7b032b58..9ecb0412193 100644 --- a/issues/a-function-the-smmuv3-does-not-route-panics-at-attach.md +++ b/issues/a-function-the-smmuv3-does-not-route-panics-at-attach.md @@ -6,10 +6,9 @@ opened: 2026-10-10 # A function the SMMUv3 does not route panics at attach -`OwnSpace::create` takes no function, and `DeviceSpace::create` asks of its -functions only `domain::untranslated`, which on the SMMUv3 -(`kernel/src/arch/aarch64/smmu/domain.rs`) answers `None` for every one, so a -domain is made for any caller. `attach` then looks the function up in the unit's +`OwnSpace::create` takes no function, so a domain is made for any caller, +and the SMMUv3's `domain::unclaimable` +(`kernel/src/arch/aarch64/smmu/domain.rs`) answers `None` for every function. `attach` then looks the function up in the unit's routes, and `Live::stream` panics where it finds none: an enumerated function whose IORT route names another unit, names none, or was refused (`kernel/src/arch/aarch64/smmu/mod.rs`, `init`'s route loop). A gap in a diff --git a/issues/a-panic-on-a-wedged-virtio-console-panics-again.md b/issues/a-panic-on-a-wedged-virtio-console-panics-again.md deleted file mode 100644 index abfab42dd55..00000000000 --- a/issues/a-panic-on-a-wedged-virtio-console-panics-again.md +++ /dev/null @@ -1,65 +0,0 @@ ---- -status: open -kind: defect -opened: 2026-07-30 ---- - -# A panic while the virtio-console TX queue is wedged *and* unlocked panics again, and the report never reaches the 16550 - -**The wait is no longer unbounded, and that is the whole of what changed.** -`Virtqueue::submit_and_wait` (`kernel/src/drivers/virtio.rs`) waits through -`wait_used`, bounded by `ANSWERS`, a `Tripwire::absurd(5 s, "far above any -completion a live device delivers")`, and on expiry it panics by name. A -`Tripwire` is the kind whose expiry *is* a panic (`kernel/src/time.rs`), so on -the panic path the bound converts a hang into a second panic — which is not the -answer, only a different wrong one. The heading this file carried until now said -"spins"; the tree refutes it. - -The unlocked case is the one that reaches it. `panic_flush` -(`kernel/src/drivers/serial.rs`) takes the registers through -`panic_registers`, and the virtio-console `disable()` that would fall the -report back to the 16550 is on the *other* branch, the one taken only when a -holder never releases. With the queue wedged but the guard free, the hold is taken -(`Hold::Held`) and `drain_locked` runs; that drain is deliberately -unbounded in records ("the report should be whole", -`kernel/src/log/console.rs`), and each record goes out through -`BackendGuard::write_raw`'s `Backend::Virtio` arm into -`virtio_console::write_bytes_locked` (`kernel/src/drivers/virtio_console.rs`) -and `submit_and_wait`. Five seconds later `wait_used` panics inside the -panic handler, which re-enters `kernel/src/main.rs`'s panic handler, takes the -reentry guard, and halts on `PANIC REENTRY: CPU halted`. - -So the machine still halts — the terminal action was never in question — but it -halts five seconds late, under a message that names the console rather than the -fault, with the real report truncated wherever the wedge caught the drain and -never delivered to the two channels that cannot wedge. - -## The ruling, 2026-09-03: a panic never waits on a device - -The panic path's wait is a `Budget`, not a `Tripwire` — the kind whose expiry is -a degraded answer and never a panic (`kernel/src/time.rs`); both kinds are -already in that file and nothing new is needed there. On expiry the console -output is **dropped**: `virtio_console::disable()`, then the panic proceeds to -the channels that cannot wedge — the 16550 (`serial::panic_raw`) and the panel -(`panic_console::render`) — which is exactly what `panic_flush`'s wedged-holder -branch already does for the other case. Terminal action -is unchanged: the machine still halts on the panic; it just no longer depends on -virtio answering to get there. The 5 s `Tripwire` stays what it is for every -non-panic caller of `submit_and_wait`; the panic path is the caller that may not -pay it. - -**Not built.** This file records what a fix owes. - -## The two checks a fix owes - -- **Negative control:** an actuator that wedges the TX queue — a device that - takes the descriptor and never publishes to the used ring — with the backend - guard left free, so the panic path enters `drain_locked` with a live - `Backend::Virtio`. On the base that arm must show the `PANIC REENTRY: CPU - halted` line and the truncated report; with the fix it must show the whole - report and no reentry. Both arms run, both pasted. -- **Oracle:** the 16550 capture, which is not the channel under test. The - wedged-queue arm's report arrives on a path that shares no code with virtio's - queue, so a report that is whole there is evidence the drop happened and the - panic proceeded, independent of anything the virtio driver reports about - itself. diff --git a/issues/an-aarch64-cargo-run-boot-hands-over-no-pci-claim.md b/issues/an-aarch64-cargo-run-boot-hands-over-no-pci-claim.md index d7dbd39b994..5e66f0dcc74 100644 --- a/issues/an-aarch64-cargo-run-boot-hands-over-no-pci-claim.md +++ b/issues/an-aarch64-cargo-run-boot-hands-over-no-pci-claim.md @@ -25,10 +25,11 @@ in this kernel is given no LPI`) and initialises no controller, so no USB keyboard reaches the compositor, which panics on "the manifest gives this program the keyboard". -So on aarch64 the clipboard agent's claim and its handshake are unbooted: -its function is declined by the console driver and the console comes up -on the next one, and the agent itself exits on "no virtio-serial function -was handed to this program". Its handshake on aarch64 rests on the code +So on aarch64 the virtio-serial drivers' claims and handshakes are unbooted: +neither the clipboard agent nor the console's driver is handed its +function, so the console port `cargo run`'s terminal shows carries nothing, +and the boot's whole log is on the UART's file +(`/tmp/toyos-uart-early.log`). Their handshakes on aarch64 rest on the code that passes on q35. The windows a claim's BAR is checked against come from the loader's list, which takes every diff --git a/issues/every-driver-is-still-in-the-kernel.md b/issues/every-driver-is-still-in-the-kernel.md index c2bc63a6a7e..e48c73e81e1 100644 --- a/issues/every-driver-is-still-in-the-kernel.md +++ b/issues/every-driver-is-still-in-the-kernel.md @@ -13,7 +13,8 @@ sdk/abi/src/syscall.rs` is the check, and it finds none: the three `SYS_NIC_*` names went with the virtio NIC driver, which is `system/netstack/src/virtio_net.rs` now, the four `SYS_GPU_*` names with virtio-gpu's, which is the compositor's (`system/compositor/virtio-gpu`), and `SYS_HDA_BIND` with the -kernel's HDA stub. The kernel's virtio code left is its console's, and it +kernel's HDA stub. No virtio code is left in the kernel: its console's is +`system/virtio-console`'s, a `drives` row beside vdagent's, and the kernel drives no audio hardware. **Ordering ruling, and it is not negotiable: the IOMMU lands first and diff --git a/issues/kernelslice-outlives-its-allocation.md b/issues/kernelslice-outlives-its-allocation.md index aa57b29a3fb..818e714df4d 100644 --- a/issues/kernelslice-outlives-its-allocation.md +++ b/issues/kernelslice-outlives-its-allocation.md @@ -13,9 +13,9 @@ half that is not: `KernelSlice` is `Copy` and carries no lifetime, so a copy taken while the `PageAlloc`/`OwnedAlloc` is alive stays nameable after it has been dropped and its pages reissued. -`mm::Dma` is the shape that answers it: `Dma<'pool>` borrows the pool it came -out of, and `DmaPool::leak` is the only way to reach `'static`. The same -signature here would be `KernelSlice<'alloc>`. +The shape that answers it is a view that borrows what it came out of, with +one named way to reach `'static` — as the deleted `mm::Dma<'pool>` did with +`DmaPool::leak`: `KernelSlice<'alloc>`. What makes it more than a `git sed`: `elf::cache` holds a `cached_image: KernelSlice` over pages the cache owns and hands it to every process that @@ -33,6 +33,5 @@ reproduction. to promote, and a memory-safety residual is the worst thing to lose to a wrong fold). `KernelSlice` being `Copy` with no lifetime means nothing but hand-checking stops a copy naming pages that have been reissued — and the hand-check was done -once, in 2026-08-22, against holders that have moved since. `mm::Dma<'pool>` is -the shape that answers it. Owed by whoever writes the four holders' lifetimes +once, in 2026-08-22, against holders that have moved since. Owed by whoever writes the four holders' lifetimes down: `elf::cache`'s `cached_image`, `LoadedLib`, `TlsModule` and `UserStack`. diff --git a/issues/the-compiler-is-the-unused-oracle-for-kernel-region-lifetime.md b/issues/the-compiler-is-the-unused-oracle-for-kernel-region-lifetime.md index 586f66301aa..a38a5dc5988 100644 --- a/issues/the-compiler-is-the-unused-oracle-for-kernel-region-lifetime.md +++ b/issues/the-compiler-is-the-unused-oracle-for-kernel-region-lifetime.md @@ -13,10 +13,9 @@ a real slice at `region.rs:68`. One record turns on that: `issues/kernelslice-outlives-its-allocation.md` — the allocation can be gone. -**The shape of the answer is already written down, in the first of those.** It -names `mm::Dma` as the precedent — `Dma<'pool>` borrows the pool it came out of, -and `DmaPool::leak` is the only way to reach `'static` — and says the same -signature here is `KernelSlice<'alloc>`. It also names who owes the work: the +**The shape of the answer is already written down, in the first of those.** A +view that borrows the allocation it came out of, with one named way to reach +`'static`: `KernelSlice<'alloc>`. It also names who owes the work: the four holders whose lifetimes have to be written down, `elf::cache`'s `cached_image`, `LoadedLib`, `TlsModule` and `UserStack`. This track adds two things that record does not carry, and nothing else. diff --git a/issues/the-kernel-console-split-does-not-re-arm-across-a-guest-reset.md b/issues/the-kernel-console-split-does-not-re-arm-across-a-guest-reset.md deleted file mode 100644 index 7a69f090a90..00000000000 --- a/issues/the-kernel-console-split-does-not-re-arm-across-a-guest-reset.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -status: open -kind: tooling -opened: 2026-09-28 ---- - -# The kernel console split does not re-arm across a guest reset - -`build/kernelconsole.rs`'s `KernelConsole` withholds the virtio port's bytes -until the kernel's first record (`toyos_logstream::kernel_opening`) and -passes every byte after it: `held` is -`None` from then on. A guest reset inside one QEMU process does not re-arm it. -`tests/common/update.rs`'s `Rig::boot` boots the Headless profile, whose -stdio is that port, with `takes_the_reset`, and each `reboot` resets that -machine in place. Whatever the next boot's firmware and loader write on the -port reaches every stdio reader as kernel console. - -What the next boot writes there has never been captured. The first boot's -stream is measured. On QEMU 11.1.1's own edk2 it opens with -`ESC[2J ESC[01;01H ESC[=3h ESC[2J ESC[01;01H` twice, then -`BdsDxe: loading Boot0001 …`, as a failing test's console dump in the -orchestrator's #572 round-2 nightly shows. No log from #572's runs shows the -port after an in-process reset: each carries that prelude only where a stream -starts. So no signal is known to be in the stream at a reset. Re-arming on the -first boot's prelude would rest on the guess that the firmware repeats it. - -Nothing reds on it. The update tests look for a needle past an offset in the -console (`owed`, `reboot_until`), and firmware lines beside the needle do not -stop it matching. Every update test passed in #572's round-2 and round-3 -nightlies. If the firmware does write its handoff line there, -`Loader log: … so [ 1.234 cpu0 kernel] …` is a line `Serial::interleaved` -reports as a torn -kernel line. - -## Exit condition - -A capture of stdio across an in-process reset on a virtio profile. If the -firmware writes on the port there, the split re-arms on a signal that capture -shows, with a host test over the captured bytes. If it writes nothing, this -file is deleted with the capture in the commit. - -## Owner - -`build/kernelconsole.rs` and `tests/common/qemu.rs`'s stdio reader; unheld. diff --git a/issues/the-update-rig-the-guest-cut-deleted-is-still-cited.md b/issues/the-update-rig-the-guest-cut-deleted-is-still-cited.md index f574c90d27f..11ce07ce5c3 100644 --- a/issues/the-update-rig-the-guest-cut-deleted-is-still-cited.md +++ b/issues/the-update-rig-the-guest-cut-deleted-is-still-cited.md @@ -9,7 +9,7 @@ opened: 2026-10-03 #660 (`06788146b`) deleted `tests/common/update.rs` with its `Rig`, `tests/common/fwvars.rs` and `tests/updatecase/system.toml`. `git grep -n 'fwvars\|updatecase\|common/update\.rs\|Rig::\|vars::plant\|vars::live'` finds -seven lines in two files, each planning or describing a test on them: +six lines in one file, each planning or describing a test on them: - `issues/the-loader-does-only-what-must-precede-the-handover.md`, six lines. Stage 3's exit takes `fwvars::live` as @@ -17,10 +17,8 @@ seven lines in two files, each planning or describing a test on them: `tests/updatecase/system.toml` grants `slots` and that two tests run `updatecase`; one of its negative controls signs with `Rig::update` and reads the floor with `fwvars::live`, which its oracles name too. -- `issues/the-kernel-console-split-does-not-re-arm-across-a-guest-reset.md`, - one line: the reset in place it describes is `Rig::boot`'s. -What each should name is the tier its test has now, and stage A of +What it should name is the tier its test has now, and stage A of `issues/the-guest-suite-runs-only-what-no-cheaper-tier-reaches.md` gives it: `update_floor_is_the_images_own`, `update_refusals_boot_the_other_slot` and `update_grant_refuses_a_stray_partition` are host tests there and guest diff --git a/issues/the-virtio-drivers-match-only-a-non-transitional-device-id.md b/issues/the-virtio-drivers-match-only-a-non-transitional-device-id.md index 9e580acc8f4..7ce2bf3d258 100644 --- a/issues/the-virtio-drivers-match-only-a-non-transitional-device-id.md +++ b/issues/the-virtio-drivers-match-only-a-non-transitional-device-id.md @@ -18,15 +18,16 @@ The tree matches the `0x1040`-based id alone: - netstack's `CARDS` (`system/netstack/src/main.rs`) and its row in `images/system.toml`, `pci:1af4:1041`; -- the kernel's console, `VIRTIO_CONSOLE_DEVICE` `0x1043` - (`kernel/src/drivers/virtio_console.rs`). +- vdagent's and virtio-console's rows in `images/system.toml`, + `pci:07:80:00/1af4:1043`, and the matcher each program's claim is endowed + under. The launcher and the harness hide it: each asks QEMU for `virtio-net-pci-non-transitional` and `virtio-serial-pci-non-transitional` (`build/qemu.rs`, `tests/common/qemu.rs`). QEMU's own default is transitional on q35's root bus — `disable-legacy=auto` resolves to off for a device not behind a PCIe port — and VirtualBox's virtio-net is `1af4:1000`. On either, -netstack has no card and the kernel no virtio console. Cloud Hypervisor and +netstack has no card and the machine no virtio console. Cloud Hypervisor and crosvm use the `0x1040`-based ids. bhyve's virtio-net and virtio-blk are legacy-only, an I/O BAR and no virtio capability, so a conforming driver matches them and refuses them by name. @@ -39,11 +40,11 @@ devices offer the bit nowhere. So behind a unit such a NIC is one the supervisor refuses every holder (`translated` in `system/supervisor/src/main.rs`); the matches reach a card only on the path `issues/a-machine-without-an-iommu-refuses-every-claim.md` rules for a machine -with no unit, and the kernel's console on every machine. +with no unit. **Owner**: whoever holds `issues/toyos-runs-in-any-open-source-hypervisor.md`. -**Exit**: netstack's row, its `CARDS` and the kernel's console each name both -ids of their type; the kernel's console carries a guest boot whose console is +**Exit**: netstack's row, its `CARDS` and the two virtio-serial rows each name +both ids of their type; virtio-console carries a guest boot whose console is QEMU's transitional `virtio-serial-pci`, and refuses the same device under `disable-modern=on` by name. diff --git a/kernel/Cargo.toml b/kernel/Cargo.toml index 5b3796d9d5d..7b499f12cf7 100644 --- a/kernel/Cargo.toml +++ b/kernel/Cargo.toml @@ -503,7 +503,6 @@ toyos-abi = { path = "../sdk/abi" } [target.'cfg(target_os = "none")'.dependencies] toyos-acpi = { path = "../lib/acpi" } toyos-amdvi = { path = "amdvi" } -toyos-dma = { path = "dma" } toyos-blackbox = { path = "../lib/blackbox" } toyos-bootmap = { path = "../lib/bootmap" } toyos-elf = { path = "../lib/elf" } @@ -512,6 +511,7 @@ toyos-its = { path = "its" } toyos-microcode = { path = "../lib/microcode" } toyos-pci = { path = "pci" } toyos-phys = { path = "phys" } +toyos-register = { path = "register" } toyos-smmu = { path = "smmu" } toyos-tco = { path = "../lib/tco" } toyos-tsc = { path = "../lib/tsc" } diff --git a/kernel/dma/Cargo.toml b/kernel/dma/Cargo.toml deleted file mode 100644 index 5111c3b5cf2..00000000000 --- a/kernel/dma/Cargo.toml +++ /dev/null @@ -1,17 +0,0 @@ -# A member of the host workspace (root `Cargo.toml`), like toyos-userbound and -# toyos-pci: it is pure, it has no dependencies, and its tests run on the host. -# -# What lives here is where a typed access into a DMA region lands and whether it -# may land there. The kernel's `mm::Dma` is the only caller; it is a crate -# rather than a private function in `kernel/src/mm/dma.rs` because the whole -# table below costs milliseconds on the host and a boot in a guest, and because -# a bound that panics in the kernel cannot be asked a question — it can only be -# survived. - -[package] -name = "toyos-dma" -description = "Every bound and alignment a DMA view or a device register window checks, pure, forbid(unsafe_code)." -version = "0.1.0" -edition = "2021" -license = "MIT OR Apache-2.0" -publish = false diff --git a/kernel/dma/src/lib.rs b/kernel/dma/src/lib.rs deleted file mode 100644 index bfe6fd64e88..00000000000 --- a/kernel/dma/src/lib.rs +++ /dev/null @@ -1,371 +0,0 @@ -//! Where an access into a bounded window lands, and whether it may land there. -//! -//! Two windows: the DMA region a driver reads and writes through, and the -//! register file a *process* names an offset in through a syscall. The -//! arithmetic is the same shape and gets wrong in the same ways, so it is asked -//! in one place — on the host, as a value — rather than survived in a guest. -//! -//! Three questions, and a caller gets them wrong in three different ways. -//! -//! **[`within`] — the length, not the offset.** The bound this replaces -//! (`KernelSlice::ptr_at`) asserted `offset <= size` and said nothing about what -//! was read through the pointer it returned, so a `u32` read at `size - 1` ran -//! three bytes past the region and passed. Every access here names the number of -//! bytes it will touch, and the addition is `checked_add`: an `offset + len` that -//! wraps is refused rather than compared after wrapping. -//! -//! **[`aligned`] — a volatile access is not a `memcpy`.** `read_volatile` and -//! `write_volatile` require the pointer to be naturally aligned for the type; on -//! x86-64 a misaligned one usually *works*, which is exactly why nothing catches -//! it. The unaligned discipline (`read_unaligned`/`write_unaligned`) has no such -//! requirement and does not ask this question at all. -//! -//! **[`register`] — the offset is the caller's.** `within` and `aligned` bound a -//! kernel driver's own arithmetic, which is a kernel bug when it is wrong. -//! `register` bounds a number that crossed the user/kernel boundary, which is -//! *never* a bug in this kernel: it answers a [`Register`] whose only -//! constructor this is, so an offset nobody checked cannot reach a device's -//! register file at all. -//! -//! Pure. No I/O, no allocation, no `unsafe`, nothing read from a device and -//! nothing named outside this crate. - -#![no_std] -#![forbid(unsafe_code)] - -use core::fmt; - -/// Why an access into a DMA region is refused. -/// -/// A refusal here is a kernel bug and not a device one: every offset and length -/// that reaches it is the driver's own arithmetic, so `mm::dma` panics on one. -/// It is a value rather than a panic *here* so that the table below can ask the -/// question on the host instead of surviving the answer in a guest. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub enum Refused { - /// `len` bytes at `offset` are not all inside a region of `size` bytes — - /// including the case where `offset + len` does not fit a `usize`. - Past { offset: usize, len: usize, size: usize }, - /// The address a `T` would be read or written at is not a multiple of - /// `align`, which a volatile access requires. - Unaligned { at: usize, align: usize }, -} - -impl fmt::Display for Refused { - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - match self { - Self::Past { offset, len, size } => write!( - f, - "{len} byte(s) at {offset:#x} run past a region of {size:#x}" - ), - Self::Unaligned { at, align } => { - write!(f, "{at:#x} is not {align}-byte aligned") - } - } - } -} - -/// Whether `len` bytes at `offset` are all inside a region of `size` bytes. -/// -/// `len` is the number of bytes the access will touch — `size_of::()` for a -/// typed read or write, `src.len()` for a byte copy — and never zero-by-default: -/// the whole point is that the length is part of the question. -/// -/// A zero-length access at `offset == size` is allowed. It touches nothing, and -/// the one-past-the-end address is the natural end of an empty subview. -#[inline] -pub fn within(offset: usize, len: usize, size: usize) -> Result<(), Refused> { - match offset.checked_add(len) { - Some(end) if end <= size => Ok(()), - _ => Err(Refused::Past { offset, len, size }), - } -} - -/// Whether a value of alignment `align` at `base + offset` is naturally aligned. -/// -/// `base` is the region's own address, because alignment is a property of the -/// address the access lands on and not of the offset alone — a region placed at -/// an odd address makes every even offset in it odd. -/// -/// `align` is `align_of::()`, which is a power of two for every `T`; the -/// mask below is that fact and not an assumption about the caller. -#[inline] -pub fn aligned(base: usize, offset: usize, align: usize) -> Result<(), Refused> { - let at = base.wrapping_add(offset); - if align != 0 && at & (align - 1) == 0 { - Ok(()) - } else { - Err(Refused::Unaligned { at, align }) - } -} - -/// One register access a caller named, already checked against the window it -/// has to land in. -/// -/// **The offset came from userland**, so there is no public field and no -/// constructor but [`register`]: a kernel register read or write that takes one -/// of these cannot be reached with an offset nobody asked the question about. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub struct Register { - offset: u64, - bytes: u64, -} - -impl Register { - pub const fn offset(self) -> u64 { - self.offset - } - - pub const fn bytes(self) -> u64 { - self.bytes - } -} - -/// Why a register access is refused. A caller's bad argument and never a -/// kernel bug, so every one of these is a value the syscall answers with. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub enum RefusedRegister { - /// An access of no width names no register. - Widthless { offset: u64 }, - /// Not a multiple of its own width. A register file is addressed by whole - /// registers, and a straddling access reads half of two of them. - Unaligned { offset: u64, bytes: u64 }, - /// `bytes` at `offset` are not all inside a window of `window` bytes — - /// including the case where `offset + bytes` does not fit a `u64`, which is - /// the one a comparison made after wrapping lets through. - Past { offset: u64, bytes: u64, window: u64 }, -} - -impl fmt::Display for RefusedRegister { - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - match self { - Self::Widthless { offset } => { - write!(f, "an access of no width at {offset:#x} names no register") - } - Self::Unaligned { offset, bytes } => { - write!(f, "{offset:#x} is not a multiple of the {bytes} byte(s) read there") - } - Self::Past { offset, bytes, window } => write!( - f, - "{bytes} byte(s) at {offset:#x} run past a register window of {window:#x}" - ), - } - } -} - -/// Whether `bytes` at `offset` name a whole register inside a window of -/// `window` bytes, and the witness that says so. -/// -/// All three questions in one answer, because a caller that has the witness has -/// had all three asked: the width is not zero, the offset is a multiple of it, -/// and the access ends inside the window. The addition is `checked_add`, so an -/// `offset` near `u64::MAX` is refused rather than compared after wrapping — -/// that is the shape that turns a caller-chosen number into a panic in a kernel -/// built with `overflow-checks`. -#[inline] -pub fn register(offset: u64, bytes: u64, window: u64) -> Result { - if bytes == 0 { - return Err(RefusedRegister::Widthless { offset }); - } - if !offset.is_multiple_of(bytes) { - return Err(RefusedRegister::Unaligned { offset, bytes }); - } - match offset.checked_add(bytes) { - Some(end) if end <= window => Ok(Register { offset, bytes }), - _ => Err(RefusedRegister::Past { offset, bytes, window }), - } -} - -#[cfg(test)] -mod tests { - use super::*; - - /// The hole the drivers sweep closed by hand at 35 sites, asked directly: - /// a four-byte read one byte before the end of the region. - #[test] - fn a_u32_at_size_minus_one_is_refused() { - assert_eq!( - within(0x1000 - 1, 4, 0x1000), - Err(Refused::Past { offset: 0xFFF, len: 4, size: 0x1000 }) - ); - } - - /// And the access the old bound and the new one agree about: the last whole - /// `T` in the region. - #[test] - fn a_u32_at_size_minus_four_is_allowed() { - assert_eq!(within(0x1000 - 4, 4, 0x1000), Ok(())); - } - - /// The offset alone is what `ptr_at` checked, and it is not the question: - /// every one of these offsets is `<= size` and every one of these accesses - /// runs past the end. - #[test] - fn an_offset_inside_the_region_does_not_make_the_access_inside_it() { - for (len, offset) in [(2usize, 0xFFFusize), (4, 0xFFD), (8, 0xFF9), (256, 0xF01)] { - assert!(offset <= 0x1000, "the offset itself is in range"); - assert!( - within(offset, len, 0x1000).is_err(), - "{len} bytes at {offset:#x} must be refused" - ); - } - } - - #[test] - fn the_whole_region_is_inside_itself() { - assert_eq!(within(0, 0x1000, 0x1000), Ok(())); - assert_eq!(within(0, 0x1001, 0x1000), Err(Refused::Past { - offset: 0, - len: 0x1001, - size: 0x1000, - })); - } - - /// An empty access at the end is not out of bounds; one past it is. - #[test] - fn a_zero_length_access_lands_at_the_end_and_no_further() { - assert_eq!(within(0x1000, 0, 0x1000), Ok(())); - assert!(within(0x1001, 0, 0x1000).is_err()); - } - - /// The addition is checked, not wrapped. Compared after wrapping, this pair - /// is `Ok` — which is the shape that turns a length into an oracle for the - /// whole address space. - #[test] - fn an_offset_plus_length_that_wraps_is_refused() { - assert!(usize::MAX.wrapping_add(2) < 0x1000, "the wrapped sum looks in range"); - assert_eq!( - within(usize::MAX, 2, 0x1000), - Err(Refused::Past { offset: usize::MAX, len: 2, size: 0x1000 }) - ); - } - - /// A region of nothing holds nothing. - #[test] - fn an_empty_region_admits_only_an_empty_access() { - assert_eq!(within(0, 0, 0), Ok(())); - assert!(within(0, 1, 0).is_err()); - } - - #[test] - fn a_natural_offset_from_a_natural_base_is_aligned() { - // A page-aligned region, the placement every DMA structure in the tree - // has, and the offsets the rings actually use. - assert_eq!(aligned(0x2000, 0, 8), Ok(())); - assert_eq!(aligned(0x2000, 16, 8), Ok(())); - assert_eq!(aligned(0x2000, 4, 4), Ok(())); - assert_eq!(aligned(0x2000, 1, 1), Ok(())); - } - - #[test] - fn an_unaligned_volatile_access_is_refused() { - assert_eq!(aligned(0x2000, 1, 4), Err(Refused::Unaligned { at: 0x2001, align: 4 })); - assert_eq!(aligned(0x2000, 2, 4), Err(Refused::Unaligned { at: 0x2002, align: 4 })); - assert_eq!(aligned(0x2000, 4, 8), Err(Refused::Unaligned { at: 0x2004, align: 8 })); - } - - /// Alignment is about the address and not about the offset: a region that is - /// itself misplaced makes an offset that looks fine land wrong. - #[test] - fn the_base_is_part_of_the_question() { - assert_eq!(aligned(0x2002, 0, 4), Err(Refused::Unaligned { at: 0x2002, align: 4 })); - assert_eq!(aligned(0x2002, 2, 4), Ok(())); - } - - /// A dword at the last dword of a 4 KiB register file, and the same dword - /// one register further on. - #[test] - fn the_last_whole_register_is_inside_the_window() { - assert_eq!(register(0xFFC, 4, 0x1000), Ok(Register { offset: 0xFFC, bytes: 4 })); - assert_eq!( - register(0x1000, 4, 0x1000), - Err(RefusedRegister::Past { offset: 0x1000, bytes: 4, window: 0x1000 }) - ); - } - - /// **The one that panics a kernel built with `overflow-checks`.** Every - /// pair here has `offset + bytes` wrapping to a small number, which a - /// comparison made after the addition reads as in range — and which the - /// addition itself does not survive. - #[test] - fn an_offset_that_wraps_its_own_width_is_refused() { - for (offset, bytes) in [(u64::MAX, 1u64), (u64::MAX - 1, 2), (0xFFFF_FFFF_FFFF_FFFC, 4)] { - assert_eq!( - register(offset, bytes, 0x1000), - Err(RefusedRegister::Past { offset, bytes, window: 0x1000 }), - "{bytes} byte(s) at {offset:#x} must be refused" - ); - } - } - - /// A register file is addressed by whole registers: half of one and half of - /// the next is not an access this answers. - #[test] - fn an_access_that_straddles_two_registers_is_refused() { - assert_eq!( - register(1, 2, 0x1000), - Err(RefusedRegister::Unaligned { offset: 1, bytes: 2 }) - ); - assert_eq!( - register(2, 4, 0x1000), - Err(RefusedRegister::Unaligned { offset: 2, bytes: 4 }) - ); - assert_eq!(register(2, 2, 0x1000), Ok(Register { offset: 2, bytes: 2 })); - assert_eq!(register(0xFFF, 1, 0x1000), Ok(Register { offset: 0xFFF, bytes: 1 })); - } - - /// Asked before the modulo, which would divide by zero, and before the - /// addition, which would admit every offset in the space. - #[test] - fn an_access_of_no_width_is_refused() { - assert_eq!(register(0, 0, 0x1000), Err(RefusedRegister::Widthless { offset: 0 })); - assert_eq!( - register(u64::MAX, 0, 0x1000), - Err(RefusedRegister::Widthless { offset: u64::MAX }) - ); - } - - /// A window of nothing holds no register. - #[test] - fn an_empty_window_admits_no_register() { - assert_eq!( - register(0, 1, 0), - Err(RefusedRegister::Past { offset: 0, bytes: 1, window: 0 }) - ); - } - - /// The refusals are what a person reading a kernel panic gets, so they say - /// the numbers rather than naming a variant. - #[test] - fn a_refusal_names_its_numbers() { - extern crate std; - use std::string::ToString; - assert_eq!( - Refused::Past { offset: 0xFFF, len: 4, size: 0x1000 }.to_string(), - "4 byte(s) at 0xfff run past a region of 0x1000" - ); - assert_eq!( - Refused::Unaligned { at: 0x2001, align: 4 }.to_string(), - "0x2001 is not 4-byte aligned" - ); - } - - /// And the same for the three a caller reads back out of a syscall. - #[test] - fn a_refused_register_names_its_numbers() { - extern crate std; - use std::string::ToString; - assert_eq!( - RefusedRegister::Widthless { offset: 0x10 }.to_string(), - "an access of no width at 0x10 names no register" - ); - assert_eq!( - RefusedRegister::Unaligned { offset: 1, bytes: 2 }.to_string(), - "0x1 is not a multiple of the 2 byte(s) read there" - ); - assert_eq!( - RefusedRegister::Past { offset: 0x1000, bytes: 4, window: 0x1000 }.to_string(), - "4 byte(s) at 0x1000 run past a register window of 0x1000" - ); - } -} diff --git a/kernel/pci/src/choose.rs b/kernel/pci/src/choose.rs deleted file mode 100644 index 4305649f1ee..00000000000 --- a/kernel/pci/src/choose.rs +++ /dev/null @@ -1,42 +0,0 @@ -//! Which of the functions one `vendor:device` names the kernel's console -//! drives: the decision alone, over what the kernel read of each. -//! -//! **Two functions can carry one ID and be two different devices.** QEMU's -//! virtio-serial is `1af4:1043` whether it carries the kernel's single-port -//! console or a multiport channel a process drives, so the console driver -//! picks by what the device offers. - -/// Virtio 1.2 §5.3.3: the console device's `VIRTIO_CONSOLE_F_MULTIPORT`. -pub const VIRTIO_CONSOLE_F_MULTIPORT: u64 = 1 << 1; - -/// The function the kernel's console drives, among console functions whose -/// device features are `features`, in enumeration order: the first that does -/// not offer `VIRTIO_CONSOLE_F_MULTIPORT`. -/// -/// A single-port driver drives nothing else. A multiport device names its -/// console ports by `CONSOLE_PORT` messages on control queues (§5.3.6.2), -/// which a single-port driver has none of. -pub fn console(features: &[u64]) -> Option { - features.iter().position(|offered| offered & VIRTIO_CONSOLE_F_MULTIPORT == 0) -} - -#[cfg(test)] -mod tests { - use super::*; - - /// `VIRTIO_F_VERSION_1` beside each, as QEMU offers it on both functions. - const VERSION_1: u64 = 1 << 32; - const SINGLE: u64 = VERSION_1; - const MULTI: u64 = VERSION_1 | VIRTIO_CONSOLE_F_MULTIPORT; - - #[test] - fn the_console_is_the_first_function_without_multiport() { - assert_eq!(console(&[MULTI, SINGLE, SINGLE]), Some(1)); - } - - #[test] - fn multiport_functions_alone_give_no_console() { - assert_eq!(console(&[MULTI, MULTI]), None); - assert_eq!(console(&[]), None); - } -} diff --git a/kernel/pci/src/lib.rs b/kernel/pci/src/lib.rs index b0585f09347..90dd9cfa7ba 100644 --- a/kernel/pci/src/lib.rs +++ b/kernel/pci/src/lib.rs @@ -31,11 +31,9 @@ pub mod aperture; pub mod af; pub mod bar; pub mod caps; -pub mod choose; pub mod express; pub mod msi; pub mod msix; pub mod pages; pub mod pm; -pub use choose::console; diff --git a/kernel/pure/carrier.rs b/kernel/pure/carrier.rs new file mode 100644 index 00000000000..728978acece --- /dev/null +++ b/kernel/pure/carrier.rs @@ -0,0 +1,116 @@ +//! Who carries the machine's log off it: the console UART, or the console +//! holder that says another channel does. One word, which the console's one +//! writer reads before each record it would put on the UART. +//! +//! **A holder's end is its release**, so a holder that dies cannot leave the +//! UART quiet; and a fatal path or the stop takes the log back to the UART for +//! good, which nothing undoes. + +use core::sync::atomic::{AtomicUsize, Ordering}; + +const UART: usize = 0; +const FOR_GOOD: usize = 1; + +pub struct Carrier(AtomicUsize); + +/// Why a holder may not carry the log. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Refused { + /// Another holder carries it. + Another, + /// A fatal path or the stop gave it to the UART for good. + ForGood, +} + +impl Carrier { + pub const fn new() -> Self { + Self(AtomicUsize::new(UART)) + } + + /// `holder` carries the log from now; answered `Ok` again for the holder + /// that already does. A holder is an address, which is never 0 or 1. + pub fn carry(&self, holder: usize) -> Result<(), Refused> { + assert!(holder > FOR_GOOD, "carrier: {holder:#x} is no holder's address"); + match self.0.compare_exchange(UART, holder, Ordering::SeqCst, Ordering::SeqCst) { + Ok(_) => Ok(()), + Err(now) if now == holder => Ok(()), + Err(FOR_GOOD) => Err(Refused::ForGood), + Err(_) => Err(Refused::Another), + } + } + + /// `holder` no longer carries the log, by its word or by its end: whether + /// it did. + pub fn release(&self, holder: usize) -> bool { + self.0.compare_exchange(holder, UART, Ordering::SeqCst, Ordering::SeqCst).is_ok() + } + + /// The UART carries everything from here to the machine's end. + pub fn for_good(&self) { + self.0.store(FOR_GOOD, Ordering::SeqCst); + } + + /// Whether a holder carries the log, so the UART takes none of it. + pub fn elsewhere(&self) -> bool { + !matches!(self.0.load(Ordering::SeqCst), UART | FOR_GOOD) + } +} + +impl Default for Carrier { + fn default() -> Self { + Self::new() + } +} + +#[cfg(test)] +mod tests { + use super::*; + + const LOGKEEPER: usize = 0x1000; + const OTHER: usize = 0x2000; + + #[test] + fn a_holder_carries_until_it_lets_go() { + let carrier = Carrier::new(); + assert!(!carrier.elsewhere()); + assert_eq!(carrier.carry(LOGKEEPER), Ok(())); + assert_eq!(carrier.carry(LOGKEEPER), Ok(())); + assert!(carrier.elsewhere()); + assert!(carrier.release(LOGKEEPER)); + assert!(!carrier.elsewhere()); + assert!(!carrier.release(LOGKEEPER)); + } + + /// The holder's console goes with its process, which is what a crash of + /// the holder is: its end releases it, and nothing else need run. + #[test] + fn a_holders_end_gives_the_log_back_to_the_uart() { + let carrier = Carrier::new(); + carrier.carry(LOGKEEPER).expect("the first holder carries"); + // The end of another holder is not this one's. + assert!(!carrier.release(OTHER)); + assert!(carrier.elsewhere()); + assert!(carrier.release(LOGKEEPER)); + assert!(!carrier.elsewhere()); + assert_eq!(carrier.carry(OTHER), Ok(()), "a later holder may carry it again"); + } + + #[test] + fn a_second_holder_is_refused() { + let carrier = Carrier::new(); + carrier.carry(LOGKEEPER).expect("the first holder carries"); + assert_eq!(carrier.carry(OTHER), Err(Refused::Another)); + assert!(carrier.release(LOGKEEPER)); + } + + #[test] + fn for_good_is_never_undone() { + let carrier = Carrier::new(); + carrier.carry(LOGKEEPER).expect("the first holder carries"); + carrier.for_good(); + assert!(!carrier.elsewhere()); + assert!(!carrier.release(LOGKEEPER), "the holder no longer carries it to let go"); + assert_eq!(carrier.carry(LOGKEEPER), Err(Refused::ForGood)); + assert!(!carrier.elsewhere()); + } +} diff --git a/kernel/pure/lib.rs b/kernel/pure/lib.rs index d1775966a5d..61b521acb3e 100644 --- a/kernel/pure/lib.rs +++ b/kernel/pure/lib.rs @@ -10,6 +10,7 @@ extern crate alloc; extern crate std; pub mod bootwrite; +pub mod carrier; pub mod clock; pub mod cpu; pub mod dirnodes; diff --git a/kernel/register/Cargo.toml b/kernel/register/Cargo.toml new file mode 100644 index 00000000000..d565a3aa229 --- /dev/null +++ b/kernel/register/Cargo.toml @@ -0,0 +1,15 @@ +# A member of the host workspace (root `Cargo.toml`), like toyos-userbound and +# toyos-pci: it is pure, it has no dependencies, and its tests run on the host. +# +# What lives here is whether a register access a process named lands inside its +# window. `pcidev` is the only caller; it is a crate rather than a private +# function because the table of refusals costs milliseconds on the host and a +# boot in a guest. + +[package] +name = "toyos-register" +description = "Whether a register access a process named lands whole inside its device's window, pure, forbid(unsafe_code)." +version = "0.1.0" +edition = "2021" +license = "MIT OR Apache-2.0" +publish = false diff --git a/kernel/register/src/lib.rs b/kernel/register/src/lib.rs new file mode 100644 index 00000000000..6cf6cee669a --- /dev/null +++ b/kernel/register/src/lib.rs @@ -0,0 +1,178 @@ +//! Whether a register access a *process* named lands inside the window it +//! has to: the offset crossed the user/kernel boundary, so a bad one is never +//! a bug in this kernel. [`register`] answers a [`Register`] whose only +//! constructor it is, so an offset nobody checked cannot reach a device's +//! register file at all. Asked on the host, as a value, rather than survived +//! in a guest. +//! +//! Pure. No I/O, no allocation, no `unsafe`, nothing read from a device and +//! nothing named outside this crate. + +#![no_std] +#![forbid(unsafe_code)] + +use core::fmt; + +/// One register access a caller named, already checked against the window it +/// has to land in. +/// +/// **The offset came from userland**, so there is no public field and no +/// constructor but [`register`]: a kernel register read or write that takes one +/// of these cannot be reached with an offset nobody asked the question about. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct Register { + offset: u64, + bytes: u64, +} + +impl Register { + pub const fn offset(self) -> u64 { + self.offset + } + + pub const fn bytes(self) -> u64 { + self.bytes + } +} + +/// Why a register access is refused. A caller's bad argument and never a +/// kernel bug, so every one of these is a value the syscall answers with. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum RefusedRegister { + /// An access of no width names no register. + Widthless { offset: u64 }, + /// Not a multiple of its own width. A register file is addressed by whole + /// registers, and a straddling access reads half of two of them. + Unaligned { offset: u64, bytes: u64 }, + /// `bytes` at `offset` are not all inside a window of `window` bytes — + /// including the case where `offset + bytes` does not fit a `u64`, which is + /// the one a comparison made after wrapping lets through. + Past { offset: u64, bytes: u64, window: u64 }, +} + +impl fmt::Display for RefusedRegister { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + Self::Widthless { offset } => { + write!(f, "an access of no width at {offset:#x} names no register") + } + Self::Unaligned { offset, bytes } => { + write!(f, "{offset:#x} is not a multiple of the {bytes} byte(s) read there") + } + Self::Past { offset, bytes, window } => write!( + f, + "{bytes} byte(s) at {offset:#x} run past a register window of {window:#x}" + ), + } + } +} + +/// Whether `bytes` at `offset` name a whole register inside a window of +/// `window` bytes, and the witness that says so. +/// +/// All three questions in one answer, because a caller that has the witness has +/// had all three asked: the width is not zero, the offset is a multiple of it, +/// and the access ends inside the window. The addition is `checked_add`, so an +/// `offset` near `u64::MAX` is refused rather than compared after wrapping — +/// that is the shape that turns a caller-chosen number into a panic in a kernel +/// built with `overflow-checks`. +#[inline] +pub fn register(offset: u64, bytes: u64, window: u64) -> Result { + if bytes == 0 { + return Err(RefusedRegister::Widthless { offset }); + } + if !offset.is_multiple_of(bytes) { + return Err(RefusedRegister::Unaligned { offset, bytes }); + } + match offset.checked_add(bytes) { + Some(end) if end <= window => Ok(Register { offset, bytes }), + _ => Err(RefusedRegister::Past { offset, bytes, window }), + } +} + +#[cfg(test)] +mod tests { + use super::*; + + /// A dword at the last dword of a 4 KiB register file, and the same dword + /// one register further on. + #[test] + fn the_last_whole_register_is_inside_the_window() { + assert_eq!(register(0xFFC, 4, 0x1000), Ok(Register { offset: 0xFFC, bytes: 4 })); + assert_eq!( + register(0x1000, 4, 0x1000), + Err(RefusedRegister::Past { offset: 0x1000, bytes: 4, window: 0x1000 }) + ); + } + + /// **The one that panics a kernel built with `overflow-checks`.** Every + /// pair here has `offset + bytes` wrapping to a small number, which a + /// comparison made after the addition reads as in range — and which the + /// addition itself does not survive. + #[test] + fn an_offset_that_wraps_its_own_width_is_refused() { + for (offset, bytes) in [(u64::MAX, 1u64), (u64::MAX - 1, 2), (0xFFFF_FFFF_FFFF_FFFC, 4)] { + assert_eq!( + register(offset, bytes, 0x1000), + Err(RefusedRegister::Past { offset, bytes, window: 0x1000 }), + "{bytes} byte(s) at {offset:#x} must be refused" + ); + } + } + + /// A register file is addressed by whole registers: half of one and half of + /// the next is not an access this answers. + #[test] + fn an_access_that_straddles_two_registers_is_refused() { + assert_eq!( + register(1, 2, 0x1000), + Err(RefusedRegister::Unaligned { offset: 1, bytes: 2 }) + ); + assert_eq!( + register(2, 4, 0x1000), + Err(RefusedRegister::Unaligned { offset: 2, bytes: 4 }) + ); + assert_eq!(register(2, 2, 0x1000), Ok(Register { offset: 2, bytes: 2 })); + assert_eq!(register(0xFFF, 1, 0x1000), Ok(Register { offset: 0xFFF, bytes: 1 })); + } + + /// Asked before the modulo, which would divide by zero, and before the + /// addition, which would admit every offset in the space. + #[test] + fn an_access_of_no_width_is_refused() { + assert_eq!(register(0, 0, 0x1000), Err(RefusedRegister::Widthless { offset: 0 })); + assert_eq!( + register(u64::MAX, 0, 0x1000), + Err(RefusedRegister::Widthless { offset: u64::MAX }) + ); + } + + /// A window of nothing holds no register. + #[test] + fn an_empty_window_admits_no_register() { + assert_eq!( + register(0, 1, 0), + Err(RefusedRegister::Past { offset: 0, bytes: 1, window: 0 }) + ); + } + + /// The refusals are what a caller reads back out of a syscall, so they + /// say the numbers rather than naming a variant. + #[test] + fn a_refused_register_names_its_numbers() { + extern crate std; + use std::string::ToString; + assert_eq!( + RefusedRegister::Widthless { offset: 0x10 }.to_string(), + "an access of no width at 0x10 names no register" + ); + assert_eq!( + RefusedRegister::Unaligned { offset: 1, bytes: 2 }.to_string(), + "0x1 is not a multiple of the 2 byte(s) read there" + ); + assert_eq!( + RefusedRegister::Past { offset: 0x1000, bytes: 4, window: 0x1000 }.to_string(), + "4 byte(s) at 0x1000 run past a register window of 0x1000" + ); + } +} diff --git a/kernel/src/arch/aarch64/irqchip.rs b/kernel/src/arch/aarch64/irqchip.rs index a61856ab5f7..0e18dcfae63 100644 --- a/kernel/src/arch/aarch64/irqchip.rs +++ b/kernel/src/arch/aarch64/irqchip.rs @@ -130,8 +130,9 @@ fn read_sysreg_pmr() -> u64 { pub struct Gic { /// Every GIC CPU interface the MADT names and enables, in its order. pub cpus: Vec, - /// Each redistributor range's physical base, and the range, mapped whole. - ranges: Vec<(u64, Mmio)>, + /// Each redistributor range's physical base and length, and the range, + /// mapped whole. + ranges: Vec<(u64, u64, Mmio)>, } impl Gic { @@ -146,8 +147,8 @@ impl Gic { } self.ranges .iter() - .find_map(|&(base, range)| { - toyos_gicv3::find_redistributor(range.size(), affinity, |at| range.read_u64(at + GICR_TYPER)) + .find_map(|&(base, length, range)| { + toyos_gicv3::find_redistributor(length, affinity, |at| range.read_u64(at + GICR_TYPER)) .map(|offset| base + offset) }) .unwrap_or_else(|| panic!("GIC: no redistributor answers for MPIDR affinity {affinity:#x}")) @@ -169,6 +170,7 @@ pub fn init(rsdp_addr: u64) -> Gic { MadtEntry::Gicd { base, .. } => gicd = Some(base), MadtEntry::Gicr { base, length } => ranges.push(( base, + u64::from(length), crate::mm::paging::map_mmio(base, u64::from(length), MmioPolicy::Uncacheable), )), MadtEntry::Gicc(gicc) if gicc.enabled => cpus.push(gicc), diff --git a/kernel/src/arch/aarch64/smmu/domain.rs b/kernel/src/arch/aarch64/smmu/domain.rs index ae58e56feff..ca4fb22b9b7 100644 --- a/kernel/src/arch/aarch64/smmu/domain.rs +++ b/kernel/src/arch/aarch64/smmu/domain.rs @@ -86,13 +86,8 @@ impl Live { } } -/// Why `function` keeps the machine's own addresses rather than a domain of -/// its own: never, since nothing this kernel reads of the IORT says so. -pub fn untranslated(_function: StreamId) -> Option<&'static str> { - None -} - -/// Why no claim may have `function`: never, as [`untranslated`]. +/// Why no claim may have `function`: never, since nothing this kernel reads +/// of the IORT says it keeps the machine's own addresses. pub fn unclaimable(_function: StreamId) -> Option<&'static str> { None } diff --git a/kernel/src/arch/aarch64/trap.rs b/kernel/src/arch/aarch64/trap.rs index 6400a0a2512..594e38bc82a 100644 --- a/kernel/src/arch/aarch64/trap.rs +++ b/kernel/src/arch/aarch64/trap.rs @@ -337,6 +337,10 @@ fn user_abort(frame: &mut Frame) { fn user_fatal(frame: &Frame) -> ! { let prev = percpu::swap_fault_state(CpuFaultState::Fatal); let recursive = matches!(prev, CpuFaultState::Fatal | CpuFaultState::Panic); + // Before the first record of a report that halts the machine. + if recursive { + crate::log::console::on_the_uart_for_good(); + } // First: a panic anywhere below reaches the panic handler as DOUBLE // PANIC, which can only report what was captured here. crate::panic::record_fault(class_name(frame.esr), frame.elr, frame.far, frame.esr); diff --git a/kernel/src/arch/x86_64/amdvi/domain.rs b/kernel/src/arch/x86_64/amdvi/domain.rs index 5350c88a29e..74f983749ab 100644 --- a/kernel/src/arch/x86_64/amdvi/domain.rs +++ b/kernel/src/arch/x86_64/amdvi/domain.rs @@ -97,7 +97,7 @@ pub(super) fn claimed(id: u16) { /// own, if it does: no programmed unit serves it, so nothing would translate /// one; or an IVMD names it, so firmware reserves memory it must keep /// reaching, which the identity domain maps and a domain of its own would not. -pub fn untranslated(stream: StreamId) -> Option<&'static str> { +fn untranslated(stream: StreamId) -> Option<&'static str> { let held = DOMAINS.lock(); let amd = held.as_ref()?; let id = stream.requester(); diff --git a/kernel/src/arch/x86_64/idt/exceptions.rs b/kernel/src/arch/x86_64/idt/exceptions.rs index 9d8cc94f24a..7de3b76138c 100644 --- a/kernel/src/arch/x86_64/idt/exceptions.rs +++ b/kernel/src/arch/x86_64/idt/exceptions.rs @@ -362,6 +362,7 @@ fn crash_report_panic(info: &core::panic::PanicInfo, rbp: u64) { /// Double fault handler — runs on IST1. Always from kernel. Never returns. pub(super) fn double_fault_handler(frame: &TrapFrame) -> ! { + crate::log::console::on_the_uart_for_good(); let cr2 = cpu::read_cr2(); let cpu_id = percpu::cpu_id(); let tid = percpu::current_tid(); @@ -448,6 +449,7 @@ pub(super) fn double_fault_handler(frame: &TrapFrame) -> ! { // Untested: firmware leaves CR4.MCE set, so a machine check reaches here, but // nothing in the suite can stage one. pub(super) fn machine_check_handler(frame: &TrapFrame) -> ! { + crate::log::console::on_the_uart_for_good(); log!("MACHINE CHECK on CPU {}", percpu::cpu_id()); let ctx = ExceptionContext { frame, cr2: 0 }; crash_report(&CrashInfo::Exception(&ctx)); @@ -518,6 +520,10 @@ pub(super) fn exception_handler(frame: &TrapFrame) -> ! { fn fatal_exception(ctx: &ExceptionContext) -> ! { let prev = percpu::swap_fault_state(CpuFaultState::Fatal); let recursive = prev == CpuFaultState::Fatal || prev == CpuFaultState::Panic; + // Before the first record of a report that halts the machine. + if recursive || !ctx.ring().is_user() { + crate::log::console::on_the_uart_for_good(); + } // Must run first: a panic anywhere below reaches the panic handler as // DOUBLE PANIC, which can only report what was captured here. diff --git a/kernel/src/arch/x86_64/idt/nmi.rs b/kernel/src/arch/x86_64/idt/nmi.rs index a968b3e3348..ba3098cad81 100644 --- a/kernel/src/arch/x86_64/idt/nmi.rs +++ b/kernel/src/arch/x86_64/idt/nmi.rs @@ -131,7 +131,7 @@ fn stage_nested_if_armed() { /// Logs nothing: the interrupted context may be mid-publish of its own record, and one from here would garble the ring `halt_all_cpus` reads. `build/sourcegate.rs`'s `nmi_does_not_log` is the gate. extern "sysv64" fn nested_nmi(rip: u64, rsp: u64) -> ! { // Let go of before the halt: its flush, finding this CPU's own fatal path - // holding the registers, would drain raw and leave virtio-console out. + // holding the registers, would drain raw. { let mut uart = crate::drivers::serial::panic_registers(); uart.write(b"\n[nmi] NESTED NMI on cpu "); diff --git a/kernel/src/arch/x86_64/iommu_unit.rs b/kernel/src/arch/x86_64/iommu_unit.rs index b34cbc47ba7..d27805fb5f7 100644 --- a/kernel/src/arch/x86_64/iommu_unit.rs +++ b/kernel/src/arch/x86_64/iommu_unit.rs @@ -147,20 +147,10 @@ pub mod domain { } } - /// Why `stream` keeps the machine's own addresses rather than a domain of - /// its own, if it does: AMD-Vi's reasons. VT-d answers `None` for every - /// function, an RMRR's included, which it moves off the identity domain - /// like any other. - pub fn untranslated(stream: StreamId) -> Option<&'static str> { - match backend() { - Backend::AmdVi => amdvi::domain::untranslated(stream), - Backend::Vtd | Backend::None => None, - } - } - - /// Why no claim may have `stream`, if none may: [`untranslated`]'s - /// reasons and the requester ids AMD-Vi's platform speaks as. VT-d - /// answers `None`. + /// Why no claim may have `stream`, if none may: AMD-Vi's reasons for a + /// function that keeps the machine's own addresses, and the requester ids + /// its platform speaks as. VT-d answers `None` for every function, an + /// RMRR's included, which it moves off the identity domain like any other. pub fn unclaimable(stream: StreamId) -> Option<&'static str> { match backend() { Backend::AmdVi => amdvi::domain::unclaimable(stream), diff --git a/kernel/src/drivers/mod.rs b/kernel/src/drivers/mod.rs index b2774fed09c..ab30683f59b 100644 --- a/kernel/src/drivers/mod.rs +++ b/kernel/src/drivers/mod.rs @@ -1,17 +1,11 @@ //! Device drivers. //! A `SAFETY:` comment here must justify why the block could not be removed, not just why it is sound. //! `warn` suffices here because CI's clippy step already runs with `-D warnings`. -//! No driver may touch DMA memory except through [`crate::mm::Dma`], whose constructor is private to `mm::dma`. #![warn(clippy::undocumented_unsafe_blocks)] pub mod serial; pub mod serial_lock; pub mod acpi; pub mod pci; -pub mod virtio; -pub mod virtio_console; pub mod gop; pub mod panic_console; - -/// The pool every driver here allocates its DMA out of. -pub use crate::mm::DmaPool; diff --git a/kernel/src/drivers/pci.rs b/kernel/src/drivers/pci.rs index fd3bc681a00..1c591c22d6f 100644 --- a/kernel/src/drivers/pci.rs +++ b/kernel/src/drivers/pci.rs @@ -406,6 +406,7 @@ impl PciDevice { } } + #[cfg(feature = "boot-actuators")] pub fn is_id(&self, vendor: u16, device: u16) -> bool { self.vendor_id() == vendor && self.device_id() == device } @@ -599,3 +600,106 @@ fn assigned_bars(pci: &PciDevice) -> alloc::string::String { } out } + +/// Drive the real capability walk over config space no device produces: a +/// cyclic or forbidden link, a head the specification forbids, and a +/// capability reached before a link that ends the walk early. +#[cfg(feature = "boot-actuators")] +pub fn cap_selftest() { + use crate::mm::{pmm, Mmio}; + use alloc::vec::Vec; + + const CASES: usize = 6; + const SPLITS: usize = 13; + /// PCI 3.0 §6.7: a vendor-specific capability, which the walk yields like any other. + const VENDOR: u8 = 0x09; + // No device is told it: it stands in for config space. + let frame = pmm::alloc_frame().expect("pci-cap-selftest: no frame to stand in for config space"); + // SAFETY: `frame` owns this page until it drops at the end of this + // function, and no `Mmio` built over it escapes. + let cfg = unsafe { Mmio::over_phys(frame.direct_map(), crate::mm::PAGE_SIZE) }; + let device = PciDevice::over_config(cfg); + + let mut passed = 0usize; + let mut split_passed = 0usize; + let mut walk_case = |name: &str, + head: u8, + links: &[(u64, u8, u8)], + want: &[u64], + early: bool, + reached: Option| { + for off in 0..0x100u64 { + cfg.write_u8(off, 0); + } + cfg.write_u8(CAPABILITIES_PTR, head); + for &(at, id, next) in links { + cfg.write_u8(at, id); + cfg.write_u8(at + 1, next); + } + // Bounded, so a walk that fails to terminate is a wrong length here, not a hang. + let got: Vec = device.capabilities().take(64).map(|c| c.offset()).collect(); + if got.as_slice() == want { + passed += 1; + } else { + log!("pci cap selftest FAILED on {name}: yielded {got:?}, want {want:?}"); + } + // The split a claimed function's arming turns on: only a walk that + // reached the list's terminator may answer a capability it did not + // yield as absent rather than as never read. + let classify = |id: u8| match device.capability(id) { + Ok(_) => "found", + Err(NoCapability::Absent) => "absent", + Err(NoCapability::Truncated) => "truncated", + }; + let want_split = if early { "truncated" } else { "absent" }; + let split = classify(msix::CAP_ID); + if split == want_split { + split_passed += 1; + } else { + log!("pci cap selftest FAILED on {name}: a capability the walk never \ + reaches reads {split}, want {want_split}"); + } + // An early end is a fact about the rest of the list, never about what + // was read. + if let Some(id) = reached { + let found = classify(id); + if found == "found" { + split_passed += 1; + } else { + log!("pci cap selftest FAILED on {name}: a capability it publishes \ + before the link that ends the walk reads {found}, want found"); + } + } + // No layout here publishes an MSI-X capability the walk reaches, so + // this returns at the capability lookup and touches no MMIO. + let armed = match device.msix_entry() { + Ok(_) => "found", + Err(NoEntry::NoTable(NoCapability::Absent)) => "absent", + Err(NoEntry::NoTable(NoCapability::Truncated)) => "truncated", + Err(NoEntry::Unusable) => "unusable", + }; + if armed == want_split { + split_passed += 1; + } else { + log!("pci cap selftest FAILED on {name}: the arming reads {armed}, \ + want {want_split}"); + } + }; + + let chain: &[(u64, u8, u8)] = &[(0x40, VENDOR, 0x50), (0x50, VENDOR, 0)]; + walk_case("a well-formed chain", 0x40, chain, &[0x40, 0x50], false, None); + walk_case("a list that cycles on itself", 0x40, &[(0x40, VENDOR, 0x40)], &[0x40], true, None); + walk_case("a link that is not dword-aligned", 0x40, &[(0x40, VENDOR, 0x43)], &[0x40], true, None); + walk_case("a link below the standard header", 0x40, &[(0x40, VENDOR, 0x10)], &[0x40], true, None); + walk_case("a head the spec forbids", 0x41, &[], &[], true, None); + walk_case( + "a capability reached before a link the spec forbids", + 0x40, + &[(0x40, msi::CAP_ID, 0x43), (0x44, msix::CAP_ID, 0)], + &[0x40], + true, + Some(msi::CAP_ID), + ); + log!("pci cap split {split_passed}/{SPLITS}"); + log!("pci cap selftest {passed}/{CASES}"); +} diff --git a/kernel/src/drivers/serial.rs b/kernel/src/drivers/serial.rs index 988084d2dcc..c00c670db6f 100644 --- a/kernel/src/drivers/serial.rs +++ b/kernel/src/drivers/serial.rs @@ -1,13 +1,10 @@ -//! The console UART and the virtio-console, and the two locks that serialise -//! them. Where the UART is and what it is are the architecture's -//! (`arch::console_uart`). +//! The console UART and the two locks that serialise it. Where the UART is +//! and what it is are the architecture's (`arch::console_uart`). //! //! [`BackendGuard`] is the registers' lock, held with interrupts off for one -//! burst: what the UART's transmitter takes at once; the publish of a -//! transmit buffer to virtio-console, and each look for its completion, which -//! the host takes at its own pace; a byte read. The panic path takes the -//! registers alone, as a [`PanicUart`], and bypasses them once they stay held. -//! Nothing that holds a kernel lock formats here. +//! burst: what the UART's transmitter takes at once, or a byte read. The +//! panic path takes the registers alone, as a [`PanicUart`], and bypasses them +//! once they stay held. Nothing that holds a kernel lock formats here. use core::cell::Cell; use core::sync::atomic::{AtomicBool, Ordering}; @@ -25,50 +22,20 @@ use crate::arch::console_uart as uart; static UART_PRESENT: AtomicBool = AtomicBool::new(false); /// Find and program the console UART, off the firmware tables at `rsdp_addr` -/// where the architecture places it by them. +/// where the architecture places it by them, and put the boot so far on it. pub fn init(rsdp_addr: u64) { // What the architecture logs under the lock is not drained under it: // `has_console` is false until the store below. let present = uart::init(&mut BackendGuard::lock().registers, rsdp_addr); UART_PRESENT.store(present, Ordering::Relaxed); - console_changed(); -} - -/// A backend arrived or improved. Forwards to `log::console`, which owns the replay argument. -pub fn console_changed() { - crate::log::console::backend_changed(); -} - -pub fn uart_present() -> bool { - UART_PRESENT.load(Ordering::Relaxed) + crate::log::console::drain_inline(); } /// Whether anything can carry a byte off this machine; the same check `panic_flush` refuses on. pub fn has_console() -> bool { - !matches!(backend(), Backend::None) -} - -/// Which channel a write goes to right now; virtio-console is preferred over the UART. -#[derive(Clone, Copy, PartialEq, Eq)] -#[repr(u8)] -pub enum Backend { - /// Nothing can carry a byte off this machine. - None = 0, - Uart = 1, - Virtio = 2, -} - -pub fn backend() -> Backend { - if super::virtio_console::is_ready() { - Backend::Virtio - } else if uart_present() { - Backend::Uart - } else { - Backend::None - } + UART_PRESENT.load(Ordering::Relaxed) } - static BACKEND: BackendLock = BackendLock::new(); /// The console UART's registers, asked for: what every `arch::console_uart` @@ -93,29 +60,15 @@ impl BackendGuard { /// Writes raw bytes with no escape stripping. pub fn write_raw(&mut self, bytes: &[u8]) { - match backend() { - Backend::Virtio => super::virtio_console::write_bytes_locked(self, bytes), - Backend::Uart => uart_write_bytes(&mut self.registers, bytes), - Backend::None => {} - } + uart_write_bytes(&mut self.registers, bytes); } pub fn has_data(&mut self) -> bool { - if super::virtio_console::is_ready() { - super::virtio_console::has_data_locked(self) - } else { - uart_present() && uart::rx_ready(&mut self.registers) - } + has_console() && uart::rx_ready(&mut self.registers) } pub fn try_read_byte(&mut self) -> Option { - if super::virtio_console::is_ready() { - super::virtio_console::try_read_byte_locked(self) - } else if uart_present() && uart::rx_ready(&mut self.registers) { - Some(uart::read_byte(&mut self.registers)) - } else { - None - } + self.has_data().then(|| uart::read_byte(&mut self.registers)) } } @@ -169,6 +122,7 @@ const DRAINED_RAW: &[u8] = /// The registers for a fatal path: waited for while another holder may still /// let them go, and had at once where this CPU's fatal path holds them. pub fn panic_registers() -> PanicUart { + crate::log::console::on_the_uart_for_good(); let irq = IrqGuard::close(); let seized = Cell::new(None); // Uncalibrated, this asks once: only the boot CPU runs, so a holder is beneath this frame and never lets go. @@ -197,7 +151,7 @@ impl PanicUart { } } - /// Straight to the UART, never virtio-console: no allocation, bounded per byte. + /// Straight to the UART: no allocation, bounded per byte. pub fn write(&mut self, bytes: &[u8]) { uart_write_bytes(self.registers(), bytes); } @@ -235,10 +189,8 @@ impl PanicUart { /// Flushes pending logs on the panic path. /// -/// Waits for a live guard holder to release before bypassing it — bypassing -/// immediately would race its live ring/virtqueue mutation — and only -/// bypasses a holder that never releases, or its own CPU's fatal path, which -/// may have stopped inside a virtqueue publish. +/// Waits for a live guard holder to release before bypassing it, and only +/// bypasses a holder that never releases, or its own CPU's fatal path. /// /// # Safety /// Panic context only: the bypass reads the drain position with no lock held. @@ -256,12 +208,6 @@ pub unsafe fn panic_flush() { if matches!(uart.0, Hold::Reentered(_)) { uart.write(DRAINED_RAW); } - // Disables virtio-console first: a half-submitted TX queue would panic - // recursively if a bypassing write reached it. - if !uart_present() { - return; - } - super::virtio_console::disable(); // SAFETY: the registers are a wedged holder's or a fatal path's beneath // this one, neither of which runs again to publish, so reading the // position unlocked is safe. @@ -320,24 +266,13 @@ pub fn wire_is_mine() -> bool { /// Whether the UART has refused a write, which is said once. static UART_REFUSED: AtomicBool = AtomicBool::new(false); -/// `bytes` onto the wire the caller holds, in bursts: what the UART's -/// transmitter takes at once, one transmit buffer's to virtio-console, with interrupts on -/// between every two looks at the device. +/// `bytes` onto the wire the caller holds, in bursts of what the UART's +/// transmitter takes at once: each burst waits for the transmitter with +/// interrupts on, and takes the register lock only to ask it and fill it. pub fn write_wire(_wire: &SleepGuard<'_, ()>, bytes: &[u8]) { - match backend() { - Backend::Virtio => { - for chunk in bytes.chunks(super::virtio_console::TX_BUF_SIZE) { - super::virtio_console::write_burst(chunk); - } - } - Backend::Uart => uart_write_fifo(bytes), - Backend::None => {} + if !has_console() { + return; } -} - -/// The burst writer: each burst waits for the transmitter with interrupts on, -/// and takes the register lock only to ask it and fill it. -fn uart_write_fifo(bytes: &[u8]) { for chunk in bytes.chunks(uart::TX_BURST) { let mut asked = 0; loop { @@ -476,7 +411,7 @@ pub const MAX_CONSOLE_LINE: usize = 1024; const THRE_SPIN_LIMIT: u32 = 100_000; fn uart_write_bytes(registers: &mut Registers, bytes: &[u8]) { - if !uart_present() { + if !has_console() { return; } for &b in bytes { diff --git a/kernel/src/drivers/virtio.rs b/kernel/src/drivers/virtio.rs deleted file mode 100644 index 386b8456357..00000000000 --- a/kernel/src/drivers/virtio.rs +++ /dev/null @@ -1,850 +0,0 @@ -use crate::arch::barrier; - -use crate::mm::Mmio; -use super::pci::PciDevice; -use crate::mm::policy::MmioPolicy; -use crate::log; - -const VIRTIO_PCI_CAP_COMMON_CFG: u8 = 1; -const VIRTIO_PCI_CAP_NOTIFY_CFG: u8 = 2; -const VIRTIO_PCI_CAP_ISR_CFG: u8 = 3; -const VIRTIO_PCI_CAP_DEVICE_CFG: u8 = 4; - -const PCI_CAP_ID_VENDOR: u8 = 0x09; - -/// Ceiling on how much of a BAR the kernel maps: the window's page tables are -/// real memory, so a device advertising more gets this much mapped and a named -/// refusal for any capability past it. -const BAR_WINDOW_CEILING: u64 = 16 << 20; - -const STATUS_ACKNOWLEDGE: u8 = 1; -const STATUS_DRIVER: u8 = 2; -const STATUS_DRIVER_OK: u8 = 4; -const STATUS_FEATURES_OK: u8 = 8; - -pub const VIRTIO_F_VERSION_1: u64 = 1 << 32; - -/// Virtio 1.2 §6: the device's buffer addresses are the platform's, so what -/// translates any other function's translates this one's. A function never -/// offered it is one no unit sees, so every driver accepts it and the negotiated -/// set is logged for a gate to read back. -pub const VIRTIO_F_ACCESS_PLATFORM: u64 = 1 << 33; - -pub const COMMON_DEVICE_FEATURE_SELECT: u64 = 0x00; -pub const COMMON_DEVICE_FEATURE: u64 = 0x04; -pub const COMMON_DRIVER_FEATURE_SELECT: u64 = 0x08; -pub const COMMON_DRIVER_FEATURE: u64 = 0x0C; -pub const COMMON_DEVICE_STATUS: u64 = 0x14; -const COMMON_QUEUE_SELECT: u64 = 0x16; -pub const COMMON_QUEUE_SIZE: u64 = 0x18; -pub const COMMON_QUEUE_ENABLE: u64 = 0x1C; -pub const COMMON_QUEUE_NOTIFY_OFF: u64 = 0x1E; -pub const COMMON_QUEUE_DESC: u64 = 0x20; -pub const COMMON_QUEUE_DRIVER: u64 = 0x28; -pub const COMMON_QUEUE_DEVICE: u64 = 0x30; - -const VIRTQ_DESC_F_NEXT: u16 = 1; -const VIRTQ_DESC_F_WRITE: u16 = 2; - -#[repr(C)] -#[derive(Clone, Copy)] -struct VirtqDesc { - addr: u64, - len: u32, - flags: u16, - next: u16, -} - -/// One descriptor, for a caller sizing a table it allocates itself. -pub const DESC_BYTES: usize = core::mem::size_of::(); - -/// Byte size of a split virtqueue's available ring for `queue_size` entries (virtio 1.2 §2.7.6). -/// Excludes `used_event`, which belongs to `VIRTIO_F_EVENT_IDX`, never negotiated by this kernel. -pub const fn avail_bytes(queue_size: u16) -> usize { - AVAIL_RING_OFF + queue_size as usize * 2 -} - -/// Byte size of a split virtqueue's used ring for `queue_size` entries (virtio 1.2 §2.7.8). -pub const fn used_bytes(queue_size: u16) -> usize { - USED_RING_OFF + queue_size as usize * USED_ELEM_SIZE -} - -// Avail ring layout: flags(u16) + idx(u16) + ring[size](u16 each) -const AVAIL_IDX_OFF: usize = 2; -const AVAIL_RING_OFF: usize = 4; - -// Used ring layout: flags(u16) + idx(u16) + ring[size](id:u32 + len:u32 each) -const USED_IDX_OFF: usize = 2; -const USED_RING_OFF: usize = 4; -const USED_ELEM_SIZE: usize = 8; // id(u32) + len(u32) - -/// Parsed VirtIO PCI capability locations. -struct VirtioPciConfig { - common: Mmio, - notify: Mmio, - notify_off_multiplier: u32, - #[allow(dead_code)] // parsed from spec, used for interrupt-based operation - isr: Mmio, -} - -/// Why a virtio PCI capability names no config window, or the window it names. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -enum CapWindow { - Unmapped, - NoBar { index: u8 }, - PastBar { offset: u32, length: u32, window: u64 }, -} - -impl core::fmt::Display for CapWindow { - fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result { - match self { - Self::Unmapped => write!(f, "names a BAR no window was mapped for"), - Self::NoBar { index } => write!(f, "names BAR {index}, past the six a header has"), - Self::PastBar { offset, length, window } => { - write!(f, "names {length:#x} bytes at {offset:#x}, past its BAR's {window:#x}") - } - } - } -} - -/// The required capability a device's chain never produced a window for; its -/// chain crossed a trust boundary, so the device is refused, never asserted on. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub enum MissingCap { - Common, - Notify, - Isr, - Device, -} - -impl core::fmt::Display for MissingCap { - fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result { - let name = match self { - Self::Common => "COMMON_CFG", - Self::Notify => "NOTIFY_CFG", - Self::Isr => "ISR_CFG", - Self::Device => "DEVICE_CFG", - }; - write!(f, "published no usable {name} capability") - } -} - -/// Why [`VirtioDevice::offer`] or [`Offered::accept`] refused the device; -/// either way it keeps no DMA. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub enum InitRefusal { - /// Its chain never produced a window for a required capability. - Cap(MissingCap), - /// It never acknowledged its reset: `DEVICE_STATUS` stayed non-zero. - ResetUnanswered, - /// `FEATURES_OK` did not stick. Refused, not panicked: this is what a device - /// offered [`VIRTIO_F_ACCESS_PLATFORM`] answers a driver that declined it, - /// and a kernel that dies on a device's answer is one a device can kill. - FeaturesRefused { offered: u64, status: u32 }, -} - -impl From for InitRefusal { - fn from(missing: MissingCap) -> Self { - Self::Cap(missing) - } -} - -impl core::fmt::Display for InitRefusal { - fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result { - match self { - Self::Cap(missing) => missing.fmt(f), - Self::ResetUnanswered => { - write!(f, "did not zero DEVICE_STATUS for its reset within {RESET}") - } - Self::FeaturesRefused { offered, status } => write!( - f, - "refused the feature set {offered:#x} the driver accepted, leaving \ - DEVICE_STATUS={status:#x} without FEATURES_OK" - ), - } - } -} - -/// Virtio names no reset bound (1.2 §3.1.1 only orders the handshake); this long unanswered is dead. -const RESET: crate::time::Budget = crate::time::Budget::of( - crate::time::Duration::from_secs(2), - "the device is refused, never waited on", -); - -/// The config sub-window a virtio PCI capability names, or why it names none: -/// the device's index, offset and length checked before a subregion is taken. -fn cap_subwindow( - mapped_bars: &[Option; 6], - bar_byte: u8, - offset: u32, - length: u32, -) -> Result { - let slot = mapped_bars.get(usize::from(bar_byte)).ok_or(CapWindow::NoBar { index: bar_byte })?; - let bar = slot.as_ref().ok_or(CapWindow::Unmapped)?; - // The mapped window is the BAR's advertised size or the ceiling, never under 16 bytes. - let window = bar.size(); - let size = u64::from(length).max(4); - // size ≤ window keeps `window - size` from underflowing and offset + size inside the window. - if size > window || u64::from(offset) > window - size { - return Err(CapWindow::PastBar { offset, length, window }); - } - Ok(bar.subregion(u64::from(offset), size)) -} - -impl VirtioPciConfig { - fn parse(pci_dev: &PciDevice) -> Result { - let mut common = None; - let mut notify = None; - let mut notify_off_multiplier = 0u32; - let mut isr = None; - let mut device = None; - - let mut mapped_bars: [Option; 6] = [None, None, None, None, None, None]; - for cap in pci_dev.capabilities() { - if cap.id() != PCI_CAP_ID_VENDOR { continue; } - // The BAR index is the device's; one past the six the array holds skips the capability rather than indexing it. - let bar_idx = usize::from(cap.read_u8(4)); - if bar_idx >= mapped_bars.len() { - continue; - } - if mapped_bars[bar_idx].is_none() { - // A capability naming a non-memory or unsizable BAR maps to no window. - match pci_dev.memory_bar(bar_idx as u8) { - Ok(memory) => match pci_dev.bar_size(bar_idx as u8) { - Ok(advertised) => { - mapped_bars[bar_idx] = Some(crate::mm::paging::map_mmio( - memory.address(), - advertised.min(BAR_WINDOW_CEILING), - MmioPolicy::Uncacheable, - )); - } - Err(why) => log!( - "VirtIO: PCI {:02x}:{:02x}.{} BAR {bar_idx}: {why} — skipping \ - every capability in it", - pci_dev.bus, pci_dev.dev, pci_dev.func), - }, - Err(why) => log!( - "VirtIO: PCI {:02x}:{:02x}.{} names BAR {bar_idx} and {why} — skipping \ - every capability in it", - pci_dev.bus, pci_dev.dev, pci_dev.func), - } - } - } - - for cap in pci_dev.capabilities() { - if cap.id() != PCI_CAP_ID_VENDOR { - continue; - } - let cfg_type = cap.read_u8(3); - // Index, offset and length are the device's; a bad one skips this - // capability rather than indexing the array or asserting past the window. - let mmio = match cap_subwindow(&mapped_bars, cap.read_u8(4), cap.read_u32(8), cap.read_u32(12)) { - Ok(mmio) => mmio, - Err(CapWindow::Unmapped) => continue, - Err(why) => { - log!("VirtIO: PCI {:02x}:{:02x}.{} capability {why} — skipped", - pci_dev.bus, pci_dev.dev, pci_dev.func); - continue; - } - }; - - match cfg_type { - VIRTIO_PCI_CAP_COMMON_CFG if common.is_none() => common = Some(mmio), - VIRTIO_PCI_CAP_NOTIFY_CFG if notify.is_none() => { - notify = Some(mmio); - notify_off_multiplier = cap.read_u32(16); - } - VIRTIO_PCI_CAP_ISR_CFG if isr.is_none() => isr = Some(mmio), - VIRTIO_PCI_CAP_DEVICE_CFG if device.is_none() => device = Some(mmio), - _ => {} - } - } - - let config = Self { - common: common.ok_or(MissingCap::Common)?, - notify: notify.ok_or(MissingCap::Notify)?, - notify_off_multiplier, - isr: isr.ok_or(MissingCap::Isr)?, - }; - device.ok_or(MissingCap::Device)?; - Ok(config) - } -} - -use crate::mm::Dma; - -/// Split virtqueue rings, reached only through [`Dma`]'s volatile, bounds-checked reads and writes. -pub struct VirtqueueRegions<'pool> { - pub desc: Dma<'pool>, - pub avail: Dma<'pool>, - pub used: Dma<'pool>, -} - -impl<'pool> VirtqueueRegions<'pool> { - /// Compute regions from a single contiguous DMA buffer. - pub fn from_contiguous(buf: Dma<'pool>, queue_size: u16) -> Self { - let desc_size = queue_size as usize * DESC_BYTES; - let avail_size = avail_bytes(queue_size); - let used_size = used_bytes(queue_size); - let avail_off = (desc_size + 1) & !1; - let used_off = (avail_off + avail_size + 3) & !3; - Self { - desc: buf.subview(0, desc_size), - avail: buf.subview(avail_off, avail_size), - used: buf.subview(used_off, used_size), - } - } -} - -/// Proof a descriptor slot is available for submission; `id()` is always below the queue's size. -/// Non-Copy, non-Clone: `submit()` consumes it, so a spent slot cannot be reused for another chain. -pub struct DescSlot(u16); - -impl DescSlot { - pub fn id(&self) -> u16 { self.0 } -} - - -/// A VirtIO split virtqueue. -pub struct Virtqueue<'pool> { - desc: Dma<'pool>, - avail: Dma<'pool>, - used: Dma<'pool>, - size: u16, - last_used_idx: u16, - notify_offset: u16, - /// Bytes each chain's descriptor was given; the one bound a device-reported `len` is compared against. 0 means no chain. - chain_bytes: alloc::vec::Vec, -} - -/// Direction of a buffer in a descriptor chain. -pub enum BufDir { - Readable, - Writable, -} - -/// How many bytes a chain gives the device to write into. -/// Saturating, not wrapping: a wrap would shrink the bound below what was actually posted. -fn chain_bytes(bufs: &[(u64, u32, BufDir)]) -> u32 { - bufs.iter().fold(0u32, |sum, (_, len, _)| sum.saturating_add(*len)) -} - -impl<'pool> Virtqueue<'pool> { - /// Create a new virtqueue from a contiguous DMA region. - /// Zeroes the whole buffer, including the padding, so the caller gets a page it can reason about. - pub fn new(buf: Dma<'pool>, queue_size: u16) -> Self { - buf.zero(); - Self::from_regions(&VirtqueueRegions::from_contiguous(buf, queue_size), queue_size) - } - - /// Create a new virtqueue from explicit DMA regions. - pub fn from_regions(regions: &VirtqueueRegions<'pool>, queue_size: u16) -> Self { - let (desc, avail, used) = (regions.desc, regions.avail, regions.used); - desc.zero(); - avail.zero(); - used.zero(); - Self { - desc, - avail, - used, - size: queue_size, - last_used_idx: 0, - notify_offset: 0, - chain_bytes: alloc::vec![0u32; queue_size as usize], - } - } - - /// What the device is programmed with for each ring. - pub fn descs_addr(&self) -> u64 { self.desc.device_addr() } - pub fn avail_addr(&self) -> u64 { self.avail.device_addr() } - pub fn used_addr(&self) -> u64 { self.used.device_addr() } - - /// Where in the notification region this queue's doorbell sits; meaningless before `setup_queue` runs. - pub fn notify_bytes(&self, multiplier: u32) -> u64 { - self.notify_offset as u64 * multiplier as u64 - } - - /// Where the used element at ring position `i` sits. - fn used_elem_at(&self, i: u16) -> usize { - USED_RING_OFF + i as usize * USED_ELEM_SIZE - } - - /// The two fields of a used element, as the device wrote them: [`Self::parse_used`] is their only reader. - fn used_ring_id(&self, i: u16) -> u32 { - self.used.read(self.used_elem_at(i)) - } - fn used_ring_len(&self, i: u16) -> u32 { - self.used.read(self.used_elem_at(i) + 4) - } - - /// The initial pool of descriptor slots; call once after construction. - pub fn initial_slots(&self) -> alloc::vec::Vec { - (0..self.size).map(DescSlot).collect() - } - - /// Submit a descriptor chain and notify the device (non-blocking); consumes the proving `DescSlot`. - pub fn submit( - &mut self, - slot: DescSlot, - bufs: &[(u64, u32, BufDir)], - notify_mmio: Mmio, - notify_multiplier: u32, - queue_index: u16, - ) -> u16 { - let first_desc = slot.0; - self.publish(slot, bufs, notify_mmio, notify_multiplier, queue_index).ring(); - first_desc - } - - /// [`submit`](Self::submit) with the doorbell handed back unrung, for a - /// caller that rings it after letting go of a lock: an emulated device - /// can serve the doorbell's write synchronously, and that is not work to - /// hold a lock across. - pub fn publish( - &mut self, - slot: DescSlot, - bufs: &[(u64, u32, BufDir)], - notify_mmio: Mmio, - notify_multiplier: u32, - queue_index: u16, - ) -> Doorbell { - let size = self.size; - let first_desc = slot.0; - self.chain_bytes[first_desc as usize] = chain_bytes(bufs); - for (i, (addr, len, dir)) in bufs.iter().enumerate() { - let desc_idx = (first_desc + i as u16) % size; - let is_last = i == bufs.len() - 1; - let next_idx = (desc_idx + 1) % size; - - let mut flags: u16 = match dir { - BufDir::Readable => 0, - BufDir::Writable => VIRTQ_DESC_F_WRITE, - }; - if !is_last { - flags |= VIRTQ_DESC_F_NEXT; - } - - let desc = VirtqDesc { addr: *addr, len: *len, flags, next: next_idx }; - self.desc.write(desc_idx as usize * core::mem::size_of::(), desc); - } - - let avail_idx: u16 = self.avail.read(AVAIL_IDX_OFF); - self.avail.write(AVAIL_RING_OFF + (avail_idx % size) as usize * 2, first_desc); - // The descriptors and the ring entry before the idx that publishes them - // (virtio 1.2 §2.7.13). - barrier::dma_wmb(); - self.avail.write(AVAIL_IDX_OFF, avail_idx.wrapping_add(1)); - - self.doorbell(notify_mmio, notify_multiplier, queue_index) - } - - /// This queue's doorbell, for a caller that finds a chain published and - /// cannot know it was rung: a second ring costs the device one look. - pub fn doorbell(&self, notify_mmio: Mmio, notify_multiplier: u32, queue_index: u16) -> Doorbell { - Doorbell { mmio: notify_mmio, at: self.notify_bytes(notify_multiplier), queue_index } - } - - /// Check if the device has completed any request. - pub fn has_used(&self) -> bool { - let used_idx: u16 = self.used.read(USED_IDX_OFF); - used_idx != self.last_used_idx - } - - /// Non-blocking poll of the used ring: `(DescSlot, written_len)` on completion, `None` if nothing new. - /// A refused element is skipped, never returned, so one forged element cannot hide the ones behind it. - /// Never logs: the caller may hold `serial::BackendGuard`, the lock the log backend itself takes. - pub fn poll_used(&mut self) -> Option<(DescSlot, u32)> { - loop { - let used_idx: u16 = self.used.read(USED_IDX_OFF); - if used_idx == self.last_used_idx { - return None; - } - // The device writes the element before it bumps the idx (virtio 1.2 - // §2.7.8), so the element is read after the idx that counts it. - barrier::dma_rmb(); - let slot = self.last_used_idx % self.size; - let id = self.used_ring_id(slot); - let len = self.used_ring_len(slot); - self.last_used_idx = self.last_used_idx.wrapping_add(1); - // Forfeit rather than recovered: losing a token costs throughput, believing a bad one costs memory. - if let Some(elem) = self.parse_used(id, len) { - return Some(elem); - } - } - } - - /// What a used-ring element must satisfy: a head inside this queue's table, at a published - /// chain, written no further than that chain. Refused rather than clamped: there is nothing - /// here to recover from a forged completion. - fn parse_used(&self, id: u32, len: u32) -> Option<(DescSlot, u32)> { - // `chain_bytes` is exactly `size` long: the descriptor table's own bound, not a constant beside it. - let head = usize::try_from(id).ok().filter(|&head| head < self.chain_bytes.len())?; - let chain = self.chain_bytes[head]; - if chain == 0 || len > chain { - return None; - } - // Exact: `head < size`, a `u16`. - Some((DescSlot(head as u16), len)) - } - - /// Submit a descriptor chain and block until the device completes it, returning the recovered `DescSlot`. - pub fn submit_and_wait( - &mut self, - slot: DescSlot, - bufs: &[(u64, u32, BufDir)], - notify_mmio: Mmio, - notify_multiplier: u32, - queue_index: u16, - ) -> DescSlot { - self.submit(slot, bufs, notify_mmio, notify_multiplier, queue_index); - wait_used(queue_index, || self.poll_used().map(|(slot, _)| slot)) - } -} - -/// A queue's notification, owed for a chain [`Virtqueue::publish`] made -/// visible and rung once. -#[must_use = "a published chain the device is never told of is never taken"] -pub struct Doorbell { - mmio: Mmio, - at: u64, - queue_index: u16, -} - -impl Doorbell { - pub fn ring(self) { - // The idx before the notification: the notify is an `Mmio` write, which - // orders every earlier store before it. - self.mmio.write_u16(self.at, self.queue_index); - } -} - -/// A completion this late from an emulated device never comes; the panic -/// names the queue where an unbounded spin hung silently. -const ANSWERS: crate::time::Tripwire = crate::time::Tripwire::absurd( - crate::time::Duration::from_secs(5), - "far above any completion a live device delivers", -); - -/// Looks between two readings of the clock: `nanos_since_boot`'s 128-bit -/// divide is too costly per look. -const LOOKS_PER_CHECK: u32 = 1024; - -/// Spin on `look` until it finds a completion, bounded by [`ANSWERS`]: the one -/// wait on a used ring. A caller that has to lock around each look does so -/// inside `look`, so what it holds is held for one look at a time. -pub fn wait_used(queue_index: u16, look: impl FnMut() -> Option) -> T { - let at = crate::clock::nanos_since_boot().saturating_add(ANSWERS.nanos()); - wait_until(at, crate::clock::nanos_since_boot, look) - .unwrap_or_else(|| panic!("virtio: queue {queue_index} completed nothing in {ANSWERS}")) -} - -/// [`wait_used`] on the clock `now`: `None` is a device that completed nothing -/// by `at`. **Past `at`, `look` is asked once more**: the waiter may have been -/// off its CPU for the whole bound, and a device that finished meanwhile is -/// not the one that failed. -fn wait_until(at: u64, mut now: impl FnMut() -> u64, mut look: impl FnMut() -> Option) -> Option { - let mut looks = 0u32; - loop { - if let Some(done) = look() { - return Some(done); - } - core::hint::spin_loop(); - looks += 1; - if looks == LOOKS_PER_CHECK { - looks = 0; - if now() >= at { - return look(); - } - } - } -} - -/// Drive the real walk, window check and parse over config space no device produces: a cyclic -/// or forbidden link, a BAR/offset/length past the window, a chain missing a required capability. -#[cfg(feature = "boot-actuators")] -pub fn cap_selftest() { - use super::pci::{NoCapability, NoEntry, PciDevice, CAPABILITIES_PTR}; - use super::DmaPool; - use crate::mm::{DirectMap, Mmio}; - use alloc::vec::Vec; - use toyos_pci::{msi, msix}; - - const CASES: usize = 15; - const SPLITS: usize = 13; - // Twice the 0x4000 the window bound used to guess, so a capability past the - // guess but inside the BAR's real size has a case to be accepted by. - const WINDOW: u64 = 0x8000; - // One buffer for both the config space and the BAR window, so a permitted subregion stays inside it. - // No device is told it: it stands in for config space. - let pool = DmaPool::alloc_in(WINDOW as usize, crate::iommu::DeviceSpace::Untranslated, crate::iommu::Limit::WIDEST) - .expect("pci-cap-selftest: a 64-bit untranslated pool is never refused"); - let phys = pool.view().host_phys(); - // SAFETY: the pool owns this page until it drops at end of function, and no Mmio built over it escapes. - let cfg = unsafe { Mmio::over_phys(DirectMap::from_phys(phys), WINDOW) }; - let device = PciDevice::over_config(cfg); - - let mut passed = 0usize; - let mut split_passed = 0usize; - let mut walk_case = |name: &str, - head: u8, - links: &[(u64, u8, u8)], - want: &[u64], - early: bool, - reached: Option| { - for off in 0..0x100u64 { - cfg.write_u8(off, 0); - } - cfg.write_u8(CAPABILITIES_PTR, head); - for &(at, id, next) in links { - cfg.write_u8(at, id); - cfg.write_u8(at + 1, next); - } - // Bounded, so a walk that fails to terminate is a wrong length here, not a hang. - let got: Vec = device.capabilities().take(64).map(|c| c.offset()).collect(); - if got.as_slice() == want { - passed += 1; - } else { - log!("virtio: pci cap selftest FAILED on {name}: yielded {got:?}, want {want:?}"); - } - // The split a claimed function's arming turns on: only a walk that - // reached the list's terminator may answer a capability it did not - // yield as absent rather than as never read. - let classify = |id: u8| match device.capability(id) { - Ok(_) => "found", - Err(NoCapability::Absent) => "absent", - Err(NoCapability::Truncated) => "truncated", - }; - let want_split = if early { "truncated" } else { "absent" }; - let split = classify(msix::CAP_ID); - if split == want_split { - split_passed += 1; - } else { - log!("virtio: pci cap selftest FAILED on {name}: a capability the walk never \ - reaches reads {split}, want {want_split}"); - } - // An early end is a fact about the rest of the list, never about what - // was read. - if let Some(id) = reached { - let found = classify(id); - if found == "found" { - split_passed += 1; - } else { - log!("virtio: pci cap selftest FAILED on {name}: a capability it publishes \ - before the link that ends the walk reads {found}, want found"); - } - } - // No layout here publishes an MSI-X capability the walk reaches, so - // this returns at the capability lookup and touches no MMIO. - let armed = match device.msix_entry() { - Ok(_) => "found", - Err(NoEntry::NoTable(NoCapability::Absent)) => "absent", - Err(NoEntry::NoTable(NoCapability::Truncated)) => "truncated", - Err(NoEntry::Unusable) => "unusable", - }; - if armed == want_split { - split_passed += 1; - } else { - log!("virtio: pci cap selftest FAILED on {name}: the arming reads {armed}, \ - want {want_split}"); - } - }; - - const ID: u8 = PCI_CAP_ID_VENDOR; - let chain: &[(u64, u8, u8)] = &[(0x40, ID, 0x50), (0x50, ID, 0)]; - walk_case("a well-formed chain", 0x40, chain, &[0x40, 0x50], false, None); - walk_case("a list that cycles on itself", 0x40, &[(0x40, ID, 0x40)], &[0x40], true, None); - walk_case("a link that is not dword-aligned", 0x40, &[(0x40, ID, 0x43)], &[0x40], true, None); - walk_case("a link below the standard header", 0x40, &[(0x40, ID, 0x10)], &[0x40], true, None); - walk_case("a head the spec forbids", 0x41, &[], &[], true, None); - walk_case( - "a capability reached before a link the spec forbids", - 0x40, - &[(0x40, msi::CAP_ID, 0x43), (0x44, msix::CAP_ID, 0)], - &[0x40], - true, - Some(msi::CAP_ID), - ); - log!("virtio: pci cap split {split_passed}/{SPLITS}"); - - let mut bars: [Option; 6] = [None, None, None, None, None, None]; - bars[0] = Some(cfg); - let mut win_case = |name: &str, bar: u8, offset: u32, length: u32, want_ok: bool| { - let got = cap_subwindow(&bars, bar, offset, length); - if got.is_ok() == want_ok { - passed += 1; - } else { - log!("virtio: pci cap selftest FAILED on {name}: got ok={}, wanted ok={want_ok}", got.is_ok()); - } - }; - win_case("a window inside the BAR", 0, 0x100, 0x100, true); - win_case("a window ending exactly at the BAR's edge", 0, WINDOW as u32 - 4, 4, true); - // Reds on the guessed 0x4000 constant this bound used to be. - win_case("a window past the old guess, inside the BAR's real size", 0, 0x4000, 0x100, true); - win_case("a BAR index past the six a header has", 7, 0, 4, false); - win_case("a BAR the device did not map", 1, 0x100, 0x100, false); - win_case("an offset that carries the window past the BAR", 0, WINDOW as u32 - 3, 0x100, false); - win_case("a length past the whole BAR window", 0, 0, WINDOW as u32 + 1, false); - - // A chain lacking a required window is refused by name — this was four `expect`s and a panic. - for off in 0..0x100u64 { - cfg.write_u8(off, 0); - } - match VirtioPciConfig::parse(&device) { - Err(MissingCap::Common) => passed += 1, - Err(why) => log!("virtio: pci cap selftest FAILED on a chain missing its required caps: {why}"), - Ok(_) => log!("virtio: pci cap selftest FAILED: an empty chain parsed"), - } - - // A refused device keeps no bus mastering: COMMAND arrives armed (bit 2, Bus - // Master Enable, PCI 3.0 §6.2.2) and `offer`'s refusal must clear it. - cfg.write_u16(0x04, 0x0006); - let refused = VirtioDevice::offer(&device); - let command = cfg.read_u16(0x04); - match refused { - Err(InitRefusal::Cap(MissingCap::Common)) if command & 0x4 == 0 => passed += 1, - Err(InitRefusal::Cap(MissingCap::Common)) => log!( - "virtio: pci cap selftest FAILED: a refused device keeps bus mastering (COMMAND={command:#x})" - ), - Err(why) => log!("virtio: pci cap selftest FAILED on the refused device's chain: {why}"), - Ok(_) => log!("virtio: pci cap selftest FAILED: the refused device's empty chain parsed"), - } - - log!("virtio: pci cap selftest {passed}/{CASES}"); -} - -/// A fully initialized VirtIO device. -pub struct VirtioDevice { - config: VirtioPciConfig, -} - -/// A device reset and acknowledged, and the features it offers: virtio 1.2 -/// §3.1.1's first four steps, read before the function masters the bus. -/// -/// **Before bus mastering because [`PciDevice::enable_bus_master`] is what -/// records a function as this kernel's**: a driver that reads what a device -/// offers to decide whether to drive it has not yet driven it. -pub struct Offered { - config: VirtioPciConfig, - pub features: u64, -} - -impl Offered { - /// Leave the device to someone else: reset again, so nothing this driver - /// wrote outlives the read. - pub fn decline(self) { - self.config.common.write_u32(COMMON_DEVICE_STATUS, 0); - } - - /// The rest of §3.1.1 up to queue setup: the function masters the bus from - /// here, and negotiates `accepted_features` of what it offered. - pub fn accept(self, pci_dev: &PciDevice, accepted_features: u64) -> Result { - let Self { config, features: device_features } = self; - pci_dev.enable_bus_master(); - let common = config.common; - - let features = device_features & (accepted_features | VIRTIO_F_ACCESS_PLATFORM); - log!( - "VirtIO: PCI {:02x}:{:02x}.{} features device={device_features:#x} \ - negotiated={features:#x} access_platform={}", - pci_dev.bus, - pci_dev.dev, - pci_dev.func, - if features & VIRTIO_F_ACCESS_PLATFORM != 0 { 'y' } else { 'n' }, - ); - - common.write_u32(COMMON_DRIVER_FEATURE_SELECT, 0); - common.write_u32(COMMON_DRIVER_FEATURE, features as u32); - common.write_u32(COMMON_DRIVER_FEATURE_SELECT, 1); - common.write_u32(COMMON_DRIVER_FEATURE, (features >> 32) as u32); - - let status = STATUS_ACKNOWLEDGE as u32 | STATUS_DRIVER as u32 | STATUS_FEATURES_OK as u32; - common.write_u32(COMMON_DEVICE_STATUS, status); - - let answered = common.read_u32(COMMON_DEVICE_STATUS); - if answered & STATUS_FEATURES_OK as u32 == 0 { - pci_dev.disable_bus_master(); - return Err(InitRefusal::FeaturesRefused { offered: features, status: answered }); - } - - Ok(VirtioDevice { config }) - } -} - -impl VirtioDevice { - /// Reset `pci_dev` and read what it offers. `Err` says why the device was - /// refused — a capability its chain never published a usable window for, - /// or a reset it never acknowledged — and it keeps no bus mastering. - pub fn offer(pci_dev: &PciDevice) -> Result { - // Firmware may have armed bus mastering before the kernel ran, and a - // refused device keeps no DMA. - let config = match VirtioPciConfig::parse(pci_dev) { - Ok(config) => config, - Err(missing) => { - pci_dev.disable_bus_master(); - return Err(missing.into()); - } - }; - pci_dev.enable_memory_space(); - let common = config.common; - - // Order fixed by virtio 1.2 §3.1.1: reset, ACKNOWLEDGE, DRIVER, read the features. - common.write_u32(COMMON_DEVICE_STATUS, 0); - if !crate::clock::settles(RESET.nanos(), || common.read_u32(COMMON_DEVICE_STATUS) == 0) { - pci_dev.disable_bus_master(); - return Err(InitRefusal::ResetUnanswered); - } - - common.write_u32(COMMON_DEVICE_STATUS, STATUS_ACKNOWLEDGE as u32); - - common.write_u32(COMMON_DEVICE_STATUS, - STATUS_ACKNOWLEDGE as u32 | STATUS_DRIVER as u32); - - common.write_u32(COMMON_DEVICE_FEATURE_SELECT, 0); - let device_features_lo = common.read_u32(COMMON_DEVICE_FEATURE); - common.write_u32(COMMON_DEVICE_FEATURE_SELECT, 1); - let device_features_hi = common.read_u32(COMMON_DEVICE_FEATURE); - let features = (device_features_hi as u64) << 32 | device_features_lo as u64; - Ok(Offered { config, features }) - } - - /// Configure a virtqueue's addresses and size; does not enable it — call `enable_queue()` after MSI-X vectors are set. - pub fn setup_queue(&self, index: u16, queue: &mut Virtqueue<'_>) { - let common = self.config.common; - - common.write_u16(COMMON_QUEUE_SELECT, index); - - let max_size = common.read_u16(COMMON_QUEUE_SIZE); - assert!(max_size >= queue.size, "VirtIO: queue {} too small (max={}, need={})", index, max_size, queue.size); - common.write_u16(COMMON_QUEUE_SIZE, queue.size); - - common.write_u64(COMMON_QUEUE_DESC, queue.descs_addr()); - common.write_u64(COMMON_QUEUE_DRIVER, queue.avail_addr()); - common.write_u64(COMMON_QUEUE_DEVICE, queue.used_addr()); - - queue.notify_offset = common.read_u16(COMMON_QUEUE_NOTIFY_OFF); - } - - /// Enable a previously configured virtqueue. - pub fn enable_queue(&self, index: u16) { - let common = self.config.common; - common.write_u16(COMMON_QUEUE_SELECT, index); - common.write_u16(COMMON_QUEUE_ENABLE, 1); - } - - /// Set DRIVER_OK — device is now live. - pub fn activate(&self) { - let status = STATUS_ACKNOWLEDGE as u32 - | STATUS_DRIVER as u32 - | STATUS_FEATURES_OK as u32 - | STATUS_DRIVER_OK as u32; - self.config.common.write_u32(COMMON_DEVICE_STATUS, status); - } - - pub fn notify_mmio(&self) -> Mmio { - self.config.notify - } - - pub fn notify_off_multiplier(&self) -> u32 { - self.config.notify_off_multiplier - } -} diff --git a/kernel/src/drivers/virtio_console.rs b/kernel/src/drivers/virtio_console.rs deleted file mode 100644 index 29e603f96c3..00000000000 --- a/kernel/src/drivers/virtio_console.rs +++ /dev/null @@ -1,316 +0,0 @@ -//! VirtIO console: single-port (no MULTIPORT), replacing the 16550 UART as -//! the kernel log channel after init. Uses queues 0 (RX) and 1 (TX). -//! -//! **It drives the first console function that does not offer MULTIPORT** -//! (`toyos_pci::console`), read before any function masters the bus; a -//! multiport function is reset and left to a claim. -//! -//! RX is poll-driven: no UART IRQ handler is wired (see `arch/x86_64/idt/mod.rs`). - -use alloc::vec::Vec; -use core::cell::UnsafeCell; -use core::mem::MaybeUninit; -use core::sync::atomic::{AtomicBool, Ordering}; - -use super::pci::PciDevice; -use super::serial::BackendGuard; -use super::virtio::{wait_used, BufDir, DescSlot, Offered, Virtqueue, VirtioDevice, VIRTIO_F_ACCESS_PLATFORM, VIRTIO_F_VERSION_1}; -use super::DmaPool; -use crate::log; -use crate::mm::{Dma, Unaligned}; - -const VIRTIO_VENDOR: u16 = 0x1AF4; -const VIRTIO_CONSOLE_DEVICE: u16 = 0x1043; // 0x1040 + device_id 3 - -const QUEUE_SIZE: u16 = 16; -pub(crate) const TX_BUF_SIZE: usize = 4096; -const RX_BUF_SIZE: u32 = 256; -const RX_BUF_COUNT: usize = 8; - -const OFF_TX_BUF: usize = 0x0000; // 1 page (4KB) -const OFF_RX_BUFS: usize = 0x1000; // 8 × 256 = 2KB, fits in 1 page -const OFF_RXVQ: usize = 0x2000; // virtqueue desc/avail/used -const OFF_TXVQ: usize = 0x3000; -const DMA_SIZE: usize = 0x4000; - -struct RxPending { - buf_idx: usize, - slot: DescSlot, - len: u32, - pos: u32, -} - -// Every field must stay individually `Send` (no raw pointers), or this needs an explicit `unsafe impl Send`. -struct VConsole { - device: VirtioDevice, - rx: Virtqueue<'static>, - tx: Virtqueue<'static>, - tx_buf: Dma<'static>, - tx_slot: Option, - // Read only after the used ring returns the descriptor, so no read races a fill. - rx_bufs: [Dma<'static, Unaligned>; RX_BUF_COUNT], - /// Maps virtqueue desc id → rx_buf index (filled at refill, read at poll). - desc_to_rx: [u8; QUEUE_SIZE as usize], - // Draining RX buffer: slot recovered from the used ring, refilled once fully consumed. - rx_pending: Option, -} - -struct ConsoleCell(UnsafeCell>); -// SAFETY: sound because every access goes through `with_console`, gated by -// `READY` and serialized by the caller's `serial::BackendGuard`. -unsafe impl Sync for ConsoleCell {} - -// Written once in `init`; every reader/writer goes through `with_console`, -// gated by `READY` and serialized by the caller's `serial::BackendGuard`. -static CONSOLE: ConsoleCell = ConsoleCell(UnsafeCell::new(MaybeUninit::uninit())); -static READY: AtomicBool = AtomicBool::new(false); - -#[inline] -pub fn is_ready() -> bool { - READY.load(Ordering::Acquire) -} - -/// Disables the fast path; `is_ready()` then returns false and logging falls back to UART. -pub fn disable() { - READY.store(false, Ordering::Release); -} - -// Check-then-deref stays one function: split, a caller could deref without checking. -#[inline] -fn with_console(f: impl FnOnce(&mut VConsole) -> R) -> Option { - if !is_ready() { - return None; - } - // SAFETY: `READY`'s Acquire load (checked above) pairs with `init`'s Release - // store, proving initialization; the caller's `serial::BackendGuard` provides exclusion. - Some(f(unsafe { (*CONSOLE.0.get()).assume_init_mut() })) -} - -fn refill_rx(c: &mut VConsole, buf_idx: usize, slot: DescSlot) { - let desc_id = c.rx.submit( - slot, - &[(c.rx_bufs[buf_idx].device_addr(), RX_BUF_SIZE, BufDir::Writable)], - c.device.notify_mmio(), - c.device.notify_off_multiplier(), - 0, - ); - c.desc_to_rx[desc_id as usize] = buf_idx as u8; -} - -/// Synchronous: blocks until the host consumes each chunk. -pub fn write_bytes_locked(_: &mut BackendGuard, bytes: &[u8]) { - with_console(|c| { - let mut off = 0; - while off < bytes.len() { - let n = (bytes.len() - off).min(TX_BUF_SIZE); - // `n` is bounded to `TX_BUF_SIZE`, and the device is done with - // `tx_buf`: the slot is back, from the previous chunk or from the - // burst whose completion `slot` waits out. - let slot = tx_slot(c); - c.tx_buf.copy_from(0, &bytes[off..off + n]); - c.tx_slot = Some(c.tx.submit_and_wait( - slot, - &[(c.tx_buf.device_addr(), n as u32, BufDir::Readable)], - c.device.notify_mmio(), - c.device.notify_off_multiplier(), - 1, - )); - off += n; - } - }); -} - -/// The transmit slot, waiting out a burst [`write_burst`] left in flight when -/// the caller is the panic path that found it so — rung again first, since -/// the panic may have stopped its CPU between publishing it and ringing. -fn tx_slot(c: &mut VConsole) -> DescSlot { - match c.tx_slot.take() { - Some(slot) => slot, - None => { - c.tx.doorbell(c.device.notify_mmio(), c.device.notify_off_multiplier(), 1).ring(); - wait_used(1, || c.tx.poll_used().map(|(slot, _)| slot)) - } - } -} - -/// One transmit buffer's worth, with interrupts off only to publish it and to -/// look for its completion: the doorbell is rung with them on, because QEMU -/// runs the device's output into its chardev inside that write, and the -/// device takes the buffer at the host's pace, which a loaded host can -/// stretch to tens of milliseconds. The caller holds the wire, so no other -/// burst is in flight. -pub fn write_burst(bytes: &[u8]) { - assert!(bytes.len() <= TX_BUF_SIZE, "vconsole: a burst of {} bytes", bytes.len()); - let doorbell = { - let _burst = super::serial::BackendGuard::lock(); - with_console(|c| { - let slot = tx_slot(c); - c.tx_buf.copy_from(0, bytes); - c.tx.publish( - slot, - &[(c.tx_buf.device_addr(), bytes.len() as u32, BufDir::Readable)], - c.device.notify_mmio(), - c.device.notify_off_multiplier(), - 1, - ) - }) - }; - let Some(doorbell) = doorbell else { return }; - doorbell.ring(); - wait_used(1, || { - let _look = super::serial::BackendGuard::lock(); - with_console(|c| { - // The panic path may have waited this burst out already. - if c.tx_slot.is_none() { - c.tx_slot = c.tx.poll_used().map(|(slot, _)| slot); - } - c.tx_slot.is_some() - }) - .unwrap_or(true) - .then_some(()) - }); -} - -/// Read one byte from RX. -pub fn try_read_byte_locked(_: &mut BackendGuard) -> Option { - with_console(|c| { - if c.rx_pending.is_none() { - // `slot`/`len` are bounded by `poll_used`: id indexes `desc_to_rx` - // (length `QUEUE_SIZE`), len is at most `RX_BUF_SIZE` — unchecked, - // an over-long `len` would hand kernel memory to the console as input. - let (slot, len) = c.rx.poll_used()?; - let buf_idx = c.desc_to_rx[slot.id() as usize] as usize; - c.rx_pending = Some(RxPending { buf_idx, slot, len, pos: 0 }); - } - let p = c.rx_pending.as_mut().unwrap(); - // Bounded twice: `read` refuses `pos + 1 > RX_BUF_SIZE`, and `pos < len` - // is already bounded by `poll_used` above. - let byte: u8 = c.rx_bufs[p.buf_idx].read(p.pos as usize); - p.pos += 1; - if p.pos >= p.len { - let p = c.rx_pending.take().unwrap(); - refill_rx(c, p.buf_idx, p.slot); - } - Some(byte) - }) - .flatten() -} - -pub fn has_data_locked(_: &BackendGuard) -> bool { - with_console(|c| c.rx_pending.is_some() || c.rx.has_used()).unwrap_or(false) -} - -pub fn init(devices: &[PciDevice]) -> bool { - // Every console function's offer, in enumeration order: the one this - // driver takes is decided by what each offers (`toyos_pci::console`). - let mut offers: Vec<(PciDevice, Offered)> = Vec::new(); - for pci_dev in devices.iter().filter(|d| d.is_id(VIRTIO_VENDOR, VIRTIO_CONSOLE_DEVICE)) { - match VirtioDevice::offer(pci_dev) { - Ok(offered) => offers.push((*pci_dev, offered)), - Err(why) => log!("virtio-console: PCI {:02x}:{:02x}.{} {why} — device refused", - pci_dev.bus, pci_dev.dev, pci_dev.func), - } - } - let features: Vec = offers.iter().map(|(_, offered)| offered.features).collect(); - let chosen = toyos_pci::console(&features); - let mut taken = None; - for (at, (pci_dev, offered)) in offers.into_iter().enumerate() { - if Some(at) == chosen { - taken = Some((pci_dev, offered)); - continue; - } - log!( - "virtio-console: PCI {:02x}:{:02x}.{} offers MULTIPORT, which this single-port driver \ - does not drive — reset and left to a claim", - pci_dev.bus, pci_dev.dev, pci_dev.func - ); - offered.decline(); - } - let Some((pci_dev, offered)) = taken else { - log!("virtio-console: no device found"); - return false; - }; - log!("virtio-console: found at PCI {:02x}:{:02x}.{}", pci_dev.bus, pci_dev.dev, pci_dev.func); - - // An address space of this device's own, holding one pool and nothing - // else — unless it does not offer ACCESS_PLATFORM: then it uses physical - // addresses whatever translates the bus, and a driver MUST give it physical - // ones (virtio 1.2 §6.1); a domain's addresses would be other memory's. - let space = if offered.features & VIRTIO_F_ACCESS_PLATFORM != 0 { - crate::iommu::DeviceSpace::create(&[pci_dev]) - } else { - log!( - "virtio-console: PCI {:02x}:{:02x}.{} offers no ACCESS_PLATFORM, so it is told physical addresses", - pci_dev.bus, pci_dev.dev, pci_dev.func - ); - crate::iommu::DeviceSpace::Untranslated - }; - // Leaked, not held in a `static`: the console is never unbound. - // Virtio 1.x carries every buffer address as 64 bits (§2.7.5). - let dma = match DmaPool::alloc_in(DMA_SIZE, space, crate::iommu::Limit::WIDEST) { - Ok(pool) => pool.leak(), - Err(why) => { - log!("virtio-console: PCI {:02x}:{:02x}.{} {why} — device refused", pci_dev.bus, pci_dev.dev, pci_dev.func); - offered.decline(); - return false; - } - }; - // Before the device is told an address, and after the only mapping it gets. - space.attach(pci_dev.bus, pci_dev.dev, pci_dev.func); - - let device = match offered.accept(&pci_dev, VIRTIO_F_VERSION_1) { - Ok(device) => device, - Err(why) => { - log!("virtio-console: PCI {:02x}:{:02x}.{} {why} — device refused", - pci_dev.bus, pci_dev.dev, pci_dev.func); - return false; - } - }; - - let mut rx = Virtqueue::new(dma.subview(OFF_RXVQ, 0x1000), QUEUE_SIZE); - let mut tx = Virtqueue::new(dma.subview(OFF_TXVQ, 0x1000), QUEUE_SIZE); - - device.setup_queue(0, &mut rx); - device.setup_queue(1, &mut tx); - device.enable_queue(0); - device.enable_queue(1); - device.activate(); - - let tx_buf = dma.subview(OFF_TX_BUF, TX_BUF_SIZE); - - let rx_bufs: [Dma<'static, Unaligned>; RX_BUF_COUNT] = core::array::from_fn(|i| { - dma.subview(OFF_RX_BUFS + i * RX_BUF_SIZE as usize, RX_BUF_SIZE as usize).unaligned() - }); - - let mut tx_slots = tx.initial_slots(); - let tx_slot = tx_slots.pop().expect("vconsole: no tx slots"); - drop(tx_slots); - - let mut rx_slots = rx.initial_slots(); - - let mut console = VConsole { - device, rx, tx, - tx_buf, - tx_slot: Some(tx_slot), - rx_bufs, - desc_to_rx: [0; QUEUE_SIZE as usize], - rx_pending: None, - }; - - for i in 0..RX_BUF_COUNT { - let slot = rx_slots.pop().expect("vconsole: not enough rx slots"); - refill_rx(&mut console, i, slot); - } - drop(rx_slots); - - // SAFETY: single write, called once before `READY` is set and before any - // AP starts, so nothing is reading `CONSOLE` yet; `with_console` only reads - // after this function's Release store below. - unsafe { (*CONSOLE.0.get()).write(console); } - READY.store(true, Ordering::Release); - crate::drivers::serial::console_changed(); - - log!("virtio-console: initialized ({} RX bufs of {} bytes, TX buf {} bytes)", - RX_BUF_COUNT, RX_BUF_SIZE, TX_BUF_SIZE); - true -} diff --git a/kernel/src/iommu/mod.rs b/kernel/src/iommu/mod.rs index 96397140197..e1ca21a03ae 100644 --- a/kernel/src/iommu/mod.rs +++ b/kernel/src/iommu/mod.rs @@ -155,71 +155,6 @@ impl core::fmt::Display for IommuError { } } -/// Where a device's addresses come from. Not a bare `DomainId`: a machine with no -/// unit has to be something this type can say, or every driver grows the branch. -#[derive(Clone, Copy)] -pub enum DeviceSpace { - /// Nothing translates here, so a device address is a physical address. - Untranslated, - /// The device reaches exactly what is mapped in this and nothing else. - Own(DomainId), -} - -impl DeviceSpace { - /// One for whichever of `functions` is attached to it, or the machine's - /// own with the reason: the machine's own where any of them keeps it, so - /// the one attached reaches what its device addresses name. For a driver - /// **in this kernel**, whose addresses are the kernel's either way. - pub fn create(functions: &[crate::drivers::pci::PciDevice]) -> Self { - for function in functions { - let stream = StreamId::pci(function.bus, function.dev, function.func); - if let Some(why) = unit::domain::untranslated(stream) { - log!("iommu: {stream} keeps the machine's addresses: {why}"); - return Self::Untranslated; - } - } - match unit::domain::create(0) { - Ok((id, _)) => Self::Own(id), - Err(why) => { - log!("iommu: no domain of its own for a device: {why}"); - Self::Untranslated - } - } - } - - /// Put `bytes` at `phys` in this space, ending at or below `limit`, the - /// device's own address width; returns what to program the device with. - /// - /// Read and write both, always: nothing here can give a permission set a - /// second value. The only leaf is 2 MiB, coarser than any split a driver's - /// pools offer, and QEMU drops an access its cached translation denies - /// rather than recording a fault — unexpressible and unobservable both. - pub fn map(self, phys: u64, bytes: u64, limit: Limit) -> Result { - match self { - // The pages are the address, so nothing chooses one the device reaches. - Self::Untranslated if phys.saturating_add(bytes) > limit.end() => Err(IommuError::AddressesExhausted(limit)), - Self::Untranslated => Ok(phys), - Self::Own(id) => unit::domain::map(id, phys, bytes, limit).map(Iova::raw), - } - } - - /// Take `bytes` at `at` back, so the pages behind them can be reused. - pub fn unmap(self, at: u64, bytes: u64) -> Result<(), IommuError> { - match self { - Self::Untranslated => Ok(()), - Self::Own(id) => unit::domain::unmap(id, Iova::translated(at), bytes), - } - } - - /// Move `bus:device.function` onto this space, every mapping it needs already - /// in place: the device is translating the moment this returns. - pub fn attach(self, bus: u8, device: u8, function: u8) { - if let Self::Own(id) = self { - OwnSpace(id).attach(bus, device, function); - } - } -} - /// A device's own address space, for a function a *process* drives: it has no /// untranslated form, so nothing holding one can hand that process a physical /// address to write into a descriptor. @@ -234,7 +169,13 @@ impl OwnSpace { unit::domain::create(room).map(|(id, at)| (Self(id), at.raw())) } - /// [`DeviceSpace::map`]. + /// Put `bytes` at `phys` in this space, ending at or below `limit`, the + /// device's own address width; returns what to program the device with. + /// + /// Read and write both, always: nothing here can give a permission set a + /// second value. The only leaf is 2 MiB, and QEMU drops an access its + /// cached translation denies rather than recording a fault — + /// unexpressible and unobservable both. pub fn map(self, phys: u64, bytes: u64, limit: Limit) -> Result { unit::domain::map(self.0, phys, bytes, limit).map(Iova::raw) } @@ -252,12 +193,13 @@ impl OwnSpace { unit::domain::place(self.0, Iova::translated(at), phys, bytes) } - /// [`DeviceSpace::unmap`]. + /// Take `bytes` at `at` back, so the pages behind them can be reused. pub fn unmap(self, at: u64, bytes: u64) -> Result<(), IommuError> { unit::domain::unmap(self.0, Iova::translated(at), bytes) } - /// [`DeviceSpace::attach`]. + /// Move `bus:device.function` onto this space, every mapping it needs already + /// in place: the device is translating the moment this returns. pub fn attach(self, bus: u8, device: u8, function: u8) { unit::domain::attach(StreamId::pci(bus, device, function), self.0); } diff --git a/kernel/src/log/console.rs b/kernel/src/log/console.rs index 37dc93972e9..97525599ccb 100644 --- a/kernel/src/log/console.rs +++ b/kernel/src/log/console.rs @@ -16,8 +16,13 @@ //! record the stop commits reaches the wire before a CPU is taken down. A //! holder that keeps it through [`LET_GO`] has the stop write over it, through //! the registers, from a position the stop alone walks. +//! +//! **A console holder may carry the log elsewhere** ([`carried_by`]): `klogd` +//! says so on the wire and advances past what it would have written, until +//! the holder lets go or its console ends. A fatal path and the stop take the +//! log back to the UART for good ([`on_the_uart_for_good`]). -use core::sync::atomic::{AtomicPtr, AtomicU64, AtomicU8, Ordering}; +use core::sync::atomic::{AtomicPtr, AtomicU64, Ordering}; use alloc::sync::Arc; @@ -102,7 +107,7 @@ pub fn post_wake() { } /// Put every committed record this machine has not yet spoken on the wire, -/// where the wire is free: the boot before `klogd` runs, and a backend's +/// where the wire is free: the boot before `klogd` runs, and the UART's /// arrival. Declining loses nothing: the record stays committed, and whoever /// holds the wire drains it too. pub fn drain_inline() { @@ -192,6 +197,7 @@ impl StopWire { /// sync behind this is not spent behind a slow wire: the boot's last word /// drains it. pub fn take_for_the_stop(parkable: &Parkable) -> StopWire { + on_the_uart_for_good(); // Armed before the ask: a release posted between the ask and the park is // one the park returns on at once. let armed = watch::arm(&RELEASED, 0, WaitClass::Other).expect("console: the stop holds no task to park"); @@ -240,10 +246,12 @@ pub fn drain_locked(guard: &mut BackendGuard) { drain_ordered(&Shards, &mut &DRAINED, &mut sink); } -/// Advances the position with no backend: standing still, an armed waiter would find the same record on every rescan and spin. -/// Safe to advance: shards keep every record for the panel regardless, and a backend arriving later rewinds this position whole. -fn discard_pending() { - let mut sink = Discard; +/// Advances the position with no UART, or while a holder carries the log +/// (`carried`), until it no longer does: standing still, an armed waiter +/// would find the same record on every rescan and spin. Safe to advance: +/// shards keep every record for the panel and the log's reader regardless. +fn discard_pending(carried: bool) { + let mut sink = Discard { carried }; drain_ordered(&Shards, &mut &DRAINED, &mut sink); } @@ -263,19 +271,34 @@ pub unsafe fn drain_bypassed(uart: &mut PanicUart) { drain_ordered(&Shards, &mut &DRAINED, &mut sink); } -/// Which backend the drain has already spoken to, as `serial::Backend`'s discriminant. -static SPOKEN_TO: AtomicU8 = AtomicU8::new(serial::Backend::None as u8); +/// Who carries the log off this machine ([`kernel::carrier`]). +static CARRIER: kernel::carrier::Carrier = kernel::carrier::Carrier::new(); -/// A backend has appeared or changed. Rewind and drain the boot again into the current one. -/// Fires only on an actual change, and the wire targets one backend at a time, so the replay never duplicates onto the backend already spoken to. -pub fn backend_changed() { - let now = serial::backend() as u8; - if SPOKEN_TO.swap(now, Ordering::Relaxed) != now { - DRAINED.rewind(); - } - drain_inline(); - // post_wake: the rewind above moved the position backwards under a parked `klogd`, with no new commit to wake it. +/// What `klogd` puts on the wire as a holder takes the log off it, and as the +/// log comes back to it: the UART's account of where its lines went. +pub const CARRIED_ELSEWHERE: &str = + "console: the log is carried elsewhere from here; the UART takes the stop's and a fatal path's lines"; +pub const BACK_ON_THE_UART: &str = "console: the UART carries the log again"; + +/// The console holder `holder` carries the log from now, and `klogd` advances +/// past every record and queued line without putting it on the UART. +pub fn carried_by(holder: usize) -> Result<(), kernel::carrier::Refused> { + CARRIER.carry(holder)?; post_wake(); + Ok(()) +} + +/// `holder` no longer carries the log, by its word or by its console's end. +pub fn released_by(holder: usize) { + if CARRIER.release(holder) { + post_wake(); + } +} + +/// Every record from here goes on the UART, whoever carried the log: a fatal +/// path's, before its first record, and the stop's. +pub fn on_the_uart_for_good() { + CARRIER.for_good(); } /// Lines console holders queued for the wire. Fixed, because a console @@ -492,12 +515,16 @@ impl RecordSink for Registers<'_> { } } -/// Advances the position with no backend; no bytes are built. -struct Discard; +/// Advances the position; no bytes are built. Read after its record, so a +/// fatal path's records, which follow its [`on_the_uart_for_good`], are +/// never advanced past while a holder carried the log. +struct Discard { + carried: bool, +} impl RecordSink for Discard { fn put(&mut self, _record: &LogRecord) -> bool { - true + !self.carried || CARRIER.elsewhere() } } @@ -517,17 +544,34 @@ extern "C" fn body(_arg: u64) -> ! { let parkable = scheduler::Parkable::at_entry(); let handle = crate::sched::driver::current_handle().expect("klogd runs as a task"); + // Whether the wire last said the log is carried elsewhere. + let mut said_elsewhere = false; loop { let (freed, asked) = if serial::has_console() { // A chunk of each per hold, with interrupts on throughout. let wire = serial::wire(&parkable); #[cfg(feature = "boot-actuators")] staged::hold_if_staged(&parkable, &handle); - drain_records(&wire, CHUNK, Yield::ToTheStop); - let freed = drain_queue(&wire, CHUNK as usize, Yield::ToTheStop); - (freed, let_wire_go(wire)) + let elsewhere = CARRIER.elsewhere(); + if elsewhere != said_elsewhere { + said_elsewhere = elsewhere; + crate::log!("{}", if elsewhere { CARRIED_ELSEWHERE } else { BACK_ON_THE_UART }); + // The change's own line is on the wire before the UART goes + // quiet, so what the UART carried ends at it. + if elsewhere { + drain_records(&wire, u64::MAX, Yield::ToTheStop); + } + } + if elsewhere { + discard_pending(true); + (discard_queue(), let_wire_go(wire)) + } else { + drain_records(&wire, CHUNK, Yield::ToTheStop); + let freed = drain_queue(&wire, CHUNK as usize, Yield::ToTheStop); + (freed, let_wire_go(wire)) + } } else { - discard_pending(); + discard_pending(false); (discard_queue(), HANDOFF.asked()) }; if asked { @@ -554,7 +598,7 @@ extern "C" fn body(_arg: u64) -> ! { // on one watch at a time. `post_wake` notifies this thread's word // directly; nothing else posts this watch but the thread's own end. let armed = watch::arm(handle.watch(), 0, WaitClass::Other).expect("klogd runs as a task"); - // Safe with no backend because `discard_pending` still advances the position each pass. + // Safe with no UART, or with the log carried elsewhere, because `discard_pending` still advances the position each pass. if shard::arm_waiter(shard::log_waiter(), || { // The registration above forgot the staging's wake if it came // while this thread ran: the staging's own word is the recheck. diff --git a/kernel/src/log/read.rs b/kernel/src/log/read.rs index 467e91fa0b8..55b1f94a89b 100644 --- a/kernel/src/log/read.rs +++ b/kernel/src/log/read.rs @@ -153,15 +153,6 @@ impl Published { self.lost.store(other.lost(), Ordering::Relaxed); } - /// Resets the position to the oldest record every shard still holds. - pub fn rewind(&self) { - for ring in 0..self.count() { - self.word(ring).store(FIRST_SEQ, Ordering::Relaxed); - } - // Also zeroes `lost`: after a rewind it counts only future drops, not what a prior consumer missed. - self.lost.store(0, Ordering::Relaxed); - } - /// Is there a committed record this position has not taken? Lock-free. pub fn any_pending(&self) -> bool { any_committed(self) diff --git a/kernel/src/main.rs b/kernel/src/main.rs index a1c30c9a701..45bf2745d1e 100644 --- a/kernel/src/main.rs +++ b/kernel/src/main.rs @@ -99,13 +99,15 @@ mod late_panic { use alloc::boxed::Box; use arch::{cpu, percpu}; -use drivers::{acpi, gop, pci, serial, virtio_console}; +use drivers::{acpi, gop, pci, serial}; use toyos_abi::boot::{KernelArgs, MemoryMapEntry}; use toyos_rootimage::handoff::{held, Descriptor}; #[panic_handler] fn panic(info: &core::panic::PanicInfo) -> ! { cpu::disable_interrupts(); + // Before the first record of the report: no holder carries it off the UART. + log::console::on_the_uart_for_good(); // A handler's own panic reports through locks like any other. #[cfg(feature = "mask-windows")] windows::stand_down(); @@ -427,7 +429,7 @@ pub(crate) unsafe extern "C" fn kernel_main(loader_args: &mut KernelArgs) -> ! { pcidev::publish(&pci_devices, toyos_acpi::SEGMENT_GROUP, kernel_args.root_bridge_windows()); #[cfg(feature = "boot-actuators")] if actuator::pci_cap_selftest() { - drivers::virtio::cap_selftest(); + drivers::pci::cap_selftest(); } // After ACPI is readable and PCI is enumerable, before any driver `init`: each enumerated device needs a context entry before it can DMA. // Refuses nothing — a machine with no usable IOMMU boots exactly as one without it. @@ -471,8 +473,6 @@ pub(crate) unsafe extern "C" fn kernel_main(loader_args: &mut KernelArgs) -> ! { #[cfg(feature = "boot-actuators")] arch::boot::interrupt_selftests(); - virtio_console::init(&pci_devices); - if kernel_args.gop_framebuffer != 0 { log!("GPU: using UEFI GOP"); match gop::init( diff --git a/kernel/src/mm/dma.rs b/kernel/src/mm/dma.rs deleted file mode 100644 index 5fab9cfa77e..00000000000 --- a/kernel/src/mm/dma.rs +++ /dev/null @@ -1,225 +0,0 @@ -//! DMA memory, as a bounds-checked view rather than a raw pointer; [`DmaPool`] owns the pages, and [`Dma`] is the only safe way to touch them. -//! -//! A view carries two addresses because they are two different things: what a -//! *device* is programmed with, and where the bytes are. They are equal only -//! where nothing translates, so there is no `phys()` to reach for by accident. - -use alloc::vec::Vec; -use core::marker::PhantomData; -use core::ptr::{copy_nonoverlapping, read_unaligned, read_volatile, write_bytes, write_volatile}; - -use crate::iommu::{DeviceSpace, IommuError, Limit}; - -use super::pmm::PhysPage; -use super::DirectMap; - -mod sealed { - pub trait Sealed {} -} - -/// Sealed: exactly two disciplines exist, [`Volatile`] and [`Unaligned`]. -pub trait Discipline: sealed::Sealed {} - -/// The discipline for memory that races the device. -pub enum Volatile {} -/// The discipline for memory the protocol has fenced. -pub enum Unaligned {} - -impl sealed::Sealed for Volatile {} -impl sealed::Sealed for Unaligned {} -impl Discipline for Volatile {} -impl Discipline for Unaligned {} - -/// A bounds-checked, `Copy` view of DMA memory, scoped to its [`DmaPool`]'s lifetime. -pub struct Dma<'pool, D: Discipline = Volatile> { - base: *mut u8, - /// An address in the device's domain, or the physical one where nothing translates. - device: u64, - size: usize, - pool: PhantomData<&'pool DmaPool>, - how: PhantomData, -} - -impl Clone for Dma<'_, D> { - fn clone(&self) -> Self { - *self - } -} -impl Copy for Dma<'_, D> {} - -// SAFETY: `base` is a direct-mapped address, valid identically from any CPU; `Dma` is `Copy` with no lock, so sharing it shares only that address and length, never exclusive access to the memory it names. -unsafe impl Send for Dma<'_, D> {} -// SAFETY: see the `Send` impl above. -unsafe impl Sync for Dma<'_, D> {} - -impl<'pool, D: Discipline> Dma<'pool, D> { - #[inline] - fn new(base: *mut u8, device: u64, size: usize) -> Self { - Self { base, device, size, pool: PhantomData, how: PhantomData } - } - - /// The address to program a device with for the first byte; never a - /// physical address once a unit translates this pool's domain. - #[inline] - pub fn device_addr(self) -> u64 { - self.device - } - - /// Where the first byte is: what a page table maps, not what a descriptor - /// carries. The virtio parse selftest's alone, which lays a config space - /// over a pool. - #[cfg(feature = "boot-actuators")] - #[inline] - pub fn host_phys(self) -> u64 { - DirectMap::phys_of(self.base) - } - - /// The `size` bytes at `offset`, refused if not wholly inside `self`. - #[inline] - pub fn subview(self, offset: usize, size: usize) -> Self { - self.check(offset, size); - // SAFETY: `check` refused anything but `offset + size <= self.size`. - Self::new(unsafe { self.base.add(offset) }, self.device + offset as u64, size) - } - - /// Clear the whole view. - #[inline] - pub fn zero(self) { - // SAFETY: exactly `self.size` bytes from `self.base`, the region this view covers. - unsafe { write_bytes(self.base, 0, self.size) } - } - - /// Copy `src` into the view at `offset`. - #[inline] - pub fn copy_from(self, offset: usize, src: &[u8]) { - self.check(offset, src.len()); - // SAFETY: `check` bounded the destination for `src.len()` bytes; `src` and `self` cannot overlap since the heap is never allocated from DMA pages. - unsafe { copy_nonoverlapping(src.as_ptr(), self.base.add(offset), src.len()) } - } - - // Panics, not `Result`: every offset/length here is the driver's own arithmetic, never a device-chosen number (those are bounded at their decode before becoming an offset). - #[inline] - fn check(self, offset: usize, len: usize) { - if let Err(why) = toyos_dma::within(offset, len, self.size) { - refuse(why, self.base); - } - } -} - -// Out of line and cold: inlined, the panic keeps LLVM from inlining the volatile accessors themselves. -#[cold] -#[inline(never)] -fn refuse(why: toyos_dma::Refused, base: *mut u8) -> ! { - panic!("DMA: {why}, in the region at {base:p}"); -} - -impl<'pool> Dma<'pool, Volatile> { - /// Read the `T` at `offset` with a volatile load; bounded and aligned for `T`. - #[inline] - pub fn read(self, offset: usize) -> T { - // SAFETY: `at` bounded and aligned the pointer for `size_of::()`; volatile because the device may write concurrently, so the load may not be elided or reordered. - unsafe { read_volatile(self.at::(offset) as *const T) } - } - - /// Write `value` to the `T` at `offset` with a volatile store. - #[inline] - pub fn write(self, offset: usize, value: T) { - // SAFETY: `at` bounded and aligned the pointer for `size_of::()`; volatile because the device may read concurrently, so the store may not be elided or reordered. - unsafe { write_volatile(self.at::(offset), value) } - } - - /// Switches to the unaligned discipline; one-way, since nothing here needs both over the same memory. - #[inline] - pub fn unaligned(self) -> Dma<'pool, Unaligned> { - Dma::new(self.base, self.device, self.size) - } - - #[inline] - fn at(self, offset: usize) -> *mut T { - self.check(offset, core::mem::size_of::()); - if let Err(why) = - toyos_dma::aligned(self.base as usize, offset, core::mem::align_of::()) - { - refuse_unaligned(why, core::any::type_name::()); - } - // SAFETY: `check` above refused anything but `offset + size_of::() <= self.size`. - unsafe { self.base.add(offset) as *mut T } - } -} - -#[cold] -#[inline(never)] -fn refuse_unaligned(why: toyos_dma::Refused, what: &str) -> ! { - panic!("DMA: {why}, and a volatile access of {what} needs it"); -} - -impl Dma<'_, Unaligned> { - /// Read the `T` at `offset`, whatever it is aligned to. - #[inline] - pub fn read(self, offset: usize) -> T { - self.check(offset, core::mem::size_of::()); - // SAFETY: `check` refused anything but `size_of::()` bytes; not volatile, since this discipline is for a structure the device has finished writing. - unsafe { read_unaligned(self.base.add(offset) as *const T) } - } -} - -/// Contiguous DMA memory backed by 2 MiB physical pages; dropping it frees them. -pub struct DmaPool { - pages: Vec, - base: DirectMap, - size: usize, - /// The space and where the pool sits in it: the pages go back to the - /// allocator on drop, so the device loses them first. - space: DeviceSpace, - device: u64, -} - -impl DmaPool { - /// Take enough contiguous 2 MiB pages to cover `size` bytes, mapped into - /// `space` below `limit`, the device's address width, and reachable by - /// nothing outside it; refused where none below it is left — for an - /// untranslated device, where the pages lie above it. - pub fn alloc_in(size: usize, space: DeviceSpace, limit: Limit) -> Result { - let pages_2m = size.div_ceil(super::PAGE_2M as usize); - let pages = super::pmm::alloc_contiguous_for_kernel(pages_2m) - .expect("DmaPool: out of physical memory"); - let base = pages[0].direct_map(); - let size = pages_2m * super::PAGE_2M as usize; - // A device's width is its answer; any other refusal is a kernel bug: - // the addresses are the allocator's own and the pages are 2 MiB. - let device = match space.map(base.phys(), size as u64, limit) { - Ok(device) => device, - Err(why @ IommuError::AddressesExhausted(_)) => return Err(why), - Err(why) => panic!("DmaPool: {why}"), - }; - Ok(Self { pages, base, size, space, device }) - } - - /// The whole pool as a view, borrowed so it cannot outlive `self`: the - /// virtio parse selftest's alone. - #[cfg(feature = "boot-actuators")] - pub fn view(&self) -> Dma<'_> { - Dma::new(self.base.as_mut_ptr(), self.device, self.size) - } - - /// Consumes the pool and leaks its pages and its mapping for a `'static` view; never freed, deliberately. - pub fn leak(mut self) -> Dma<'static> { - let (base, device, size) = (self.base, self.device, self.size); - self.space = DeviceSpace::Untranslated; - for page in core::mem::take(&mut self.pages) { - #[expect(clippy::disallowed_methods, reason = "a bound device keeps its pool for the boot, which `Dma<'static>` says")] - core::mem::forget(page); - } - Dma::new(base.as_mut_ptr(), device, size) - } -} - -impl Drop for DmaPool { - fn drop(&mut self) { - // Before the pages go back: a device still holding these addresses - // would reach whatever the allocator hands out next. - self.space - .unmap(self.device, self.size as u64) - .unwrap_or_else(|why| panic!("DmaPool: {why}")); - } -} diff --git a/kernel/src/mm/mmio.rs b/kernel/src/mm/mmio.rs index 9c6bd43375e..afb3389d772 100644 --- a/kernel/src/mm/mmio.rs +++ b/kernel/src/mm/mmio.rs @@ -54,11 +54,6 @@ impl Mmio { self.base as u64 } - /// The window's byte size — the bound every access here is checked against. - pub fn size(self) -> u64 { - self.size - } - pub fn subregion(self, offset: u64, size: u64) -> Mmio { assert!(offset + size <= self.size, "Mmio subregion OOB: offset={:#x} size={:#x} total={:#x}", offset, size, self.size); diff --git a/kernel/src/mm/mod.rs b/kernel/src/mm/mod.rs index bb5b4330bcb..8ba828a10a8 100644 --- a/kernel/src/mm/mod.rs +++ b/kernel/src/mm/mod.rs @@ -7,7 +7,6 @@ pub mod pmm; pub use crate::arch::paging; mod alloc; pub mod charge; -mod dma; mod image; mod mmio; pub mod policy; @@ -23,7 +22,6 @@ pub use alloc::sweep as sweep_heap_bands; /// Isolates the sweep's lock hold from the sweep itself, to attribute contention correctly. #[cfg(feature = "heap-lockspin")] pub use alloc::hold_lock as hold_heap_lock; -pub use dma::{Dma, DmaPool, Unaligned}; pub use image::Image; pub use mmio::Mmio; pub use region::{Allocation, KernelSlice}; diff --git a/kernel/src/mm/pmm.rs b/kernel/src/mm/pmm.rs index d893612aa8d..4bc48c64b73 100644 --- a/kernel/src/mm/pmm.rs +++ b/kernel/src/mm/pmm.rs @@ -344,14 +344,6 @@ pub fn alloc_contiguous(count: usize) -> Option> { Some(run(base, count)) } -/// `count` physically contiguous zeroed 2 MiB pages for the kernel's own use, -/// such as a kernel driver's DMA pool. -pub fn alloc_contiguous_for_kernel(count: usize) -> Option> { - assert!(count > 0, "pmm: a kernel run of no pages"); - let base = FRAMES.lock().as_mut()?.alloc_run(count)?; - Some(run(base, count)) -} - /// The pages of a run just taken at `base`, zeroed. fn run(base: u64, count: usize) -> alloc::vec::Vec { let mut pages = alloc::vec::Vec::with_capacity(count); diff --git a/kernel/src/object/device.rs b/kernel/src/object/device.rs index b0e5df3789f..8932bdc84cf 100644 --- a/kernel/src/object/device.rs +++ b/kernel/src/object/device.rs @@ -172,11 +172,29 @@ impl ConsoleObject { pub fn write(&self, buf: &crate::user_ptr::UserBytes) -> usize { self.line.lock().write(buf) } + + /// Whether this holder carries the log off the machine, so the UART does + /// not (`log::console::carried_by`); its end lets go of it. + pub fn carry(&self, elsewhere: bool) -> Result<(), kernel::carrier::Refused> { + match elsewhere { + true => crate::log::console::carried_by(self.holder()), + false => { + crate::log::console::released_by(self.holder()); + Ok(()) + } + } + } + + fn holder(&self) -> usize { + self as *const Self as usize + } } -// Flushes the partial line on last handle instead of dropping it. +// Flushes the partial line on last handle instead of dropping it, and gives +// the log back to the UART if this holder carried it. impl Drop for ConsoleObject { fn drop(&mut self) { self.line.lock().finish(); + crate::log::console::released_by(self.holder()); } } diff --git a/kernel/src/panic.rs b/kernel/src/panic.rs index 45f37a49500..115804e8808 100644 --- a/kernel/src/panic.rs +++ b/kernel/src/panic.rs @@ -346,6 +346,7 @@ pub fn last_words( /// path's one funnel. // panic_flush bypasses the log-ring and serial locks — once the others are stopped a wedged holder never releases them, so taking them normally could deadlock. pub fn halt_all_cpus() -> ! { + crate::log::console::on_the_uart_for_good(); // Before the halt and the panel: from here this machine holds a report for // whoever is in front of it, with interrupts masked and under a bound of its // own — which to a hard-lockup sample or a deadline poll is diff --git a/kernel/src/pcidev/mod.rs b/kernel/src/pcidev/mod.rs index f03e0e41a17..e49b5f4ff92 100644 --- a/kernel/src/pcidev/mod.rs +++ b/kernel/src/pcidev/mod.rs @@ -124,7 +124,7 @@ use toyos_abi::boot::RootBridgeWindow; use toyos_abi::inventory::Arrival; use toyos_abi::pci::{DeviceIrqRecord, PciFunctionInfo, BARS, LEND_WINDOW}; use toyos_abi::syscall::{PciId, RegWidth, SyscallError}; -use toyos_dma::Register; +use toyos_register::Register; use toyos_pci::{af, aperture, bar, express, pages, pm}; use crate::device::{Claim, ClaimError}; @@ -1530,8 +1530,8 @@ pub fn dma_unmap(binding: &Binding, at: u64) -> Result, Sys /// The window a claim's configuration reads are checked against, for the one /// caller that turns an offset from userland into a [`Register`]. -pub fn config_window(offset: u64, width: RegWidth) -> Result { - toyos_dma::register(offset, width.bytes(), CONFIG_BYTES) +pub fn config_window(offset: u64, width: RegWidth) -> Result { + toyos_register::register(offset, width.bytes(), CONFIG_BYTES) } /// One dword, word or byte of this function's own config space. diff --git a/kernel/src/syscall/dispatch.rs b/kernel/src/syscall/dispatch.rs index 28ac025fe00..0c15aab1bf3 100644 --- a/kernel/src/syscall/dispatch.rs +++ b/kernel/src/syscall/dispatch.rs @@ -32,7 +32,7 @@ use super::fs::{ }; use super::handles::{sys_close, sys_handle_dup, with_object, with_object_ref}; use super::io::{ - sys_null, sys_random, sys_read, sys_read_nonblock, sys_write, sys_write_nonblock, + sys_console_carried, sys_null, sys_random, sys_read, sys_read_nonblock, sys_write, sys_write_nonblock, }; use super::ipc::{ sys_accept, sys_handle_recv, sys_inbox_setup, sys_message_send, @@ -487,6 +487,7 @@ pub(crate) fn syscall_dispatch(num: u64, a1: u64, a2: u64, a3: u64, a4: u64) -> SYS_SHM_CREATE => sys_shm_create(a1, a2), SYS_SHM_MAP => sys_shm_map(RawHandle(a1 as u32)), SYS_SHM_SUB => sys_shm_sub(RawHandle(a1 as u32), a2, a3), + SYS_CONSOLE_CARRIED => sys_console_carried(RawHandle(a1 as u32), a2), SYS_PORT_CREATE => sys_port_create(), SYS_PORT_MINT => { // Copied in before any handle is looked up: the kernel holds the badge, never the caller's page. @@ -549,7 +550,11 @@ pub(crate) fn syscall_dispatch(num: u64, a1: u64, a2: u64, a3: u64, a4: u64) -> crate::scheduler::yield_now(); 0 } - DA::FATAL_HALT => { log!("{}", FATAL_HALT_NONCE); crate::panic::halt_all_cpus(); } + DA::FATAL_HALT => { + crate::log::console::on_the_uart_for_good(); + log!("{}", FATAL_HALT_NONCE); + crate::panic::halt_all_cpus(); + } DA::DOUBLE_FAULT => { log!("SYS_DEBUG: provoking a double fault"); crate::arch::trap::provoke_double_fault() diff --git a/kernel/src/syscall/io.rs b/kernel/src/syscall/io.rs index 3ce765d996b..3d46be2c958 100644 --- a/kernel/src/syscall/io.rs +++ b/kernel/src/syscall/io.rs @@ -235,3 +235,24 @@ pub(super) fn sys_random(out: &mut UserBytesMut) -> u64 { crate::random::fill_user(out); 0 } + +/// `SYS_CONSOLE_CARRIED`: whether the holder of the console `h` carries the +/// log off the machine ([`crate::object::device::ConsoleObject::carry`]). +pub(super) fn sys_console_carried(h: RawHandle, elsewhere: u64) -> u64 { + let elsewhere = match elsewhere { + 0 => false, + 1 => true, + _ => return SyscallError::InvalidArgument.to_u64(), + }; + let console = match process::with_process_data(|data| { + data.handles.get::(h, Rights::WRITE) + }) { + Ok(console) => console, + Err(e) => return e.refuse(), + }; + match console.carry(elsewhere) { + Ok(()) => 0, + Err(kernel::carrier::Refused::Another) => SyscallError::AlreadyExists.to_u64(), + Err(kernel::carrier::Refused::ForGood) => SyscallError::NotSupported.to_u64(), + } +} diff --git a/lib/logkeeper-api/src/lib.rs b/lib/logkeeper-api/src/lib.rs index 0a18aad8da1..df341e49114 100644 --- a/lib/logkeeper-api/src/lib.rs +++ b/lib/logkeeper-api/src/lib.rs @@ -1,11 +1,12 @@ //! Asking `logkeeper` for this boot's log on this machine: one [`READ`] on //! its [`SERVICE`] port, answered by one [`SERVED`] frame carrying how many //! bytes are the boot so far and the read end of a pipe the log is written -//! into, from its first line. +//! into, from its first line; or the same as the machine's console carrier +//! ([`carry`]). use toyos::endow; -use toyos::Pipe; -use toyos_logstream::{READ, SERVED, SERVICE}; +use toyos::{Connection, Pipe}; +use toyos_logstream::{CARRIER, CARRY, DELIVERED, READ, SERVED, SERVICE}; /// `logkeeper`'s answer. pub struct Served { @@ -18,6 +19,11 @@ pub struct Served { pub fn read() -> Result { let conn = endow::service(SERVICE).map_err(|e| format!("no `{SERVICE}` service: {e:?}"))?; conn.signal(READ).map_err(|e| format!("logkeeper would not take the request: {e:?}"))?; + answer(&conn) +} + +/// The [`SERVED`] frame a request on `conn` is answered with. +fn answer(conn: &Connection) -> Result { let header = conn.recv_header().map_err(|e| format!("logkeeper did not answer: {e:?}"))?; if header.msg_type != SERVED { return Err(format!("logkeeper answered frame type {}", header.msg_type)); @@ -30,3 +36,20 @@ pub fn read() -> Result { let pipe = unsafe { Pipe::from_raw(raw) }; Ok(Served { pipe, boot_so_far }) } + +/// Ask for the log as the machine's console carrier, on +/// [`toyos_logstream::CARRIER`]: the answer is [`read`]'s, and the connection +/// stays open for the carrier to say what its channel has taken +/// ([`delivered`]). +pub fn carry() -> Result<(Connection, Served), String> { + let conn = endow::service(CARRIER).map_err(|e| format!("no `{CARRIER}` service: {e:?}"))?; + conn.signal(CARRY).map_err(|e| format!("logkeeper would not take the request: {e:?}"))?; + let served = answer(&conn)?; + Ok((conn, served)) +} + +/// Tell `logkeeper` the carrier's channel has taken `through` bytes of its +/// pipe. Never waits: a later count says everything a refused one would have. +pub fn delivered(conn: &Connection, through: u64) { + let _ = conn.try_send(DELIVERED, &through); +} diff --git a/lib/logstream/src/lib.rs b/lib/logstream/src/lib.rs index 622c71a9001..88c49561a1b 100644 --- a/lib/logstream/src/lib.rs +++ b/lib/logstream/src/lib.rs @@ -47,6 +47,22 @@ pub const SERVICE: &str = "log"; /// A reader's request on [`SERVICE`]: a bare frame, answered by [`SERVED`]. pub const READ: u32 = 3; +/// The name of the port the machine's console carrier asks `logkeeper` for the +/// log on: the one reader the console UART goes quiet for, once its channel has +/// taken the log's first bytes, and back from once it goes or the machine +/// stops. Its row alone receives it. +pub const CARRIER: &str = "log-carrier"; + +/// The one row that may receive [`CARRIER`]: the VM console's driver. +pub const CARRIER_HOLDER: &str = "virtio-console"; + +/// A carrier's request on [`CARRIER`]: a bare frame, answered by [`SERVED`]. +pub const CARRY: u32 = 8; + +/// The carrier → `logkeeper`, on [`CARRIER`]: how many bytes of its pipe its +/// channel has taken, counted from the pipe's first byte, as a `u64` payload. +pub const DELIVERED: u32 = 9; + /// The manifest's name for the program that is the log: the supervisor endows it the /// [`ORIGINS`] acceptor and a console it may write ([`CONSOLE`]), and gives /// it a ring like every other program's, which it reads like every other. diff --git a/lib/manifest/src/lib.rs b/lib/manifest/src/lib.rs index 07525d3838e..f0d811e85ee 100644 --- a/lib/manifest/src/lib.rs +++ b/lib/manifest/src/lib.rs @@ -477,7 +477,9 @@ pub enum RenderError { /// no PCI matcher. NoSuchDevice { program: String, entry: String }, /// Two entries one device can answer, by row and entry: the second claim - /// would be refused at boot, or two rows would race for one function. + /// would be refused at boot. Two rows' `drives` entries may: the function + /// goes to the first row's process, and to the next row's once one ends + /// with exit 0, which says the function is not its own. Overlap { first: (String, String), second: (String, String) }, /// A `drives` row that also has what one process per function cannot /// carry: `serves`, `roles` or `devices`, or a place in `[boot] start` or @@ -574,9 +576,10 @@ pub fn render(manifest: &Manifest) -> Result, RenderError> { } /// Every entry names a device, a `drives` row is one process per function and -/// nothing else, and no device can answer two entries — but two different -/// `drives` entries of one row, which start one process on a function either -/// way, under the first that matches it. +/// nothing else, and no device can answer two entries — but two `drives` +/// entries other than one written twice: one row's start one process on a +/// function under the first that matches it, and two rows' take turns at it, +/// in their order, until one's process keeps it. fn claims_apart(manifest: &Manifest) -> Result<(), RenderError> { let mut seen: Vec<(&str, &str, device::Entry, bool)> = Vec::new(); for program in &manifest.programs { @@ -602,7 +605,7 @@ fn claims_apart(manifest: &Manifest) -> Result<(), RenderError> { for entry in devices.chain(drives) { let (text, parsed, driven) = entry?; let apart = |row: &str, other: &str, earlier_driven: bool| { - driven && earlier_driven && row == program.name && other != text.as_str() + driven && earlier_driven && (row != program.name || other != text.as_str()) }; if let Some((row, other, ..)) = seen.iter().find(|(row, other, earlier, was)| earlier.overlaps(&parsed) && !apart(row, other, *was)) @@ -1077,16 +1080,16 @@ mod tests { /// **Two entries one device can answer are refused where the image is /// built, naming both**: across rows, within one row's `devices`, and an - /// entry written twice. Two different `drives` entries of one row are not: - /// the row starts one process on the function under the first. + /// entry written twice. Two different `drives` entries are not: one row + /// starts one process on the function under the first, and two rows take + /// turns at it. #[test] fn two_entries_one_device_can_answer_are_refused_by_name() { - let cases: [(&[Row<'_>], _, _); 6] = [ + let cases: [(&[Row<'_>], _, _); 5] = [ (&[("a", &[], &["pci:ff:00:00", "pci:ff:00:00"])], ("a", "pci:ff:00:00"), ("a", "pci:ff:00:00")), (&[("a", &["framebuffer"], &[]), ("b", &["framebuffer"], &[])], ("a", "framebuffer"), ("b", "framebuffer")), (&[("a", &["isa:0060,0064:1,12"], &[]), ("b", &["isa:0060,0064:1,12"], &[])], ("a", "isa:0060,0064:1,12"), ("b", "isa:0060,0064:1,12")), (&[("a", &["pci:01:08:02/1b36:0010"], &[]), ("b", &[], &["pci:01:08:02"])], ("a", "pci:01:08:02/1b36:0010"), ("b", "pci:01:08:02")), - (&[("a", &[], &["pci:0c:03:30"]), ("b", &[], &["pci:0c:03:30/1b36:000d"])], ("a", "pci:0c:03:30"), ("b", "pci:0c:03:30/1b36:000d")), (&[("a", &["pci:02:00:00", "pci:02:00:00/1af4:1041"], &[])], ("a", "pci:02:00:00"), ("a", "pci:02:00:00/1af4:1041")), ]; for (rows, first, second) in cases { @@ -1100,6 +1103,8 @@ mod tests { assert!(render(&apart).is_ok()); let one_row = with_rows(&[("usbd", &[], &["pci:0c:03:30/1033:0194", "pci:0c:03:30@boot"])]); assert!(render(&one_row).is_ok()); + let two_rows = with_rows(&[("vdagent", &[], &["pci:07:80:00/1af4:1043"]), ("console", &[], &["pci:07:80:00/1af4:1043"])]); + assert!(render(&two_rows).is_ok()); } #[test] diff --git a/lib/vdagent/src/lib.rs b/lib/vdagent/src/lib.rs index 923198a7d29..03cb1aa9926 100644 --- a/lib/vdagent/src/lib.rs +++ b/lib/vdagent/src/lib.rs @@ -1,15 +1,13 @@ -//! QEMU's clipboard channel, as a guest agent speaks it: the channel's -//! declaration, virtio-serial's control messages ([`control`]) and the spice -//! vdagent stream ([`stream`] decodes it, [`encode`] writes it). +//! QEMU's clipboard channel, as a guest agent speaks it: the agent's port's +//! name and the spice vdagent stream ([`stream`] decodes it, [`encode`] writes +//! it). The virtio-serial function it is on is `toyos-virtio-serial`'s. //! //! **Every word decoded here is the host's**, written by QEMU's `qemu-vdagent` //! chardev through its virtio-serial device. Each is bounded before it sizes a //! buffer, and one that fails its bound is a refusal by name: a decode never //! panics and never buffers more than arrived. //! -//! What it cites: *Virtual I/O Device (VIRTIO) Version 1.2*, OASIS Committee -//! Specification 01, §5.3 for the console device and its control messages, and -//! spice-protocol's `spice/vd_agent.h` at `499cc832` for the stream, its +//! What it cites: spice-protocol's `spice/vd_agent.h` at `499cc832` for the stream, its //! packed layouts and its numbers. The header leaves a `GRAB`'s serial without //! a meaning; what QEMU 11.1.1 does with it is the agent's to state, measured. @@ -20,26 +18,13 @@ extern crate alloc; #[cfg(test)] extern crate std; -pub mod control; pub mod encode; pub mod stream; /// The agent's port, as `vd_agent.h:63-68` names it: what the launch names the -/// `virtserialport` it creates. +/// `virtserialport` it creates, and how the agent knows its function. pub const PORT_NAME: &str = "com.redhat.spice.0"; -/// The virtio-serial function's `max_ports`: port 0, which nothing uses, and -/// the agent's. -pub const MAX_PORTS: u32 = 2; - -/// The agent's port id, `nr=` at the launch: the one id the launch and the -/// agent both read, so a `PORT_NAME`'s bytes are never decoded. -pub const PORT_ID: u32 = 1; - -/// How many control receive buffers the agent keeps posted: virtio-serial drops -/// a control event that finds none. -pub const CONTROL_BUFFERS: usize = 8; - /// The longest clipboard text that crosses, in bytes: the compositor's /// `window::MAX_CLIPBOARD_BYTES`, which the agent asserts this equals. pub const MAX_TEXT: usize = 2 * 1024 * 1024; diff --git a/lib/virtio-serial/Cargo.toml b/lib/virtio-serial/Cargo.toml new file mode 100644 index 00000000000..f7583479ff7 --- /dev/null +++ b/lib/virtio-serial/Cargo.toml @@ -0,0 +1,15 @@ +# A member of the host workspace (root `Cargo.toml`), like toyos-vdagent: it is +# pure and its tests run on the host. It is an input boundary of its own: every +# control message it decodes is the host's, written by QEMU's virtio-serial +# device. + +[package] +name = "toyos-virtio-serial" +description = "virtio-serial as this project's two drivers of it speak it, pure: the launch's declarations, the control messages and port names decoded bounded, and which port of a function is a driver's." +version = "0.1.0" +edition = "2021" +license = "MIT OR Apache-2.0" +publish = false + +[lib] +doctest = false diff --git a/lib/virtio-serial/claim/Cargo.toml b/lib/virtio-serial/claim/Cargo.toml new file mode 100644 index 00000000000..3af07620d55 --- /dev/null +++ b/lib/virtio-serial/claim/Cargo.toml @@ -0,0 +1,18 @@ +[package] +name = "toyos-virtio-serial-claim" +description = "A virtio-serial function brought up and driven over its PCI claim, for this project's two drivers of it: its six queues, the control messages each way and the named port's buffers." +version = "0.1.0" +edition = "2021" +license = "MIT OR Apache-2.0" +publish = false + +[dependencies] +toyos = { path = "../../../sdk/toyos" } +toyos-abi = { path = "../../../sdk/abi" } +toyos-device-memory = { path = "../../device-memory" } +toyos-pci-claim = { path = "../../pci-claim" } +toyos-virtio = { path = "../../virtio" } +toyos-virtio-serial = { path = ".." } + +[lib] +doctest = false diff --git a/system/vdagent/src/channel.rs b/lib/virtio-serial/claim/src/lib.rs similarity index 51% rename from system/vdagent/src/channel.rs rename to lib/virtio-serial/claim/src/lib.rs index 15ca240ec2c..5f9d1665158 100644 --- a/system/vdagent/src/channel.rs +++ b/lib/virtio-serial/claim/src/lib.rs @@ -1,20 +1,21 @@ -//! The virtio-serial function QEMU's clipboard channel is on, driven here -//! over its claim: a multiport console device (virtio 1.2 §5.3) whose port -//! [`toyos_vdagent::PORT_ID`] carries the vdagent stream. +//! A virtio-serial function driven over its claim: a multiport console device +//! (virtio 1.2 §5.3) whose port [`toyos_virtio_serial::PORT_ID`] carries the +//! driver's stream. //! //! **The transport and the rings are `toyos-virtio`'s**, over //! `toyos-pci-claim`'s register window and grant. What is this file's is the //! console device: its six queues (§5.3.2) — port 0's two, which nothing uses, -//! the control pair and the agent's port's pair — on the one MSI-X entry the -//! kernel programs, and which buffer goes with which head. +//! the control pair and the named port's pair — on the one MSI-X entry the +//! kernel programs, and which buffer goes with which head. How many buffers +//! the named port has each way is the driver's ([`Shape`]). //! -//! **Every receive buffer is posted at its own head**, so a completion's head -//! *is* the buffer it filled and nothing the device wrote is an index. **A -//! control receive buffer is posted again as soon as it is read**, before its -//! message is handled: QEMU 11.1.1 drops a control event that finds none -//! posted, measured. +//! **Every buffer is posted at its own head**, so a completion's head *is* the +//! buffer it used and nothing the device wrote is an index. **A control +//! receive buffer is posted again as soon as it is read**, before its message +//! is handled: QEMU 11.1.1 drops a control event that finds none posted, +//! measured. //! -//! **A used ring the device is not believed on ends this program** by the +//! **A used ring the device is not believed on ends the program** by the //! refusal's own name: every message after it is one that silently never //! arrives. @@ -26,11 +27,12 @@ use toyos_abi::syscall::SyscallError; use toyos_device_memory::DmaBuffers; use toyos_pci_claim::virtio::{negotiate, Negotiated, Refusal}; use toyos_pci_claim::{Bar, Grant}; -use toyos_vdagent::control::{self, Control}; -use toyos_vdagent::{CONTROL_BUFFERS, LARGEST_TRANSMIT, MAX_PORTS}; use toyos_virtio::pci::Live; use toyos_virtio::queue::{Buffer, Parts, Used, Virtqueue}; +use toyos_virtio_serial::control::{self, Control}; +use toyos_virtio_serial::{CONTROL_BUFFERS, MAX_PORTS}; + /// §5.3.3: the device has control queues and more than one port. const VIRTIO_CONSOLE_F_MULTIPORT: u64 = 1 << 1; @@ -52,38 +54,57 @@ const PORT_TX: u16 = 5; /// Port 0 exists and is never posted to. const PORT0_SIZE: u16 = 1; const CONTROL_RX_SIZE: u16 = CONTROL_BUFFERS as u16; -/// The control messages the agent can have in flight at once: at most a -/// `DEVICE_READY`, a `PORT_READY` and two `PORT_OPEN`s between two reads of -/// the used ring. +/// The control messages a driver can have in flight at once: at most a +/// `DEVICE_READY`, a `PORT_READY` per port and two `PORT_OPEN`s between two +/// reads of the used ring. const CONTROL_TX_SIZE: u16 = 4; -const PORT_RX_SIZE: u16 = 16; -/// One transmit, of every message owed, in one element. -const PORT_TX_SIZE: u16 = 1; - -/// A control buffer holds a message and a `PORT_NAME`'s name, which is not -/// read: a longer name is cut where the buffer ends. -const CONTROL_RX_BYTES: usize = 64; -const PORT_RX_BYTES: usize = 4096; - -/// Each queue's parts, one after another on the descriptor table's alignment. -const fn after(parts: Parts, size: u16) -> usize { - (parts.end(size) + 15) & !15 -} -const Q_PORT0_RX: Parts = Parts::contiguous(0, PORT0_SIZE); -const Q_PORT0_TX: Parts = Parts::contiguous(after(Q_PORT0_RX, PORT0_SIZE), PORT0_SIZE); -const Q_CONTROL_RX: Parts = Parts::contiguous(after(Q_PORT0_TX, PORT0_SIZE), CONTROL_RX_SIZE); -const Q_CONTROL_TX: Parts = Parts::contiguous(after(Q_CONTROL_RX, CONTROL_RX_SIZE), CONTROL_TX_SIZE); -const Q_PORT_RX: Parts = Parts::contiguous(after(Q_CONTROL_TX, CONTROL_TX_SIZE), PORT_RX_SIZE); -const Q_PORT_TX: Parts = Parts::contiguous(after(Q_PORT_RX, PORT_RX_SIZE), PORT_TX_SIZE); const PAGE: usize = 0x1000; -const OFF_CONTROL_RX: usize = (Q_PORT_TX.end(PORT_TX_SIZE) + PAGE - 1) & !(PAGE - 1); -const OFF_CONTROL_TX: usize = OFF_CONTROL_RX + CONTROL_RX_SIZE as usize * CONTROL_RX_BYTES; -const OFF_PORT_RX: usize = (OFF_CONTROL_TX + CONTROL_TX_SIZE as usize * control::BYTES + PAGE - 1) & !(PAGE - 1); -const OFF_PORT_TX: usize = OFF_PORT_RX + PORT_RX_SIZE as usize * PORT_RX_BYTES; -const GRANT_BYTES: u64 = (OFF_PORT_TX + LARGEST_TRANSMIT) as u64; -/// Why the function is not the agent's to drive. +/// The named port's buffers each way: how many, and how many bytes each. +#[derive(Clone, Copy)] +pub struct Shape { + /// `None` posts nothing, so the host's bytes stay with the host. + pub receive: Option<(u16, usize)>, + pub transmit: (u16, usize), +} + +/// Where everything is in the grant. +struct Layout { + rx_size: u16, + tx_size: u16, + rx_bytes: usize, + tx_bytes: usize, + queues: [Parts; 6], + control_rx: usize, + control_tx: usize, + port_rx: usize, + port_tx: usize, + bytes: u64, +} + +impl Layout { + fn of(shape: Shape) -> Self { + let (rx_size, rx_bytes) = shape.receive.unwrap_or((1, 0)); + let (tx_size, tx_bytes) = shape.transmit; + let sizes = [PORT0_SIZE, PORT0_SIZE, CONTROL_RX_SIZE, CONTROL_TX_SIZE, rx_size, tx_size]; + let mut at = 0; + let queues = sizes.map(|size| { + let parts = Parts::contiguous(at, size); + at = (parts.end(size) + 15) & !15; + parts + }); + let page = |at: usize| (at + PAGE - 1) & !(PAGE - 1); + let control_rx = page(at); + let control_tx = control_rx + CONTROL_RX_SIZE as usize * control::BUFFER_BYTES; + let port_rx = page(control_tx + CONTROL_TX_SIZE as usize * control::BYTES); + let port_tx = page(port_rx + rx_size as usize * rx_bytes); + let bytes = (port_tx + tx_size as usize * tx_bytes) as u64; + Self { rx_size, tx_size, rx_bytes, tx_bytes, queues, control_rx, control_tx, port_rx, port_tx, bytes } + } +} + +/// Why the function is not driven. pub enum NotOpened { Bringup(Refusal), /// It negotiated no `VIRTIO_CONSOLE_F_MULTIPORT`, so it has no port but 0. @@ -123,15 +144,26 @@ impl From for NotOpened { /// The next chain the device has finished with on `rings`, or `None`; a used /// ring the device is not believed on ends the program. fn finished(rings: &mut Virtqueue) -> Option { - rings - .poll_used() - .unwrap_or_else(|why| panic!("vdagent: this device cannot be driven on — {why}")) + rings.poll_used().unwrap_or_else(|why| panic!("virtio-serial: this device cannot be driven on — {why}")) +} + +/// One control buffer the device used, as many bytes as it wrote. +pub struct Received { + bytes: [u8; control::BUFFER_BYTES], + len: usize, +} + +impl Received { + pub fn bytes(&self) -> &[u8] { + &self.bytes[..self.len] + } } /// The function, brought up and driving. pub struct Channel { dev: PciDev, device: Live, + layout: Layout, /// Held for their mappings' lives: the register window points into the /// first and the grant's into the second. _bar: SharedMemory, @@ -145,16 +177,20 @@ pub struct Channel { control_free: Vec, port_rx: Virtqueue, port_tx: Virtqueue, - /// When the transmit in flight was made available. - sent: Option, + /// Transmit heads nothing is in flight on. + transmit_free: Vec, + /// Each transmit head's length and when it was made available, while it + /// is in flight. + in_flight: Vec>, } impl Channel { /// Bring the function up — §3.1.1's order, every queue before /// `DRIVER_OK` — post every receive buffer, and send `DEVICE_READY`. - pub fn open(dev: PciDev) -> Result { + /// `who` names the driver in what it says. + pub fn open(dev: PciDev, shape: Shape, who: &str) -> Result { let Negotiated { setup, mapping, features } = negotiate(&dev, VIRTIO_CONSOLE_F_MULTIPORT)?; - toyos::say!("vdagent: {features}"); + toyos::say!("{who}: {features}"); if setup.features() & VIRTIO_CONSOLE_F_MULTIPORT == 0 { return Err(NotOpened::NotMultiport); } @@ -163,15 +199,17 @@ impl Channel { return Err(NotOpened::Ports(ports)); } - let (grant, region) = Grant::alloc(&dev, GRANT_BYTES)?; + let layout = Layout::of(shape); + let (grant, region) = Grant::alloc(&dev, layout.bytes)?; + let [q0, q1, q2, q3, q4, q5] = layout.queues; let port0 = [ - Virtqueue::new(grant, PORT0_RX, PORT0_SIZE, Q_PORT0_RX), - Virtqueue::new(grant, PORT0_TX, PORT0_SIZE, Q_PORT0_TX), + Virtqueue::new(grant, PORT0_RX, PORT0_SIZE, q0), + Virtqueue::new(grant, PORT0_TX, PORT0_SIZE, q1), ]; - let control_rx = Virtqueue::new(grant, CONTROL_RX, CONTROL_RX_SIZE, Q_CONTROL_RX); - let control_tx = Virtqueue::new(grant, CONTROL_TX, CONTROL_TX_SIZE, Q_CONTROL_TX); - let port_rx = Virtqueue::new(grant, PORT_RX, PORT_RX_SIZE, Q_PORT_RX); - let port_tx = Virtqueue::new(grant, PORT_TX, PORT_TX_SIZE, Q_PORT_TX); + let control_rx = Virtqueue::new(grant, CONTROL_RX, CONTROL_RX_SIZE, q2); + let control_tx = Virtqueue::new(grant, CONTROL_TX, CONTROL_TX_SIZE, q3); + let port_rx = Virtqueue::new(grant, PORT_RX, layout.rx_size, q4); + let port_tx = Virtqueue::new(grant, PORT_TX, layout.tx_size, q5); let setup = setup .config_vector(MSIX_ENTRY)? .enable(&port0[0], MSIX_ENTRY)? @@ -193,13 +231,17 @@ impl Channel { control_free: (0..CONTROL_TX_SIZE).rev().collect(), port_rx, port_tx, - sent: None, + transmit_free: (0..layout.tx_size).rev().collect(), + in_flight: vec![None; layout.tx_size as usize], + layout, }; for head in 0..CONTROL_RX_SIZE { channel.post_control(head); } - for head in 0..PORT_RX_SIZE { - channel.post_port(head); + if shape.receive.is_some() { + for head in 0..channel.layout.rx_size { + channel.post_port(head); + } } channel.send_control(Control { id: 0, event: control::Event::DeviceReady, value: 1 }); Ok(channel) @@ -215,36 +257,35 @@ impl Channel { pub fn take_interrupt(&self) { match self.dev.irq() { Ok(_) | Err(SyscallError::WouldBlock) => {} - Err(why) => panic!("vdagent: the claim refused its interrupt record: {why:?}"), + Err(why) => panic!("virtio-serial: the claim refused its interrupt record: {why:?}"), } } fn post_control(&mut self, head: u16) { - let at = OFF_CONTROL_RX + head as usize * CONTROL_RX_BYTES; - let buffer = Buffer::writable(self.grant.device_addr(at), CONTROL_RX_BYTES as u32); + let at = self.layout.control_rx + head as usize * control::BUFFER_BYTES; + let buffer = Buffer::writable(self.grant.device_addr(at), control::BUFFER_BYTES as u32); let published = self.control_rx.publish(head, &[buffer]); self.device.notify(published); } fn post_port(&mut self, head: u16) { - let at = OFF_PORT_RX + head as usize * PORT_RX_BYTES; - let buffer = Buffer::writable(self.grant.device_addr(at), PORT_RX_BYTES as u32); + let at = self.layout.port_rx + head as usize * self.layout.rx_bytes; + let buffer = Buffer::writable(self.grant.device_addr(at), self.layout.rx_bytes as u32); let published = self.port_rx.publish(head, &[buffer]); self.device.notify(published); } - /// The next control message the host sent, its buffer posted again - /// before it is answered. - pub fn control(&mut self) -> Option> { + /// The next control buffer the host filled, posted again before it is + /// answered. + pub fn control(&mut self) -> Option { let Used { head, written } = finished(&mut self.control_rx)?; - let at = OFF_CONTROL_RX + head as usize * CONTROL_RX_BYTES; - let mut bytes = [0u8; control::BYTES]; - let len = (written as usize).min(control::BYTES); - for (i, byte) in bytes[..len].iter_mut().enumerate() { + let at = self.layout.control_rx + head as usize * control::BUFFER_BYTES; + let mut received = Received { bytes: [0; control::BUFFER_BYTES], len: written as usize }; + for (i, byte) in received.bytes[..received.len].iter_mut().enumerate() { *byte = self.grant.window().read(at + i); } self.post_control(head); - Some(control::decode(&bytes[..len], MAX_PORTS)) + Some(received) } /// Send `message` on the control queue. @@ -259,8 +300,8 @@ impl Channel { let head = self .control_free .pop() - .unwrap_or_else(|| panic!("vdagent: the device took back none of {CONTROL_TX_SIZE} control messages")); - let at = OFF_CONTROL_TX + head as usize * control::BYTES; + .unwrap_or_else(|| panic!("virtio-serial: the device took back none of {CONTROL_TX_SIZE} control messages")); + let at = self.layout.control_tx + head as usize * control::BYTES; for (i, byte) in control::encode(message).into_iter().enumerate() { self.grant.window().write(at + i, byte); } @@ -269,11 +310,12 @@ impl Channel { self.device.notify(published); } - /// Hand every buffer the host filled on the agent's port to `each`, and + /// Hand every buffer the host filled on the named port to `each`, and /// post it again. pub fn received(&mut self, mut each: impl FnMut(&[u8])) { while let Some(Used { head, written }) = finished(&mut self.port_rx) { - let window = self.grant.window().sub(OFF_PORT_RX + head as usize * PORT_RX_BYTES, written as usize); + let at = self.layout.port_rx + head as usize * self.layout.rx_bytes; + let window = self.grant.window().sub(at, written as usize); // SAFETY: the window is inside the grant, which lives as long as // `self`; the device has finished with this buffer — its used // element said so, and `written` is at most the buffer's length — @@ -283,39 +325,50 @@ impl Channel { } } - /// Whether the device has taken the last transmit back, so a new one can - /// be made. - pub fn transmit_free(&mut self) -> bool { - if finished(&mut self.port_tx).is_some() { - self.sent = None; + /// Take back every transmit the device has finished with: answers the + /// bytes they carried. + pub fn transmitted(&mut self) -> u64 { + let mut bytes = 0; + while let Some(Used { head, .. }) = finished(&mut self.port_tx) { + let (len, _) = self.in_flight[head as usize].take().expect("a used head was published"); + bytes += u64::from(len); + self.transmit_free.push(head); } - self.sent.is_none() + bytes + } + + /// Whether a transmit can be made now. + pub fn can_transmit(&self) -> bool { + !self.transmit_free.is_empty() } - /// When the transmit in flight was made, if one is. - pub fn sent(&self) -> Option { - self.sent + /// When the oldest transmit still in flight was made, if one is. + pub fn oldest(&self) -> Option { + self.in_flight.iter().flatten().map(|(_, at)| *at).min() } - /// One transmit on the agent's port, of the bytes `fill` writes into a - /// [`LARGEST_TRANSMIT`] buffer, and `fill` answers how many. Nothing is - /// sent for 0. + /// One transmit on the named port, of the bytes `fill` writes into a + /// transmit buffer, and `fill` answers how many. Nothing is sent for 0. /// /// # Panics - /// With a transmit in flight: the caller asks [`Self::transmit_free`]. - pub fn transmit(&mut self, fill: impl FnOnce(&mut [u8]) -> usize) { - assert!(self.sent.is_none(), "vdagent: a transmit was made over the one in flight"); - let window = self.grant.window().sub(OFF_PORT_TX, LARGEST_TRANSMIT); + /// With every transmit in flight: the caller asks [`Self::can_transmit`]. + pub fn transmit(&mut self, fill: impl FnOnce(&mut [u8]) -> usize) -> usize { + let head = self.transmit_free.pop().expect("virtio-serial: a transmit was made with every buffer in flight"); + let at = self.layout.port_tx + head as usize * self.layout.tx_bytes; + let window = self.grant.window().sub(at, self.layout.tx_bytes); // SAFETY: the window is inside the grant, which lives as long as - // `self`; the device is not reading it, because no transmit is in - // flight and the buffer is published only after `fill` returns. - let len = fill(unsafe { std::slice::from_raw_parts_mut(window.as_ptr(), LARGEST_TRANSMIT) }); + // `self`; the device is not reading it, because its head is free and + // the buffer is published only after `fill` returns. + let len = fill(unsafe { std::slice::from_raw_parts_mut(window.as_ptr(), self.layout.tx_bytes) }); if len == 0 { - return; + self.transmit_free.push(head); + return 0; } - let frame = Buffer::readable(self.grant.device_addr(OFF_PORT_TX), len as u32); - let published = self.port_tx.publish(0, &[frame]); + assert!(len <= self.layout.tx_bytes, "virtio-serial: a transmit of {len} bytes into a buffer of {}", self.layout.tx_bytes); + let frame = Buffer::readable(self.grant.device_addr(at), len as u32); + let published = self.port_tx.publish(head, &[frame]); self.device.notify(published); - self.sent = Some(Instant::now()); + self.in_flight[head as usize] = Some((len as u32, Instant::now())); + len } } diff --git a/lib/vdagent/src/control.rs b/lib/virtio-serial/src/control.rs similarity index 53% rename from lib/vdagent/src/control.rs rename to lib/virtio-serial/src/control.rs index 8deca61deef..8b342cc7f0a 100644 --- a/lib/vdagent/src/control.rs +++ b/lib/virtio-serial/src/control.rs @@ -1,14 +1,22 @@ //! virtio-serial's control messages (virtio 1.2 §5.3.6.2): `struct //! virtio_console_control { le32 id; le16 event; le16 value; }`, each in a -//! buffer of its own on the control queues. +//! buffer of its own on the control queues, and a `PORT_NAME`'s name behind +//! its message. //! -//! **A `PORT_NAME`'s name is not decoded.** §5.3.6.2 has it follow the message -//! "without a NUL terminator", and QEMU 11.1.1 ends it with one; the port is -//! the declared [`crate::PORT_ID`] instead. +//! **A name is UTF-8 with no control character, at most [`MAX_NAME`] bytes.** +//! §5.3.6.2 has it follow the message "without a NUL terminator", and QEMU +//! 11.1.1 ends it with one, which is not part of it. /// One control message's bytes. pub const BYTES: usize = 8; +/// The longest port name read. +pub const MAX_NAME: usize = 64; + +/// A control receive buffer: a message and the longest name, its NUL, and a +/// byte more, so a name the buffer cut is one that fills it. +pub const BUFFER_BYTES: usize = BYTES + MAX_NAME + 2; + /// §5.3.6.2's events. #[derive(Clone, Copy, Debug, PartialEq, Eq)] pub enum Event { @@ -73,7 +81,7 @@ impl core::fmt::Display for Refused { /// The control message at the start of `bytes`, a buffer the device wrote, /// for a device of `max_nr_ports` ports. What follows the message — a -/// `PORT_NAME`'s name — is not read. +/// `PORT_NAME`'s name — is [`name`]'s. pub fn decode(bytes: &[u8], max_nr_ports: u32) -> Result { let Some(message) = bytes.get(..BYTES) else { return Err(Refused::Short { len: bytes.len() }); @@ -97,9 +105,51 @@ pub fn encode(control: Control) -> [u8; BYTES] { bytes } +/// Why the bytes behind a `PORT_NAME` are not a name. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum NameRefused { + Empty, + /// Past [`MAX_NAME`] bytes, or cut where the buffer ended. + Long, + /// Not UTF-8. + Encoding, + /// A control character, a NUL inside it among them, at this byte. + Control { at: usize }, +} + +impl core::fmt::Display for NameRefused { + fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result { + match self { + Self::Empty => write!(f, "an empty port name"), + Self::Long => write!(f, "a port name past {MAX_NAME} bytes"), + Self::Encoding => write!(f, "a port name that is not UTF-8"), + Self::Control { at } => write!(f, "a port name with a control character at byte {at}"), + } + } +} + +/// The name a `PORT_NAME` message in `buffer` carries: `buffer` is the +/// [`BUFFER_BYTES`] buffer's first bytes, as many as the device wrote. +pub fn name(buffer: &[u8]) -> Result<&str, NameRefused> { + let after = buffer.get(BYTES..).unwrap_or_default(); + let name = after.strip_suffix(&[0]).unwrap_or(after); + if name.is_empty() { + return Err(NameRefused::Empty); + } + if name.len() > MAX_NAME || buffer.len() >= BUFFER_BYTES { + return Err(NameRefused::Long); + } + let name = core::str::from_utf8(name).map_err(|_| NameRefused::Encoding)?; + match name.char_indices().find(|(_, c)| c.is_control()) { + Some((at, _)) => Err(NameRefused::Control { at }), + None => Ok(name), + } +} + #[cfg(test)] mod tests { use super::*; + use std::vec::Vec; /// §5.3.6.2's layout and numbering: `DEVICE_READY` is 0, `PORT_READY` 3, /// `PORT_OPEN` 6, little-endian. @@ -108,16 +158,44 @@ mod tests { assert_eq!(encode(Control { id: 0, event: Event::DeviceReady, value: 1 }), [0, 0, 0, 0, 0, 0, 1, 0]); assert_eq!(encode(Control { id: 1, event: Event::PortReady, value: 1 }), [1, 0, 0, 0, 3, 0, 1, 0]); assert_eq!(encode(Control { id: 1, event: Event::PortOpen, value: 0 }), [1, 0, 0, 0, 6, 0, 0, 0]); - let numbers: std::vec::Vec = Event::ALL.iter().map(|e| e.number()).collect(); + let numbers: Vec = Event::ALL.iter().map(|e| e.number()).collect(); assert_eq!(numbers, [0, 1, 2, 3, 4, 5, 6, 7]); } - /// A `PORT_NAME` with its name behind it decodes as the message alone. + fn port_name(name: &[u8]) -> Vec { + let mut buffer = Vec::from([1, 0, 0, 0, 7, 0, 1, 0]); + buffer.extend_from_slice(name); + buffer + } + + /// A `PORT_NAME` with its name behind it decodes as the message alone, + /// and its name with or without QEMU's NUL. #[test] - fn a_port_name_decodes_without_its_name() { - let mut buffer = std::vec::Vec::from([1, 0, 0, 0, 7, 0, 1, 0]); - buffer.extend_from_slice(b"com.redhat.spice.0\0"); + fn a_port_name_is_its_message_and_its_name() { + let buffer = port_name(b"com.redhat.spice.0\0"); assert_eq!(decode(&buffer, 2), Ok(Control { id: 1, event: Event::PortName, value: 1 })); + assert_eq!(name(&buffer), Ok("com.redhat.spice.0")); + assert_eq!(name(&port_name(b"org.toyos.console")), Ok("org.toyos.console")); + } + + #[test] + fn a_malformed_port_name_is_refused_by_what_is_wrong() { + assert_eq!(name(&port_name(b"")), Err(NameRefused::Empty)); + assert_eq!(name(&port_name(b"\0")), Err(NameRefused::Empty)); + assert_eq!(name(&[1, 0, 0, 0, 7, 0, 1]), Err(NameRefused::Empty)); + assert_eq!(name(&port_name(b"a\0b\0")), Err(NameRefused::Control { at: 1 })); + assert_eq!(name(&port_name(b"tab\there")), Err(NameRefused::Control { at: 3 })); + assert_eq!(name(&port_name(b"\xff\xfe")), Err(NameRefused::Encoding)); + assert_eq!(name(&port_name(&[b'a'; MAX_NAME])), Ok(core::str::from_utf8(&[b'a'; MAX_NAME]).unwrap())); + assert_eq!(name(&port_name(&[b'a'; MAX_NAME + 1])), Err(NameRefused::Long)); + } + + /// A name the receive buffer cut fills the whole buffer, so a buffer the + /// device filled is never read as a shorter name. + #[test] + fn a_name_the_buffer_cut_is_refused() { + let cut = &port_name(&[b'a'; 200])[..BUFFER_BYTES]; + assert_eq!(name(cut), Err(NameRefused::Long)); } #[test] diff --git a/lib/virtio-serial/src/lib.rs b/lib/virtio-serial/src/lib.rs new file mode 100644 index 00000000000..1f1c40a69e4 --- /dev/null +++ b/lib/virtio-serial/src/lib.rs @@ -0,0 +1,40 @@ +//! virtio-serial as this project's two drivers of it speak it: the shared +//! clipboard's agent (`/system/bin/vdagent`) and the VM's console +//! (`/system/bin/virtio-console`). +//! +//! **Both are the same device, `1af4:1043`, and only a port's name says which +//! a function is**: [`probe`] reads the names the device gives its ports, and +//! a driver that finds no port of its own lets the function go to the next +//! row that drives one. [`control`] is the control messages and the names, +//! decoded bounded; `toyos-virtio-serial-claim` brings the function up over +//! its claim. +//! +//! What it cites: *Virtual I/O Device (VIRTIO) Version 1.2*, OASIS Committee +//! Specification 01, §5.3 for the console device and its control messages. +//! +//! Pure: `core` and `alloc`, no `unsafe`, no I/O. + +#![no_std] +#![forbid(unsafe_code)] + +extern crate alloc; +#[cfg(test)] +extern crate std; + +pub mod control; +pub mod probe; + +/// The VM console's port, as the launch names it and its driver probes for. +pub const CONSOLE_PORT: &str = "org.toyos.console"; + +/// Every virtio-serial function the launch makes has `max_ports` this: port 0, +/// which QEMU 11.1.1 reserves for a `virtconsole` (`hw/char/virtio-console.c`) +/// and nothing here uses, and the function's one named port. +pub const MAX_PORTS: u32 = 2; + +/// The id of a function's one named port, `nr=` at the launch. +pub const PORT_ID: u32 = 1; + +/// How many control receive buffers a driver keeps posted: virtio-serial +/// drops a control event that finds none. +pub const CONTROL_BUFFERS: usize = 8; diff --git a/lib/virtio-serial/src/probe.rs b/lib/virtio-serial/src/probe.rs new file mode 100644 index 00000000000..cb2470a081e --- /dev/null +++ b/lib/virtio-serial/src/probe.rs @@ -0,0 +1,250 @@ +//! Which function is a driver's: the one whose device names a port with the +//! driver's name. +//! +//! **A port is named once it is ready** (§5.3.6.2): the device adds each port +//! after the driver's `DEVICE_READY`, the probe answers each `DEVICE_ADD` with +//! `PORT_READY`, and the device names the port after that. The function is +//! the driver's once a port carries its name, at [`crate::PORT_ID`], and +//! another driver's once every port it added is named and none is. +//! +//! **What the device says about its ports is refused when it contradicts +//! itself** — a port added twice, named twice or named before it was added, +//! or a name [`control::name`] refuses — and a driver ends on a refusal by its +//! name: a device that misnames its ports is not one to drive. + +use alloc::string::String; +use alloc::vec::Vec; + +use crate::control::{self, Control, Event, NameRefused}; +use crate::{MAX_PORTS, PORT_ID}; + +/// What the probe knows of one port id. +#[derive(Clone, Debug, PartialEq, Eq)] +enum Port { + Absent, + Added, + Named(String), +} + +/// A function's ports, as its control messages describe them. +pub struct Probe<'w> { + wanted: &'w str, + ports: [Port; MAX_PORTS as usize], +} + +/// What the probe makes of one control buffer. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Step { + /// Send the device this. + Reply(Control), + /// A message about a port the probe knows, the driver's to act on. + Driver(Control), + Nothing, +} + +/// Whose the function is, so far. +#[derive(Clone, Debug, PartialEq, Eq)] +pub enum Verdict { + Probing, + /// A port at [`PORT_ID`] carries the driver's name. + Ours, + /// Every port the device added is named, by these names, and none is the + /// driver's. + Foreign(Vec), +} + +/// Why a control buffer is not believed. +#[derive(Clone, Debug, PartialEq, Eq)] +pub enum Refused { + Control(control::Refused), + Name { id: u32, why: NameRefused }, + /// An event about a port the device never added. + Unadded { id: u32, event: Event }, + AddedTwice { id: u32 }, + NamedTwice { id: u32 }, + /// The driver's name on a port the launch does not put it on. + Elsewhere { id: u32 }, +} + +impl core::fmt::Display for Refused { + fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result { + match self { + Self::Control(why) => write!(f, "{why}"), + Self::Name { id, why } => write!(f, "port {id}'s name: {why}"), + Self::Unadded { id, event } => write!(f, "{event:?} for port {id}, which the device never added"), + Self::AddedTwice { id } => write!(f, "the device added port {id} twice"), + Self::NamedTwice { id } => write!(f, "the device named port {id} twice"), + Self::Elsewhere { id } => write!(f, "the device gave port {id} the name the launch puts on port {PORT_ID}"), + } + } +} + +impl<'w> Probe<'w> { + pub fn new(wanted: &'w str) -> Self { + Self { wanted, ports: [const { Port::Absent }; MAX_PORTS as usize] } + } + + /// One control buffer the device used: its first bytes, as many as it + /// wrote. + pub fn take(&mut self, buffer: &[u8]) -> Result { + let message = control::decode(buffer, MAX_PORTS).map_err(Refused::Control)?; + let Control { id, event, .. } = message; + let wanted = self.wanted; + let port = &mut self.ports[id as usize]; + let absent = *port == Port::Absent; + match event { + Event::DeviceAdd if absent => { + *port = Port::Added; + Ok(Step::Reply(Control { id, event: Event::PortReady, value: 1 })) + } + Event::DeviceAdd => Err(Refused::AddedTwice { id }), + Event::PortName if *port == Port::Added => { + let name = control::name(buffer).map_err(|why| Refused::Name { id, why })?; + if name == wanted && id != PORT_ID { + return Err(Refused::Elsewhere { id }); + } + *port = Port::Named(name.into()); + Ok(Step::Nothing) + } + Event::PortName if !absent => Err(Refused::NamedTwice { id }), + _ if absent => Err(Refused::Unadded { id, event }), + Event::DeviceRemove if *port == Port::Named(wanted.into()) => Ok(Step::Driver(message)), + Event::DeviceRemove => { + *port = Port::Absent; + Ok(Step::Nothing) + } + _ => Ok(Step::Driver(message)), + } + } + + pub fn verdict(&self) -> Verdict { + let named = || self.ports.iter().filter_map(|port| match port { + Port::Named(name) => Some(name), + Port::Absent | Port::Added => None, + }); + if named().any(|name| name == self.wanted) { + return Verdict::Ours; + } + let unnamed = self.ports.contains(&Port::Added); + match named().next() { + Some(_) if !unnamed => Verdict::Foreign(named().cloned().collect()), + _ => Verdict::Probing, + } + } + + /// Whether the driver's port is the one `id` names. + pub fn is_ours(&self, id: u32) -> bool { + matches!(self.ports.get(id as usize), Some(Port::Named(name)) if name == self.wanted) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use alloc::string::ToString; + use alloc::vec; + + const CONSOLE: &str = "org.toyos.console"; + const AGENT: &str = "com.redhat.spice.0"; + + fn message(id: u32, event: Event) -> Vec { + control::encode(Control { id, event, value: 1 }).to_vec() + } + + fn named(id: u32, name: &str) -> Vec { + let mut buffer = message(id, Event::PortName); + buffer.extend_from_slice(name.as_bytes()); + buffer.push(0); + buffer + } + + /// QEMU 11.1.1's order: the port added after `DEVICE_READY`, named after + /// the driver's `PORT_READY`, then opened. + #[test] + fn a_port_with_the_drivers_name_makes_the_function_its_own() { + let mut probe = Probe::new(CONSOLE); + assert_eq!(probe.verdict(), Verdict::Probing); + assert_eq!( + probe.take(&message(1, Event::DeviceAdd)), + Ok(Step::Reply(Control { id: 1, event: Event::PortReady, value: 1 })) + ); + assert_eq!(probe.verdict(), Verdict::Probing); + assert_eq!(probe.take(&named(1, CONSOLE)), Ok(Step::Nothing)); + assert_eq!(probe.verdict(), Verdict::Ours); + assert!(probe.is_ours(1) && !probe.is_ours(0)); + let open = Control { id: 1, event: Event::PortOpen, value: 1 }; + assert_eq!(probe.take(&message(1, Event::PortOpen)), Ok(Step::Driver(open))); + } + + /// The clipboard's function, probed by the console's driver: another + /// driver's once its one port is named. + #[test] + fn a_function_whose_ports_are_all_named_otherwise_is_anothers() { + let mut probe = Probe::new(CONSOLE); + probe.take(&message(1, Event::DeviceAdd)).expect("the port is added"); + probe.take(&named(1, AGENT)).expect("the port is named"); + assert_eq!(probe.verdict(), Verdict::Foreign(vec![AGENT.to_string()])); + } + + /// A port still unnamed holds the verdict, whatever the others are. + #[test] + fn an_unnamed_port_leaves_the_function_undecided() { + let mut probe = Probe::new(AGENT); + probe.take(&message(0, Event::DeviceAdd)).expect("port 0 is added"); + probe.take(&message(1, Event::DeviceAdd)).expect("port 1 is added"); + probe.take(&named(0, "some.other.port")).expect("port 0 is named"); + assert_eq!(probe.verdict(), Verdict::Probing); + probe.take(&named(1, AGENT)).expect("port 1 is named"); + assert_eq!(probe.verdict(), Verdict::Ours); + } + + #[test] + fn a_device_that_contradicts_itself_is_refused() { + let mut probe = Probe::new(CONSOLE); + assert_eq!(probe.take(&named(1, CONSOLE)), Err(Refused::Unadded { id: 1, event: Event::PortName })); + assert_eq!(probe.take(&message(1, Event::PortOpen)), Err(Refused::Unadded { id: 1, event: Event::PortOpen })); + probe.take(&message(1, Event::DeviceAdd)).expect("the port is added"); + assert_eq!(probe.take(&message(1, Event::DeviceAdd)), Err(Refused::AddedTwice { id: 1 })); + probe.take(&named(1, CONSOLE)).expect("the port is named"); + assert_eq!(probe.take(&named(1, CONSOLE)), Err(Refused::NamedTwice { id: 1 })); + assert_eq!(probe.take(&message(2, Event::DeviceAdd)), Err(Refused::Control(control::Refused::Port { id: 2, max_nr_ports: 2 }))); + } + + /// A malformed name is a refusal about the port that carried it, and + /// leaves the port unnamed. + #[test] + fn a_malformed_port_name_is_refused() { + let mut probe = Probe::new(CONSOLE); + probe.take(&message(1, Event::DeviceAdd)).expect("the port is added"); + let mut cut = message(1, Event::PortName); + cut.extend_from_slice(&[b'x'; control::BUFFER_BYTES]); + cut.truncate(control::BUFFER_BYTES); + assert_eq!(probe.take(&cut), Err(Refused::Name { id: 1, why: NameRefused::Long })); + let mut control_byte = message(1, Event::PortName); + control_byte.extend_from_slice(b"org.toyos.\nconsole"); + assert_eq!(probe.take(&control_byte), Err(Refused::Name { id: 1, why: NameRefused::Control { at: 10 } })); + assert_eq!(probe.verdict(), Verdict::Probing); + } + + /// The driver's name is where the launch puts it, or the device is not + /// the one the launch made. + #[test] + fn the_drivers_name_on_another_port_is_refused() { + let mut probe = Probe::new(CONSOLE); + probe.take(&message(0, Event::DeviceAdd)).expect("port 0 is added"); + assert_eq!(probe.take(&named(0, CONSOLE)), Err(Refused::Elsewhere { id: 0 })); + } + + /// The removal of the driver's port is the driver's to answer; another + /// port's removal only takes it out of the verdict. + #[test] + fn a_removal_is_the_drivers_only_for_its_own_port() { + let mut probe = Probe::new(CONSOLE); + probe.take(&message(0, Event::DeviceAdd)).expect("port 0 is added"); + probe.take(&message(1, Event::DeviceAdd)).expect("port 1 is added"); + probe.take(&named(1, CONSOLE)).expect("port 1 is named"); + assert_eq!(probe.take(&message(0, Event::DeviceRemove)), Ok(Step::Nothing)); + let removed = Control { id: 1, event: Event::DeviceRemove, value: 1 }; + assert_eq!(probe.take(&message(1, Event::DeviceRemove)), Ok(Step::Driver(removed))); + } +} diff --git a/sdk/abi/src/syscall.rs b/sdk/abi/src/syscall.rs index dcb934f5cfb..d8ce7d4f5f9 100644 --- a/sdk/abi/src/syscall.rs +++ b/sdk/abi/src/syscall.rs @@ -321,6 +321,8 @@ pub const SYS_VM_DISCARD: u64 = 141; pub const SYS_VM_DECOMMIT: u64 = 142; /// A memory object over part of another's pages. See [`shm_sub`]. pub const SYS_SHM_SUB: u64 = 143; +/// Whether a console holder carries the log off the machine. See [`console_carried`]. +pub const SYS_CONSOLE_CARRIED: u64 = 144; /// Bins in the per-process syscall profile — one for every number this ABI /// issues, and one at the end for every number it does not. @@ -328,13 +330,13 @@ pub const SYS_SHM_SUB: u64 = 143; /// **The profile's parts sum to its total, and that is the whole requirement.** /// A bin array narrower than the ABI reaches drops calls out of the line while /// the total goes on counting them. -pub const SYSCALL_PROFILE_BINS: usize = 145; +pub const SYSCALL_PROFILE_BINS: usize = 146; /// Where a number this ABI does not issue is counted. Merging is a degradation /// a reader can see in the line; dropping is one nobody can. pub const SYSCALL_PROFILE_OTHER: usize = SYSCALL_PROFILE_BINS - 1; -const _: () = assert!(SYS_SHM_SUB < SYSCALL_PROFILE_OTHER as u64); +const _: () = assert!(SYS_CONSOLE_CARRIED < SYSCALL_PROFILE_OTHER as u64); pub const WNOHANG: u64 = 1; /// Every bit `SYS_PROCESS_WAIT`'s flag word defines; a wait carrying another @@ -2070,6 +2072,19 @@ pub fn shm_sub(shm: RawHandle, offset: u64, len: u64) -> Result Result<(), SyscallError> { + check(syscall(SYS_CONSOLE_CARRIED, console.0 as u64, elsewhere as u64, 0, 0)).map(drop) +} + /// The machine's [header](SYSINFO_HEADER_SIZE), then as much of the process /// roster as `buf` has room for. Answers the number of bytes written. /// diff --git a/sdk/toyos/src/lib.rs b/sdk/toyos/src/lib.rs index 1d74c8ef53f..8fe19a9daed 100644 --- a/sdk/toyos/src/lib.rs +++ b/sdk/toyos/src/lib.rs @@ -220,6 +220,13 @@ impl Console { pub fn write_nonblock(&self, buf: &[u8]) -> Result { self.0.write_nonblock(buf) } + + /// Say whether this holder carries the log off the machine by another + /// channel (`toyos_abi::syscall::console_carried`); the console's end + /// lets go of it. + pub fn carry_elsewhere(&self, elsewhere: bool) -> Result<(), toyos_abi::syscall::SyscallError> { + toyos_abi::syscall::console_carried(self.0.raw(), elsewhere) + } } impl AsHandle for Console { diff --git a/system/logkeeper/src/carrier.rs b/system/logkeeper/src/carrier.rs new file mode 100644 index 00000000000..dfa48892399 --- /dev/null +++ b/system/logkeeper/src/carrier.rs @@ -0,0 +1,449 @@ +//! The machine's console carrier, asked for on [`toyos_logstream::CARRIER`]: +//! the one reader the console UART goes quiet for. +//! +//! **The UART goes quiet once the carrier's channel has taken the log's first +//! byte, and comes back once the carrier goes or the machine stops** +//! ([`Carriage`]). The carrier is handed the boot from its first line like any +//! reader, with no stall bound: a slow channel is not a failed one. It says on +//! its connection how much of its pipe its channel has taken ([`DELIVERED`]), +//! which is how the stop knows the channel has the log up to the flush before +//! the UART takes the rest ([`Carrier::stop`]). +//! +//! **This program's own end is the kernel's to answer**: the console it says +//! the log is carried on goes with this process, and the kernel gives the log +//! back to the UART with it, so a crash here cannot leave the UART quiet. +//! +//! What the carrier's channel had not taken when the carrier went is on +//! neither: `/log` has it. + +use std::sync::{Arc, Condvar, Mutex, MutexGuard}; +use std::time::{Duration, Instant}; + +use toyos::ipc::{Connection, FrameRx, RxStep}; +use toyos::poller::{Poller, READABLE, WRITABLE}; +use toyos::port::Acceptor; +use toyos::{say, Console, Pipe}; +use toyos_abi::syscall::SyscallError; +use toyos_logstream::{CARRY, DELIVERED}; + +use crate::serve::Hub; + +/// How long the stop waits for the carrier's channel to take the log up to +/// the flush: policy, well inside the supervisor's flush bound, and far past +/// what a channel that takes bytes at all needs for the last rounds. +const STOP_WAIT: Duration = Duration::from_secs(2); + +/// What the kernel is to be told about the console UART. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Tell { + /// The log is carried elsewhere: the UART takes only the stop's and a + /// fatal path's lines. + Quiet, + /// The UART carries the log again. + Loud, +} + +/// Whether the UART is quiet, decided from what the carrier and the stop do. +/// Pure: each event answers what the kernel is told, if anything. +#[derive(Debug, Default)] +pub struct Carriage { + /// The carrier admitted, by the number of its admission. + carrier: Option, + admitted: u64, + /// Bytes of its pipe the carrier's channel has taken. + delivered: u64, + stopping: bool, + quiet: bool, +} + +/// Why a carrier is refused. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Refused { + /// One is already admitted. + Busy, + /// A count below one it gave already, or past the bytes it was sent. + Count { said: u64, delivered: u64, sent: u64 }, +} + +impl Carriage { + /// A carrier asks: its admission's number, or [`Refused::Busy`]. + pub fn admit(&mut self) -> Result { + if self.carrier.is_some() { + return Err(Refused::Busy); + } + self.admitted += 1; + self.carrier = Some(self.admitted); + self.delivered = 0; + Ok(self.admitted) + } + + /// Carrier `who`'s channel has taken `said` bytes of the `sent` it was + /// sent. + pub fn delivered(&mut self, who: u64, said: u64, sent: u64) -> Result, Refused> { + if self.carrier != Some(who) { + return Ok(None); + } + if said < self.delivered || said > sent { + return Err(Refused::Count { said, delivered: self.delivered, sent }); + } + self.delivered = said; + Ok(self.settle()) + } + + /// Carrier `who` went: by its end, or by a count refused. + pub fn gone(&mut self, who: u64) -> Option { + if self.carrier != Some(who) { + return None; + } + self.carrier = None; + self.settle() + } + + /// The machine stops: the UART takes what the log says from here. + pub fn stop(&mut self) -> Option { + self.stopping = true; + self.settle() + } + + /// The stop was refused. + pub fn resume(&mut self) -> Option { + self.stopping = false; + self.settle() + } + + pub fn quiet(&self) -> bool { + self.quiet + } + + fn settle(&mut self) -> Option { + let quiet = self.carrier.is_some() && self.delivered > 0 && !self.stopping; + if quiet == self.quiet { + return None; + } + self.quiet = quiet; + Some(if quiet { Tell::Quiet } else { Tell::Loud }) + } +} + +/// The carrier's state, shared by its threads and the main loop. +pub struct Carrier { + console: Arc, + state: Mutex, + /// Posted when the carrier's channel takes bytes, its feed moves, or it goes. + moved: Condvar, +} + +struct State { + carriage: Carriage, + /// Bytes written into the carrier's pipe, and the replay offset they reach. + sent: u64, + at: u64, +} + +impl Carrier { + /// Serve the carrier on `acceptor` for the life of the process, telling + /// the kernel through `console`. + pub fn serve(acceptor: Acceptor, console: Arc, hub: Arc) -> Arc { + let carrier = Arc::new(Self { + console, + state: Mutex::new(State { carriage: Carriage::default(), sent: 0, at: 0 }), + moved: Condvar::new(), + }); + let theirs = Arc::clone(&carrier); + std::thread::Builder::new() + .name("log-carrier".into()) + .spawn(move || theirs.run(&acceptor, &hub)) + .expect("logkeeper: the carrier's thread could not be started"); + carrier + } + + /// Whether the UART is quiet, so program lines are not written to it. + pub fn quiet(&self) -> bool { + self.lock().carriage.quiet() + } + + /// The machine stops, and `end` is how long the boot is at the flush: + /// wait, bounded, for the carrier's channel to take it all, then give the + /// UART what the log says from here. + pub fn stop(&self, end: u64) { + let began = Instant::now(); + let mut state = self.lock(); + loop { + let caught_up = state.at >= end && state.carriage.delivered >= state.sent; + if state.carriage.carrier.is_none() || caught_up { + break; + } + let Some(left) = STOP_WAIT.checked_sub(began.elapsed()) else { + say!( + "logkeeper: the console carrier's channel had taken {} of {} bytes {} ms into the stop; \ + the UART takes the stop's lines, and /log has the rest", + state.carriage.delivered, + state.sent, + STOP_WAIT.as_millis() + ); + break; + }; + state = self.moved.wait_timeout(state, left).expect("logkeeper: the carrier's state is poisoned").0; + } + let tell = state.carriage.stop(); + self.tell(tell); + } + + /// The stop was refused: the carrier carries the log again, past what + /// the UART carried meanwhile ([`Hub::append_carried`]). + pub fn resume(&self) { + let mut state = self.lock(); + let tell = state.carriage.resume(); + self.tell(tell); + } + + fn lock(&self) -> MutexGuard<'_, State> { + self.state.lock().expect("logkeeper: the carrier's state is poisoned") + } + + /// Tell the kernel, under the state's lock, so two threads' changes reach + /// it in the order they were decided. + fn tell(&self, tell: Option) { + let Some(tell) = tell else { return }; + match self.console.carry_elsewhere(tell == Tell::Quiet) { + Ok(()) => match tell { + Tell::Quiet => say!("logkeeper: the console carrier's channel takes the log, so the UART is quiet"), + Tell::Loud => say!("logkeeper: the UART carries the log again"), + }, + // A fatal path or the stop gave the UART the log for good. + Err(SyscallError::NotSupported) => {} + Err(e) => panic!("logkeeper: the kernel refused to be told where the log is carried: {e:?}"), + } + } + + /// The acceptor and the admitted carrier's connection. + fn run(self: &Arc, acceptor: &Acceptor, hub: &Arc) -> ! { + const ACCEPT: u64 = 0; + const CONNECTION: u64 = 1; + let poller = Poller::new(2); + let mut admitted: Option<(u64, Connection, FrameRx<8>)> = None; + // A connection that has not asked yet. + let mut asking: Option<(Connection, FrameRx<0>)> = None; + loop { + poller.watch(acceptor, READABLE, ACCEPT); + if let Some((_, conn, _)) = &admitted { + poller.watch(conn, READABLE, CONNECTION); + } else if let Some((conn, _)) = &asking { + poller.watch(conn, READABLE, CONNECTION); + } + let mut ready = [false; 2]; + poller.wait(1, u64::MAX, |token| ready[token as usize] = true); + + if ready[ACCEPT as usize] { + if let Ok(conn) = acceptor.accept() { + if admitted.is_some() || asking.is_some() { + say!("logkeeper: refusing a second console carrier: one is served"); + } else { + asking = Some((conn, FrameRx::new())); + } + } + } + if !ready[CONNECTION as usize] { + continue; + } + if let Some((conn, mut rx)) = asking.take() { + match rx.pump(&conn) { + RxStep::Idle => asking = Some((conn, rx)), + RxStep::Frame { msg_type: CARRY, .. } => admitted = self.admit(conn, hub), + RxStep::Eof | RxStep::Malformed | RxStep::Frame { .. } => {} + } + continue; + } + let Some((who, conn, mut rx)) = admitted.take() else { continue }; + if self.read(who, &conn, &mut rx) { + admitted = Some((who, conn, rx)); + } + } + } + + /// Hand the carrier on `conn` its pipe, and feed it from the boot's first + /// line on a thread of its own. + fn admit(self: &Arc, conn: Connection, hub: &Arc) -> Option<(u64, Connection, FrameRx<8>)> { + let who = match self.lock().carriage.admit() { + Ok(who) => who, + Err(why) => { + say!("logkeeper: refusing a console carrier: {why:?}"); + return None; + } + }; + let Some(pipe) = hub.hand_over(&conn) else { + self.went(who); + return None; + }; + { + let mut state = self.lock(); + state.sent = 0; + state.at = 0; + } + let (carrier, hub) = (Arc::clone(self), Arc::clone(hub)); + let fed = std::thread::Builder::new().name("log-carried".into()).spawn(move || { + carrier.feed(who, &pipe, &hub); + carrier.went(who); + }); + if let Err(e) = fed { + say!("logkeeper: no thread for the console carrier: {e}"); + self.went(who); + return None; + } + Some((who, conn, FrameRx::new())) + } + + /// Every frame carrier `who` sent: whether it is still served. + fn read(&self, who: u64, conn: &Connection, rx: &mut FrameRx<8>) -> bool { + loop { + match rx.pump(conn) { + RxStep::Idle => return true, + RxStep::Frame { msg_type: DELIVERED, payload_len: 8, .. } => { + let said = u64::from_le_bytes(rx.payload(8).try_into().expect("eight bytes")); + let mut state = self.lock(); + let sent = state.sent; + match state.carriage.delivered(who, said, sent) { + Ok(tell) => { + self.tell(tell); + drop(state); + self.moved.notify_all(); + } + Err(why) => { + drop(state); + say!("logkeeper: letting the console carrier go: {why:?}"); + self.went(who); + return false; + } + } + } + RxStep::Eof | RxStep::Malformed | RxStep::Frame { .. } => { + self.went(who); + return false; + } + } + } + } + + fn went(&self, who: u64) { + let mut state = self.lock(); + let tell = state.carriage.gone(who); + self.tell(tell); + drop(state); + self.moved.notify_all(); + } + + /// Write the boot into carrier `who`'s pipe until the pipe's reader goes. + fn feed(&self, who: u64, pipe: &Pipe, hub: &Hub) { + let poller = Poller::new(1); + let mut at = 0u64; + loop { + let chunk = hub.next_for_the_carrier(&mut at); + // Counted before the pipe has it, so no count the carrier gives + // of these bytes is ever past what it was sent. + { + let mut state = self.lock(); + if state.carriage.carrier != Some(who) { + return; + } + state.sent += chunk.len() as u64; + } + let mut written = 0; + while written < chunk.len() { + match pipe.write_nonblock(&chunk[written..]) { + Ok(n) => written += n, + Err(SyscallError::WouldBlock) => { + // No bound: a slow channel is not a failed one, and + // the pipe's reader going ends the wait. + poller.watch(pipe, WRITABLE, 0); + poller.wait(1, u64::MAX, |_| {}); + } + Err(_) => return, + } + } + self.lock().at = at; + self.moved.notify_all(); + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + /// The UART goes quiet at the carrier's first delivered byte, not at its + /// admission: a carrier whose channel takes nothing has carried nothing. + #[test] + fn the_uart_goes_quiet_once_the_carriers_channel_takes_a_byte() { + let mut carriage = Carriage::default(); + let who = carriage.admit().expect("the first carrier is admitted"); + assert!(!carriage.quiet()); + assert_eq!(carriage.delivered(who, 0, 100), Ok(None)); + assert_eq!(carriage.delivered(who, 40, 100), Ok(Some(Tell::Quiet))); + assert_eq!(carriage.delivered(who, 100, 100), Ok(None)); + assert!(carriage.quiet()); + } + + /// The carrier's end, its driver's death among them, gives the log back + /// to the UART. + #[test] + fn the_carriers_end_gives_the_log_back_to_the_uart() { + let mut carriage = Carriage::default(); + let who = carriage.admit().expect("the first carrier is admitted"); + carriage.delivered(who, 1, 1).expect("a count it was sent"); + assert_eq!(carriage.gone(who), Some(Tell::Loud)); + assert!(!carriage.quiet()); + assert_eq!(carriage.gone(who), None, "one end is answered once"); + // A later carrier carries it again from its own first byte. + let next = carriage.admit().expect("the next carrier is admitted"); + assert_ne!(next, who); + assert_eq!(carriage.delivered(who, 5, 5), Ok(None), "the gone carrier's counts are not this one's"); + assert_eq!(carriage.delivered(next, 5, 5), Ok(Some(Tell::Quiet))); + } + + /// A carrier that went before it delivered anything leaves the UART as it was. + #[test] + fn a_carrier_that_carried_nothing_changes_nothing() { + let mut carriage = Carriage::default(); + let who = carriage.admit().expect("the first carrier is admitted"); + assert_eq!(carriage.gone(who), None); + } + + #[test] + fn a_second_carrier_is_refused_while_one_is_served() { + let mut carriage = Carriage::default(); + carriage.admit().expect("the first carrier is admitted"); + assert_eq!(carriage.admit(), Err(Refused::Busy)); + } + + /// A count the carrier could not have, or one going backwards, is refused, + /// and the main loop lets the carrier go on it. + #[test] + fn a_count_past_what_was_sent_or_backwards_is_refused() { + let mut carriage = Carriage::default(); + let who = carriage.admit().expect("the first carrier is admitted"); + assert_eq!( + carriage.delivered(who, 11, 10), + Err(Refused::Count { said: 11, delivered: 0, sent: 10 }) + ); + carriage.delivered(who, 8, 10).expect("a count it was sent"); + assert_eq!(carriage.delivered(who, 7, 10), Err(Refused::Count { said: 7, delivered: 8, sent: 10 })); + assert_eq!(carriage.gone(who), Some(Tell::Loud)); + } + + /// The stop gives the UART the log, and a refused stop gives it back to + /// a carrier that is still there. + #[test] + fn the_stop_takes_the_log_to_the_uart_and_a_refused_one_gives_it_back() { + let mut carriage = Carriage::default(); + let who = carriage.admit().expect("the first carrier is admitted"); + carriage.delivered(who, 3, 3).expect("a count it was sent"); + assert_eq!(carriage.stop(), Some(Tell::Loud)); + assert_eq!(carriage.delivered(who, 4, 4), Ok(None), "nothing quiets the UART while the machine stops"); + assert_eq!(carriage.resume(), Some(Tell::Quiet)); + + assert_eq!(carriage.stop(), Some(Tell::Loud)); + assert_eq!(carriage.gone(who), None); + assert_eq!(carriage.resume(), None, "no carrier is left to carry it"); + assert!(!carriage.quiet()); + } +} diff --git a/system/logkeeper/src/main.rs b/system/logkeeper/src/main.rs index ed0b6c453f6..331e90580ce 100644 --- a/system/logkeeper/src/main.rs +++ b/system/logkeeper/src/main.rs @@ -48,13 +48,19 @@ //! into what the file already has, and the console takes only the whole lines //! it has room for — the rest waits here, bounded ([`CONSOLE_BYTES`]). //! +//! **One reader can carry the console instead** ([`carrier`]): a VM's console +//! channel, asked for on [`toyos_logstream::CARRIER`]. Once its channel takes +//! the log, the kernel is told the UART is quiet, and no program line is +//! written to it; its end, or the stop, gives the console back. +//! //! # Its whole authority //! //! One `SysCap` duplicate carrying `Rights::LOG | Rights::WAIT`, which its //! manifest row asks for by the name `logread`; the origins acceptor and the //! writable console the supervisor endows it; and what its row adds — the `log` acceptor, where this //! machine's readers ask for the log and `inspect` asks where it is going -//! ([`inspect`]'s module, which grants nothing). It claims no device and can +//! ([`inspect`]'s module, which grants nothing), and the `log-carrier` +//! acceptor, where the console's carrier asks. It claims no device and can //! name no process. Writing files is ambient — a known residual of the //! capability endowment, and not this program's to close. //! @@ -71,6 +77,7 @@ //! refused is the supervisor's [`toyos_logstream::RESUME`], and what was held is written //! then. +mod carrier; mod inspect; mod origin; mod policy; @@ -90,11 +97,12 @@ use toyos::syscap::SysCap; use toyos::{Console, Pipe}; use toyos_abi::syscall::SyscallError; use toyos_logstream::{ - ProgramLine, Registration, Tag, CONSOLE, FLUSH, FLUSHED, LOG_OPENED, MAX_TAG, ORIGINS, REGISTER, RESUME, - SERVICE, + ProgramLine, Registration, Tag, CARRIER, CONSOLE, FLUSH, FLUSHED, LOG_OPENED, MAX_TAG, ORIGINS, REGISTER, + RESUME, SERVICE, }; use toyos_wallclock::Civil; +use carrier::Carrier; use origin::{Origin, Said}; use policy::{Step, LOG_WRITE_BUDGET}; use store::{Volume, DIR, MAX_LOG_BYTES, MAX_LOG_FILES}; @@ -139,6 +147,7 @@ fn main() { let Some(console) = Endowments::get().take::(CONSOLE) else { panic!("logkeeper: the supervisor endowed no `{CONSOLE}`, so no program's line can reach the console"); }; + let console = Arc::new(console); // The supervisor connected before it started anything, so this is already queued. let from_supervisor: Connection = acceptor.accept().expect("logkeeper: the supervisor's origins connection"); @@ -164,6 +173,7 @@ fn main() { if let Some(acceptor) = endow::acceptor(SERVICE) { inspect::serve(acceptor, Arc::clone(&published), Arc::clone(&hub)); } + let carrier = endow::acceptor(CARRIER).map(|acceptor| Carrier::serve(acceptor, Arc::clone(&console), Arc::clone(&hub))); let words = toyos::log::cursor_words(&cap).unwrap_or_else(|e| panic!("logkeeper: SYS_LOG_READ sized no cursor ({e:?})")); @@ -173,6 +183,7 @@ fn main() { from_supervisor, supervisor_rx: ipc::FrameRx::new(), console, + carrier, console_held: Vec::new(), console_unshown: 0, origins: Vec::new(), @@ -194,7 +205,8 @@ struct Log { cap: SysCap, from_supervisor: Connection, supervisor_rx: ipc::FrameRx<{ 4 + MAX_TAG }>, - console: Console, + console: Arc, + carrier: Option>, /// Program lines the console has not taken yet, whole lines only. console_held: Vec, console_unshown: u64, @@ -259,7 +271,7 @@ impl Log { poller.watch(&self.cap, READABLE, KERNEL_TOKEN); poller.watch(&self.from_supervisor, READABLE, ORIGINS_TOKEN); if !self.console_held.is_empty() { - poller.watch(&self.console, WRITABLE, CONSOLE_TOKEN); + poller.watch(&*self.console, WRITABLE, CONSOLE_TOKEN); } for (i, origin) in self.origins.iter().enumerate() { poller.watch(&origin.alive, READABLE, ORIGIN_BASE + i as u64); @@ -498,6 +510,7 @@ impl Log { return; } lines.sort_by_key(|line| line.at_ns); + let carried = self.carried(); let mut file = String::new(); for line in &lines { match &line.kind { @@ -519,9 +532,11 @@ impl Log { text: &said.text, }; file.push_str(&format!("{line}\n")); - line.stamp = ""; - let console = format!("{line}\n"); - self.console_line(console.as_bytes()); + if !carried { + line.stamp = ""; + let console = format!("{line}\n"); + self.console_line(console.as_bytes()); + } } } } @@ -565,8 +580,19 @@ impl Log { held.extend_from_slice(line); } - /// As many held lines as the console takes now; the rest wait for its room. + /// Whether the console's carrier carries the program lines, so the UART + /// takes none. + fn carried(&self) -> bool { + self.carrier.as_ref().is_some_and(|carrier| carrier.quiet()) + } + + /// As many held lines as the console takes now; the rest wait for its + /// room. None while the carrier carries them, which has them all. fn feed_console(&mut self) { + if self.carried() { + self.console_held.clear(); + self.console_unshown = 0; + } while !self.console_held.is_empty() { match self.console.write_nonblock(&self.console_held) { Ok(0) | Err(SyscallError::WouldBlock) => break, @@ -661,6 +687,11 @@ impl Log { fn flushed(&mut self) { let refused = self.sync().err(); self.answered(refused); + // The console's carrier takes the log up to here, and the UART what + // the machine says from here as it stops. + if let Some(carrier) = &self.carrier { + carrier.stop(self.hub.end()); + } self.stopping = Some(Stopping { text: String::new(), unwritten: 0 }); self.feed_console(); if let Err(e) = self.from_supervisor.signal(FLUSHED) { @@ -683,7 +714,10 @@ impl Log { ); } self.to_volume(held.text.as_bytes(), true); - self.hub.append(held.text.as_bytes()); + self.hub.append_carried(held.text.as_bytes()); + if let Some(carrier) = &self.carrier { + carrier.resume(); + } } } diff --git a/system/logkeeper/src/serve.rs b/system/logkeeper/src/serve.rs index ec4348cc5d2..f7f0149480f 100644 --- a/system/logkeeper/src/serve.rs +++ b/system/logkeeper/src/serve.rs @@ -38,13 +38,21 @@ const MAX_READERS: usize = 2; /// slot is let go: a wait on its sink taking bytes, bounded by this. const STALLED: Duration = Duration::from_secs(10); +/// The replay, and what of it the carrier is not owed. +struct Held { + replay: Replay, + /// Bytes the console UART carried while the machine stopped, which the + /// carrier skips ([`Hub::append_carried`]). + carried: Option<(u64, u64)>, +} + /// The replay, and the wake a caught-up reader waits on. pub struct Hub { shared: Arc, } struct Shared { - replay: Mutex, + replay: Mutex, grew: Condvar, readers: AtomicUsize, /// The wall clock the boot started at, for a line a reader is owed. @@ -55,7 +63,7 @@ impl Hub { /// A reader is handed over by the `log` port's thread ([`Hub::read`]). pub fn start(cap: usize, boot_secs: Option) -> Self { let shared = Arc::new(Shared { - replay: Mutex::new(Replay::new(cap)), + replay: Mutex::new(Held { replay: Replay::new(cap), carried: None }), grew: Condvar::new(), readers: AtomicUsize::new(0), boot_secs, @@ -66,16 +74,46 @@ impl Hub { /// A reader that asked for the log: it gets the read end /// of a pipe of its own, which the log is written into. pub fn read(&self, conn: &Connection) { - let end = self.shared.replay.lock().expect("logkeeper: the replay is poisoned").end(); - let Some(pipe) = hand_over(conn, end) else { return }; + let Some(pipe) = self.hand_over(conn) else { return }; admit(&self.shared, format!("reader {}", conn.as_handle().0), PipeSink::new(pipe)); } /// The lines the file has just taken, for every reader. pub fn append(&self, lines: &[u8]) { - self.shared.replay.lock().expect("logkeeper: the replay is poisoned").append(lines); + self.shared.replay.lock().expect("logkeeper: the replay is poisoned").replay.append(lines); + self.shared.grew.notify_all(); + } + + /// [`Self::append`], of lines the console UART has carried already, which + /// the carrier skips: what the machine said while it stopped, once the + /// stop was refused. + pub fn append_carried(&self, lines: &[u8]) { + let mut held = self.shared.replay.lock().expect("logkeeper: the replay is poisoned"); + let from = held.replay.end(); + held.replay.append(lines); + held.carried = Some((from, held.replay.end())); + drop(held); self.shared.grew.notify_all(); } + + /// How long the boot is so far. + pub fn end(&self) -> u64 { + self.shared.replay.lock().expect("logkeeper: the replay is poisoned").replay.end() + } + + /// The pipe a reader on `conn` is handed, and the boot's length it starts + /// with: `None` where the reader is already gone. + pub fn hand_over(&self, conn: &Connection) -> Option { + hand_over(conn, self.end()) + } + + /// The bytes after `at` for the carrier, at most [`CHUNK`] of them, once + /// there are any, skipping what the UART carried; `at` moves past them. A + /// carrier the replay no longer holds the bytes of is handed the line that + /// says so instead. + pub fn next_for_the_carrier(&self, at: &mut u64) -> Vec { + next(&self.shared, at, true) + } } /// The read end goes to the reader with the boot's length so far, the write @@ -137,24 +175,7 @@ fn feed(shared: &Shared, mut sink: impl Write) -> (u64, Left) { let mut at = 0u64; let mut sent = 0u64; loop { - let chunk = { - let mut replay = shared.replay.lock().expect("logkeeper: the replay is poisoned"); - loop { - match replay.next(at, CHUNK) { - Next::Bytes(bytes) => { - at += bytes.len() as u64; - break bytes.to_vec(); - } - Next::Evicted { lost, at: resume } => { - at = resume; - break evicted(shared.boot_secs, lost); - } - Next::CaughtUp => { - replay = shared.grew.wait(replay).expect("logkeeper: the replay is poisoned"); - } - } - } - }; + let chunk = next(shared, &mut at, false); match sink.write_all(&chunk) { Ok(()) => sent += chunk.len() as u64, Err(e) if e.kind() == std::io::ErrorKind::TimedOut => return (sent, Left::Stalled), @@ -163,6 +184,34 @@ fn feed(shared: &Shared, mut sink: impl Write) -> (u64, Left) { } } +/// The bytes after `at`, as [`Hub::next_for_the_carrier`] says, skipping what +/// the UART carried where `carrier` asks. +fn next(shared: &Shared, at: &mut u64, carrier: bool) -> Vec { + let mut held = shared.replay.lock().expect("logkeeper: the replay is poisoned"); + loop { + let mut most = CHUNK; + if let (true, Some((from, to))) = (carrier, held.carried) { + if *at >= from { + *at = (*at).max(to); + held.carried = None; + } else { + most = most.min((from - *at) as usize); + } + } + match held.replay.next(*at, most) { + Next::Bytes(bytes) => { + *at += bytes.len() as u64; + return bytes.to_vec(); + } + Next::Evicted { lost, at: resume } => { + *at = resume; + return evicted(shared.boot_secs, lost); + } + Next::CaughtUp => held = shared.grew.wait(held).expect("logkeeper: the replay is poisoned"), + } + } +} + /// The line a reader gets in place of what the replay no longer holds. fn evicted(boot_secs: Option, lost: u64) -> Vec { let at_ns = toyos_abi::clock::stamp_ns(); diff --git a/system/netstack/i219/Cargo.toml b/system/netstack/i219/Cargo.toml index 64959d7e21b..b00ca88a7d6 100644 --- a/system/netstack/i219/Cargo.toml +++ b/system/netstack/i219/Cargo.toml @@ -1,5 +1,5 @@ # A member of the host workspace (root `Cargo.toml`), like toyos-hda, toyos-pci -# and toyos-dma: it is pure, and its tests run on the host with no guest and no +# and toyos-register: it is pure, and its tests run on the host with no guest and no # QEMU. # # What lives here is every decision the I219 driver makes — the descriptor diff --git a/system/supervisor/lifecycle/src/drives.rs b/system/supervisor/lifecycle/src/drives.rs index bffa9cedc03..ff18ba2264c 100644 --- a/system/supervisor/lifecycle/src/drives.rs +++ b/system/supervisor/lifecycle/src/drives.rs @@ -8,8 +8,15 @@ //! **A reading that changed nothing starts nothing.** A start that was refused //! — the claim, the hand-over or the spawn — and an end owed no restart leave //! the function alone until its inventory record differs from the first one -//! read after. Every start of a function after its first spends its -//! [`Budget`], and a function whose budget is spent is given up on. +//! read after. Every start of a row's process on a function after its first +//! spends the function's [`Budget`], and a function whose budget is spent is +//! given up on. +//! +//! **Rows whose matchers name one function take turns at it**, in their +//! order: a process that ends with exit 0 says the function is not its own +//! ([`Drives::declined`]), and the next row's is started on it. Once every +//! row has declined it, it is left alone like an end owed nothing, and its +//! record differing starts the turns again. use std::collections::BTreeMap; use std::time::Instant; @@ -49,7 +56,10 @@ pub struct Drives { #[derive(Default)] struct Function { budget: Budget, - started: bool, + /// The row whose process was started on it last. + started: Option, + /// Rows whose process ended with exit 0 on it. + declined: Vec, phase: Phase, } @@ -80,7 +90,9 @@ impl Drives { }); let mut acts = Vec::new(); for pci in records.iter().filter(|pci| pci.driven != Driven::Kernel) { - let Some(&(row, entry)) = rows.iter().find(|(_, matcher)| matcher.matches(pci, boot)) else { continue }; + if !rows.iter().any(|(_, matcher)| matcher.matches(pci, boot)) { + continue; + } let function = self.functions.entry(pci.arrival).or_default(); if let Phase::Held(seen) = &mut function.phase { match seen { @@ -92,12 +104,21 @@ impl Drives { if !matches!(function.phase, Phase::Owed) || pci.driven != Driven::Free { continue; } - if function.started && !function.budget.spend(now) { + let turn = rows + .iter() + .find(|(row, matcher)| matcher.matches(pci, boot) && !function.declined.contains(&row.name)); + let Some(&(row, entry)) = turn else { + function.declined.clear(); + function.phase = Phase::Held(Some(*pci)); + continue; + }; + let again = function.started.as_deref() == Some(row.name.as_str()); + if again && !function.budget.spend(now) { function.phase = Phase::GivenUp; acts.push(Act::GiveUp { row, function: *pci }); continue; } - function.started = true; + function.started = Some(row.name.clone()); function.phase = Phase::Running; acts.push(Act::Start { row, entry, function: *pci }); } @@ -118,6 +139,14 @@ impl Drives { } } + /// The process of `row` on `arrival` ended with exit 0: the function is + /// not its own, and the next row that drives it is owed one. + pub fn declined(&mut self, arrival: Arrival, row: &str) { + let function = self.functions.get_mut(&arrival).expect("supervisor: an arrival it decided nothing about"); + function.declined.push(row.to_string()); + function.phase = Phase::Owed; + } + fn leave(&mut self, arrival: Arrival) { self.set(arrival, Phase::Held(None)); } @@ -243,6 +272,59 @@ mod tests { assert!(drives.decide(&rows, &machine, None, now).is_empty()); } + /// Two rows' matchers name one function: the first row's process is + /// started on it, the second's once the first declines it, and neither + /// again until its record differs. + #[test] + fn rows_that_name_one_function_take_turns_until_one_keeps_it() { + let now = Instant::now(); + let (agent, console) = (row(), Program { name: "console".into(), drives: vec![EDU.to_string()], ..Program::default() }); + let rows = [(&agent, EDU), (&console, EDU)]; + let machine = [edu(3, Driven::Free)]; + let mut drives = Drives::default(); + let turn = |acts: Vec>| match acts.as_slice() { + [Act::Start { row, .. }] => row.name.clone(), + other => panic!("one start, not {other:?}"), + }; + assert_eq!(turn(drives.decide(&rows, &machine, None, now)), "holder"); + drives.declined(Arrival(3), "holder"); + assert_eq!(turn(drives.decide(&rows, &machine, None, now)), "console"); + // The second keeps it: its process holds the function. + assert!(drives.decide(&rows, &[edu(3, Driven::Claimed)], None, now).is_empty()); + + // Both decline: the function is left alone until its record differs. + let mut drives = Drives::default(); + drives.decide(&rows, &machine, None, now); + drives.declined(Arrival(3), "holder"); + drives.decide(&rows, &machine, None, now); + drives.declined(Arrival(3), "console"); + for round in 0..3 { + assert!(drives.decide(&rows, &machine, None, now).is_empty(), "round {round} started one again"); + } + drives.decide(&rows, &[edu(3, Driven::Claimed)], None, now); + assert_eq!(turn(drives.decide(&rows, &machine, None, now)), "holder"); + } + + /// A row's first start on a function spends nothing, whichever row went + /// before it. + #[test] + fn a_turn_spends_no_budget() { + let now = Instant::now(); + let (agent, console) = (row(), Program { name: "console".into(), drives: vec![EDU.to_string()], ..Program::default() }); + let rows = [(&agent, EDU), (&console, EDU)]; + let machine = [edu(3, Driven::Free)]; + let mut drives = Drives::default(); + drives.decide(&rows, &machine, None, now); + drives.declined(Arrival(3), "holder"); + drives.decide(&rows, &machine, None, now); + for _ in 0..RESTARTS { + drives.ended(Arrival(3), true); + assert_eq!(started(&drives.decide(&rows, &machine, None, now)), [3]); + } + drives.ended(Arrival(3), true); + assert!(matches!(drives.decide(&rows, &machine, None, now).as_slice(), [Act::GiveUp { .. }])); + } + #[test] fn every_drives_entry_is_a_matcher_with_its_row() { let manifest = Manifest { programs: vec![row(), Program { name: "other".into(), ..Program::default() }], ..Manifest::default() }; diff --git a/system/supervisor/src/main.rs b/system/supervisor/src/main.rs index ca180751a91..99dee944f25 100644 --- a/system/supervisor/src/main.rs +++ b/system/supervisor/src/main.rs @@ -35,7 +35,9 @@ //! //! **A `drives` row has one process per PCI function it names**, each holding //! that function's claim and started when the inventory shows the function -//! free ([`toyos_supervisor::drives`]). **Every end is readiness on the +//! free ([`toyos_supervisor::drives`]); rows that name one function take +//! turns at it, the next started once one's process ends with exit 0. +//! **Every end is readiness on the //! process's handle**, watched on the loop's own poller: the supervisor reads //! the inventory again after each, and what an end owes is //! [`toyos_supervisor`]'s to decide. @@ -1943,7 +1945,11 @@ impl<'a> Supervisor<'a> { any = true; let Instance { row, function, .. } = self.instances.swap_remove(at); say!("supervisor: {} on PCI {} ended, {exit}", row.name, pci_at(function.at)); - self.drives.ended(function.arrival, restart::owed(row, exit)); + if exit.success() { + self.drives.declined(function.arrival, &row.name); + } else { + self.drives.ended(function.arrival, restart::owed(row, exit)); + } } if any { self.drive(); diff --git a/system/vdagent/Cargo.toml b/system/vdagent/Cargo.toml index a0e3f50a109..6019ed59e3b 100644 --- a/system/vdagent/Cargo.toml +++ b/system/vdagent/Cargo.toml @@ -7,10 +7,9 @@ license = "MIT OR Apache-2.0" [dependencies] toyos-abi = { path = "../../sdk/abi" } toyos = { path = "../../sdk/toyos" } -toyos-device-memory = { path = "../../lib/device-memory" } -toyos-virtio = { path = "../../lib/virtio" } -toyos-pci-claim = { path = "../../lib/pci-claim" } toyos-vdagent = { path = "../../lib/vdagent" } +toyos-virtio-serial = { path = "../../lib/virtio-serial" } +toyos-virtio-serial-claim = { path = "../../lib/virtio-serial/claim" } toyos-vdagent-bridge = { path = "bridge" } toyos-manifest = { path = "../../lib/manifest" } toyos-window = { path = "../../sdk/window" } diff --git a/system/vdagent/src/main.rs b/system/vdagent/src/main.rs index 606aec45d97..9ade8b2dc47 100644 --- a/system/vdagent/src/main.rs +++ b/system/vdagent/src/main.rs @@ -3,20 +3,25 @@ //! the compositor's clipboard. //! //! **What is here is the loop and the device's handshake.** The function is -//! [`channel`]'s; the stream is `toyos-vdagent`'s; every decision about who -//! owns the clipboard and what a transmit owes is `toyos-vdagent-bridge`'s; -//! the compositor's side is `window::watcher`'s. +//! `toyos-virtio-serial-claim`'s and which port is the agent's is +//! `toyos_virtio_serial::probe`'s; the stream is `toyos-vdagent`'s; every +//! decision about who owns the clipboard and what a transmit owes is +//! `toyos-vdagent-bridge`'s; the compositor's side is `window::watcher`'s. //! -//! **The handshake is virtio 1.2 §5.3.6.2's**: `DEVICE_READY`, the device's -//! `DEVICE_ADD` of [`PORT_ID`] answered with `PORT_READY`, and the agent's -//! port opened once the host's `PORT_OPEN(1)` has come. One process runs per -//! virtio-serial function the kernel does not drive, holding its claim. The host's -//! `PORT_OPEN(0)` is answered with the agent's `PORT_OPEN(0)`, and its -//! `PORT_OPEN(1)` with `PORT_OPEN(1)`: QEMU 11.1.1 then starts a new session -//! with its capabilities, measured. Any other port the device adds, a console -//! port, the removal of the agent's port, an unframed stream and a used ring -//! the device is not believed on end the program by name. It is not started -//! again: one that ended says why and stays down for the boot. +//! **The handshake is virtio 1.2 §5.3.6.2's**: `DEVICE_READY`, each port the +//! device adds answered with `PORT_READY`, and the agent's port — the one the +//! device names [`PORT_NAME`] — opened once the host's `PORT_OPEN(1)` has +//! come. One process runs per virtio-serial function the kernel does not +//! drive, holding its claim; a function none of whose ports is the agent's is +//! another driver's, and this program ends with exit 0, which hands it to the +//! next row that drives one, before it connects to the compositor's +//! clipboard. The host's `PORT_OPEN(0)` is answered with the agent's +//! `PORT_OPEN(0)`, and its `PORT_OPEN(1)` with `PORT_OPEN(1)`: QEMU 11.1.1 then +//! starts a new session with its capabilities, measured. A console port, the +//! removal of the agent's port, a device that misnames its ports, an +//! unframed stream and a used ring the device is not believed on end the +//! program by name. It is not started again: one that ended says why and stays +//! down for the boot. //! //! **A transmit the device has not taken back within [`TRANSMIT_BOUND`] ends //! the program**: QEMU 11.1.1 takes the largest one at once, measured, and a @@ -32,17 +37,16 @@ use toyos::say; use toyos::PciDev; use toyos_abi::syscall::PciId; use toyos_manifest::device::PciMatch; -use toyos_vdagent::control::{Control, Event}; use toyos_vdagent::stream::{Decoder, Message}; -use toyos_vdagent::PORT_ID; +use toyos_vdagent::{LARGEST_TRANSMIT, PORT_NAME}; use toyos_vdagent_bridge::sets::Sets; use toyos_vdagent_bridge::{Bridge, Effect}; +use toyos_virtio_serial::control::{Control, Event}; +use toyos_virtio_serial::probe::{Probe, Step, Verdict}; +use toyos_virtio_serial::PORT_ID; +use toyos_virtio_serial_claim::{Channel, Shape}; use window::watcher::{Watched, Watcher}; -mod channel; - -use channel::Channel; - /// One declaration of the clipboard's bound, read by the codec and the /// compositor alike. const _: () = assert!(toyos_vdagent::MAX_TEXT == window::MAX_CLIPBOARD_BYTES); @@ -56,6 +60,14 @@ const VIRTIO_SERIAL: PciMatch = PciMatch::new([0x07, 0x80, 0x00], Some(PciId { v /// QEMU 11.1.1 took, measured: it took every element at once. const TRANSMIT_BOUND: Duration = Duration::from_secs(5); +/// The agent's port: sixteen receive buffers of a page, and one transmit, of +/// every message owed, in one element. +const SHAPE: Shape = Shape { receive: Some((16, 4096)), transmit: (1, LARGEST_TRANSMIT) }; + +/// How long the device may take to name its ports. Policy, far past what QEMU +/// 11.1.1 takes: it adds and names them at the driver's first messages. +const PROBE_BOUND: Duration = Duration::from_secs(5); + const TOKEN_CLAIM: u64 = 0; const TOKEN_WATCHER: u64 = 1; @@ -71,24 +83,20 @@ struct Agent { } impl Agent { - fn control(&mut self, channel: &mut Channel, control: Result) { - let Control { id, event, value } = match control { - Ok(control) => control, - Err(why) => return say!("vdagent: {why}, ignored"), - }; + /// A control message about a port the device added, the agent's to answer. + fn control(&mut self, channel: &mut Channel, Control { id, event, value }: Control) { let reply = |event, value| Control { id: PORT_ID, event, value }; match event { - Event::DeviceAdd if id == PORT_ID => channel.send_control(reply(Event::PortReady, 1)), - Event::DeviceAdd => panic!("vdagent: the device added port {id}, and the launch declares port {PORT_ID} alone"), Event::ConsolePort => panic!("vdagent: the device made port {id} a console port"), - Event::DeviceRemove if id == PORT_ID => panic!("vdagent: the device removed port {PORT_ID}"), - Event::PortOpen if id == PORT_ID && value != 0 => { + Event::DeviceRemove => panic!("vdagent: the device removed port {PORT_ID}"), + _ if id != PORT_ID => {} + Event::PortOpen if value != 0 => { if !self.open { channel.send_control(reply(Event::PortOpen, 1)); self.open = true; } } - Event::PortOpen if id == PORT_ID => { + Event::PortOpen => { say!("vdagent: the host closed port {PORT_ID}"); self.bridge.closed(); self.decoder = Decoder::default(); @@ -97,8 +105,6 @@ impl Agent { self.open = false; } } - // Its name is not read: the port is the declared id. - Event::PortName => {} other => say!("vdagent: control {other:?} for port {id}, ignored"), } } @@ -163,17 +169,28 @@ impl Agent { fn main() { let claim: PciDev = endow::claim(&VIRTIO_SERIAL.to_string()) .expect("vdagent: its row drives a function, and it is started holding the function's claim"); + let mut channel = + Channel::open(claim, SHAPE, "vdagent").unwrap_or_else(|why| panic!("vdagent: the function is not driven: {why}")); + let poller = Poller::new(2); + let mut probe = Probe::new(PORT_NAME); + let mut held = identify(&mut channel, &mut probe, &poller); let watcher = Watcher::connect(toyos_manifest::clipboard::PORT) .unwrap_or_else(|why| panic!("vdagent: the compositor's clipboard: {why}")); - let mut channel = Channel::open(claim).unwrap_or_else(|why| panic!("vdagent: the function is not driven: {why}")); let mut agent = Agent { bridge: Bridge::default(), decoder: Decoder::default(), watcher, sets: Sets::default(), open: false }; + for control in held.drain(..) { + agent.control(&mut channel, control); + } - let poller = Poller::new(2); poller.watch(channel.claim(), READABLE, TOKEN_CLAIM); poller.watch(agent.watcher.conn(), READABLE, TOKEN_WATCHER); loop { - while let Some(control) = channel.control() { - agent.control(&mut channel, control); + while let Some(received) = channel.control() { + match probe.take(received.bytes()) { + Ok(Step::Reply(reply)) => channel.send_control(reply), + Ok(Step::Driver(control)) => agent.control(&mut channel, control), + Ok(Step::Nothing) => {} + Err(why) => panic!("vdagent: {why}"), + } } let Agent { decoder, .. } = &mut agent; let mut messages = Vec::new(); @@ -188,15 +205,16 @@ fn main() { Err(refusal) => say!("vdagent: {refusal}"), } } - if channel.transmit_free() { + channel.transmitted(); + if channel.can_transmit() { if agent.bridge.owes() { channel.transmit(|out| agent.bridge.transmit(out)); } - } else if channel.sent().is_some_and(|at| at.elapsed() > TRANSMIT_BOUND) { + } else if channel.oldest().is_some_and(|at| at.elapsed() > TRANSMIT_BOUND) { panic!("vdagent: the device has not taken a transmit back within {TRANSMIT_BOUND:?}"); } - let timeout = channel.sent().map_or(u64::MAX, |at| { + let timeout = channel.oldest().map_or(u64::MAX, |at| { let left = (at + TRANSMIT_BOUND).saturating_duration_since(Instant::now()); u64::try_from(left.as_nanos()).unwrap_or(u64::MAX) }); @@ -212,3 +230,37 @@ fn main() { } } } + +/// Read the device's control messages until a port carries the agent's name, +/// and answer the messages for it that came with them; or end with exit 0 +/// where none does: the function is another driver's. +fn identify(channel: &mut Channel, probe: &mut Probe<'_>, poller: &Poller) -> Vec { + let deadline = Instant::now() + PROBE_BOUND; + let mut held = Vec::new(); + loop { + while let Some(received) = channel.control() { + match probe.take(received.bytes()) { + Ok(Step::Reply(reply)) => channel.send_control(reply), + Ok(Step::Driver(control)) => held.push(control), + Ok(Step::Nothing) => {} + Err(why) => panic!("vdagent: {why}"), + } + } + match probe.verdict() { + Verdict::Ours => return held, + Verdict::Foreign(names) => { + say!("vdagent: the function's ports are {names:?}, none {PORT_NAME}; another row drives it"); + std::process::exit(0); + } + Verdict::Probing => {} + } + let left = deadline.saturating_duration_since(Instant::now()); + if left.is_zero() { + say!("vdagent: the function named no port {PORT_NAME} in {PROBE_BOUND:?}; another row drives it"); + std::process::exit(0); + } + poller.watch(channel.claim(), READABLE, TOKEN_CLAIM); + poller.wait(1, u64::try_from(left.as_nanos()).unwrap_or(u64::MAX), |_| {}); + channel.take_interrupt(); + } +} diff --git a/system/virtio-console/Cargo.toml b/system/virtio-console/Cargo.toml new file mode 100644 index 00000000000..0fc0336a470 --- /dev/null +++ b/system/virtio-console/Cargo.toml @@ -0,0 +1,16 @@ +[package] +name = "virtio-console" +description = "The VM's console: this boot's log, from its first line, written to the virtio-serial port QEMU puts on its stdio, as the console's carrier logkeeper quiets the UART for." +edition = "2024" +license = "MIT OR Apache-2.0" + +[dependencies] +toyos = { path = "../../sdk/toyos" } +toyos-abi = { path = "../../sdk/abi" } +toyos-manifest = { path = "../../lib/manifest" } +toyos-virtio-serial = { path = "../../lib/virtio-serial" } +toyos-virtio-serial-claim = { path = "../../lib/virtio-serial/claim" } +logkeeper-api = { path = "../../lib/logkeeper-api" } + +[package.metadata.toyos.host] +exempt.owns = "the virtio-serial function QEMU's console port is on, and the log's carrier port" diff --git a/system/virtio-console/src/main.rs b/system/virtio-console/src/main.rs new file mode 100644 index 00000000000..5ef9637c535 --- /dev/null +++ b/system/virtio-console/src/main.rs @@ -0,0 +1,200 @@ +//! `/system/bin/virtio-console` — the VM's console: this boot's log, from its +//! first line, written to the virtio-serial port named +//! [`CONSOLE_PORT`], which QEMU puts on its stdio. +//! +//! **One process per virtio-serial function the kernel does not drive**, as +//! `vdagent`'s row has: a function none of whose ports carries the console's +//! name is another driver's, and this program ends with exit 0, which hands +//! the function to the next row that drives one ([`Probe`]). +//! +//! **What it writes is `logkeeper`'s**: it asks to be the console's carrier +//! ([`toyos_logstream::CARRIER`]) and says how much of the log the host has +//! taken, and `logkeeper` quiets the console UART once the host has taken the +//! first bytes and gives it back when this program goes. +//! +//! **The host's pace is the console's**: a `virtserialport` holds a transmit +//! until its chardev takes it, so a transmit is made only while the device has +//! a free buffer, and the log waits in `logkeeper` meanwhile. A host that +//! takes nothing for [`SILENT`] is said once, and never ends the program: a +//! slow host is not a failed device. +//! +//! **The port posts no receive buffer**, so what the host types stays with +//! the host. + +use std::time::{Duration, Instant}; + +use toyos::endow; +use toyos::poller::{Poller, READABLE}; +use toyos::say; +use toyos::{PciDev, Pipe}; +use toyos_abi::syscall::{PciId, SyscallError}; +use toyos_manifest::device::PciMatch; +use toyos_virtio_serial::control::{Control, Event}; +use toyos_virtio_serial::probe::{Probe, Step, Verdict}; +use toyos_virtio_serial::{CONSOLE_PORT, PORT_ID}; +use toyos_virtio_serial_claim::{Channel, Shape}; + +/// A communication controller of class `07:80:00`, which virtio 1.2 §5.3.1 +/// numbers `0x1040 + 3`: the matcher its row drives, and the label its claim +/// is endowed under. +const VIRTIO_SERIAL: PciMatch = PciMatch::new([0x07, 0x80, 0x00], Some(PciId { vendor: 0x1af4, device: 0x1043 })); + +/// Four transmits in flight, so the host is handed the next while it writes +/// one out, of a size that makes each a few hundred lines. +const SHAPE: Shape = Shape { receive: None, transmit: (4, 32 * 1024) }; + +/// How long the device may take to name its ports. Policy, far past what QEMU +/// 11.1.1 takes: it adds and names them at the driver's first messages. +const PROBE_BOUND: Duration = Duration::from_secs(5); + +/// How long the host may take nothing before that is said. +const SILENT: Duration = Duration::from_secs(10); + +const TOKEN_CLAIM: u64 = 0; +const TOKEN_LOG: u64 = 1; + +fn main() { + let claim: PciDev = endow::claim(&VIRTIO_SERIAL.to_string()) + .expect("virtio-console: its row drives a function, and it is started holding the function's claim"); + let mut channel = Channel::open(claim, SHAPE, "virtio-console") + .unwrap_or_else(|why| panic!("virtio-console: the function is not driven: {why}")); + let poller = Poller::new(2); + let mut probe = Probe::new(CONSOLE_PORT); + identify(&mut channel, &mut probe, &poller); + channel.send_control(Control { id: PORT_ID, event: Event::PortOpen, value: 1 }); + + let (conn, served) = logkeeper_api::carry().unwrap_or_else(|why| panic!("virtio-console: logkeeper: {why}")); + say!("virtio-console: carrying the log on port {PORT_ID}, from its first line"); + let mut console = Console { channel, probe, log: served.pipe, taken: 0, told: 0, silent_since: None }; + loop { + console.controls(); + let taken = console.channel.transmitted(); + console.taken += taken; + if console.taken > console.told { + logkeeper_api::delivered(&conn, console.taken); + console.told = console.taken; + } + console.transmit(); + console.say_silence(taken > 0); + + poller.watch(console.channel.claim(), READABLE, TOKEN_CLAIM); + if console.channel.can_transmit() { + poller.watch(&console.log, READABLE, TOKEN_LOG); + } + let timeout = console.channel.oldest().map_or(u64::MAX, |at| { + let left = (at + SILENT).saturating_duration_since(Instant::now()); + u64::try_from(left.as_nanos()).unwrap_or(u64::MAX) + }); + let mut interrupted = false; + poller.wait(1, timeout, |token| interrupted |= token == TOKEN_CLAIM); + if interrupted { + console.channel.take_interrupt(); + } + } +} + +/// Read the device's control messages until a port carries the console's +/// name, or end with exit 0 where none does: the function is another +/// driver's. +fn identify(channel: &mut Channel, probe: &mut Probe<'_>, poller: &Poller) { + let deadline = Instant::now() + PROBE_BOUND; + loop { + while let Some(received) = channel.control() { + match probe.take(received.bytes()) { + Ok(Step::Reply(reply)) => channel.send_control(reply), + Ok(Step::Driver(_) | Step::Nothing) => {} + Err(why) => panic!("virtio-console: {why}"), + } + } + match probe.verdict() { + Verdict::Ours => return, + Verdict::Foreign(names) => { + say!("virtio-console: the function's ports are {names:?}, none {CONSOLE_PORT}; another row drives it"); + std::process::exit(0); + } + Verdict::Probing => {} + } + let left = deadline.saturating_duration_since(Instant::now()); + if left.is_zero() { + say!("virtio-console: the function named no port {CONSOLE_PORT} in {PROBE_BOUND:?}; another row drives it"); + std::process::exit(0); + } + poller.watch(channel.claim(), READABLE, TOKEN_CLAIM); + poller.wait(1, u64::try_from(left.as_nanos()).unwrap_or(u64::MAX), |_| {}); + channel.take_interrupt(); + } +} + +/// The port, once it is the console's, and the log it carries. +struct Console<'p> { + channel: Channel, + probe: Probe<'p>, + log: Pipe, + /// Bytes of the log the host has taken. + taken: u64, + /// The most recent count `logkeeper` was told. + told: u64, + /// Since when the host has taken nothing of a transmit, once that is said. + silent_since: Option, +} + +impl Console<'_> { + /// Every control message the device sent since the port was the console's. + fn controls(&mut self) { + while let Some(received) = self.channel.control() { + match self.probe.take(received.bytes()) { + Ok(Step::Reply(reply)) => self.channel.send_control(reply), + Ok(Step::Driver(Control { id, event: Event::DeviceRemove, .. })) if self.probe.is_ours(id) => { + panic!("virtio-console: the device removed the console's port {id}") + } + Ok(Step::Driver(Control { id, event: Event::PortOpen, value })) if self.probe.is_ours(id) => { + say!("virtio-console: the host {} the console's port", if value != 0 { "opened" } else { "closed" }); + } + Ok(Step::Driver(_) | Step::Nothing) => {} + Err(why) => panic!("virtio-console: {why}"), + } + } + } + + /// As much of the log as the device has free buffers for, read straight + /// into them. + fn transmit(&mut self) { + while self.channel.can_transmit() { + let log = &self.log; + let mut ended = false; + let sent = self.channel.transmit(|buffer| match log.read_nonblock(buffer) { + Ok(0) => { + ended = true; + 0 + } + Ok(n) => n, + Err(SyscallError::WouldBlock) => 0, + Err(why) => panic!("virtio-console: logkeeper's pipe refused a read: {why:?}"), + }); + if ended { + panic!("virtio-console: logkeeper closed the log's pipe"); + } + if sent == 0 { + return; + } + } + } + + /// Say once that the host has taken nothing of a transmit for [`SILENT`], + /// and once that it takes the console again. + fn say_silence(&mut self, took: bool) { + match (self.silent_since, took) { + (Some(since), true) => { + say!("virtio-console: the host takes the console again after {} s", since.elapsed().as_secs()); + self.silent_since = None; + } + (None, false) => { + if let Some(at) = self.channel.oldest().filter(|at| at.elapsed() >= SILENT) { + say!("virtio-console: the host has taken nothing for {} s; the console waits on it", SILENT.as_secs()); + self.silent_since = Some(at); + } + } + _ => {} + } + } +} diff --git a/tests/common/iommu.rs b/tests/common/iommu.rs index 3d05f046a5f..3ff1be77bb0 100644 --- a/tests/common/iommu.rs +++ b/tests/common/iommu.rs @@ -497,20 +497,3 @@ pub fn a_virtio_function_without_access_platform_is_refused() -> Result<(), Stri eprintln!(" [iommu] {UNTRANSLATED_NIC}"); Ok(()) } - -/// The kernel's virtio console created without `iommu_platform=on` -/// ([`Profile::HeadlessConsoleUntranslated`]) bypasses the unit, so the kernel -/// tells it physical addresses and the boot reaches its ready marker over it. -/// Given a domain's addresses instead, it reads and writes whatever memory -/// those are, and the boot's console is lost. q35 only: no bench has a virtio -/// function. -pub fn the_kernel_tells_a_bypassing_console_physical_addresses() -> Result<(), String> { - let options = BootOptions { profile: Profile::HeadlessConsoleUntranslated, ..Default::default() }; - let qemu = QemuInstance::boot_with_options(&netcase(), &[], &[], options); - let log = Serial::named("boot console", qemu.boot_log().to_string()); - log.must_be_clean()?; - log.must_say("offers no ACCESS_PLATFORM, so it is told physical addresses")?; - log.must_say("Boot: complete")?; - eprintln!(" [iommu] the console bypassing the unit was told physical addresses and carried the boot"); - Ok(()) -} diff --git a/tests/common/mod.rs b/tests/common/mod.rs index b92c3162aab..07513beef62 100644 --- a/tests/common/mod.rs +++ b/tests/common/mod.rs @@ -25,5 +25,7 @@ pub mod screen; pub mod serial; pub mod slots; pub mod stateview; +/// The VM's console on a virtio machine. +pub mod vmconsole; /// A VNC client, as far as resizing a host window takes one. pub mod vnc; diff --git a/tests/common/qemu.rs b/tests/common/qemu.rs index 1d868d7d83a..a6ce68f6c39 100644 --- a/tests/common/qemu.rs +++ b/tests/common/qemu.rs @@ -1,5 +1,7 @@ use std::collections::BTreeSet; -use std::io::{BufReader, BufWriter, Read, Write}; +use std::io::{BufWriter, Read, Write}; +use std::os::fd::AsRawFd; +use std::os::unix::net::{UnixListener, UnixStream}; use std::path::{Path, PathBuf}; use std::process::{Child, ChildStdin, Command, Stdio}; use std::sync::atomic::{AtomicBool, AtomicU32, Ordering}; @@ -698,10 +700,6 @@ pub enum Profile { /// `iommu_platform=on`, so behind the unit on paper and bypassing it in /// fact: the function a claim must never be handed. HeadlessNicUntranslated, - /// [`Profile::Headless`] with the kernel's virtio console created without - /// `iommu_platform=on`: a device the kernel itself must tell physical - /// addresses. - HeadlessConsoleUntranslated, /// [`Profile::Headless`] with two of QEMU's `edu`, which the kernel drives /// neither of: two identical functions, each a `drives` row's to start a /// process on. @@ -805,7 +803,6 @@ impl Profile { | Self::HeadlessUsbSpare | Self::HeadlessNicBehindPort | Self::HeadlessNicUntranslated - | Self::HeadlessConsoleUntranslated | Self::HeadlessTwoEdu | Self::Metal | Self::MetalTwoHda @@ -950,9 +947,6 @@ struct Shape { /// channel on a virtio-serial function of its own. virtio: Virtio, nic: Nic, - /// The virtio console created without `iommu_platform=on` on a machine - /// with a unit, so its DMA bypasses the unit. - console_bypasses: bool, /// The `-device` argument for each xHCI controller, port and slot counts /// included. A list because a machine can have more than one and the T14 /// does — its keyboard is on the second. @@ -1079,7 +1073,6 @@ impl Profile { rng: true, virtio_gpu: false, hda: false, - console_bypasses: false, }, Self::Headless => Shape { vga: "none", @@ -1095,7 +1088,6 @@ impl Profile { rng: false, virtio_gpu: false, hda: false, - console_bypasses: false, }, Self::Metal => Shape { vga: "std", @@ -1115,7 +1107,6 @@ impl Profile { rng: false, virtio_gpu: false, hda: false, - console_bypasses: false, }, Self::MetalNoX2apic => Self::Metal.shape(), Self::MetalTwoHda => Shape { hda: true, ..Self::Metal.shape() }, @@ -1134,7 +1125,6 @@ impl Profile { Self::HeadlessVirtioGpu => Shape { virtio_gpu: true, ..Self::Headless.shape() }, Self::HeadlessNicBehindPort => Shape { nic: Nic::BehindPort, ..Self::Headless.shape() }, Self::HeadlessNicUntranslated => Shape { nic: Nic::Untranslated, ..Self::Headless.shape() }, - Self::HeadlessConsoleUntranslated => Shape { console_bypasses: true, ..Self::Headless.shape() }, Self::HeadlessTwoEdu => Shape { probes: Probes::Edus, ..Self::Headless.shape() }, Self::HeadlessUsbSpare => Shape { xhci: &[XHCI_DEFAULT, XHCI_SPARE], @@ -1276,7 +1266,11 @@ pub struct QemuInstance { child: Child, /// What ends QEMU when this process dies without dropping this. _tether: Tether, - stdin: BufWriter, + /// The guest's commands: the UART's socket on a virtio machine, QEMU's + /// stdin otherwise. + stdin: BufWriter>, + /// QEMU's stdin on a virtio machine, held open and never written. + _console_input: Option, rx: Receiver, _reader_thread: thread::JoinHandle, /// Held for the claim: one live guest per NVMe image. @@ -1465,6 +1459,9 @@ impl QemuInstance { rust_tests: &[(String, Vec)], options: BootOptions, ) -> Self { + if options.profile.shape().virtio.present() { + carries_its_console(config).unwrap_or_else(|why| panic!("[qemu] {}: {why}", config.display())); + } let mut features: Vec<&str> = kernel_of(&options); if options.debug_wait { features.push(toyos_build::build::DEBUG_KERNEL_BUILD); @@ -1558,7 +1555,7 @@ impl QemuInstance { .and_then(|firmware| firmware.fresh_vars(&vars)) .unwrap_or_else(|why| panic!("[qemu] {why}")); - let qemu = qemu_command(&boot_image, nvme.path(), &uart_log, &sockets.dir, &vars, &options); + let qemu = qemu_command(&boot_image, nvme.path(), &sockets.dir, &vars, &options); spawn_and_wait_ready( qemu, options, @@ -1709,7 +1706,7 @@ impl QemuInstance { self.nvme.path() } - pub fn stdin_mut(&mut self) -> &mut BufWriter { + pub fn stdin_mut(&mut self) -> &mut BufWriter> { &mut self.stdin } @@ -2241,7 +2238,7 @@ impl QmpInput { /// unused — so this is what a profile assertion has to read. pub fn profile_argv(options: &BootOptions) -> Vec { let p = Path::new("/nonexistent"); - qemu_command(p, p, p, p, p, options) + qemu_command(p, p, p, p, options) .get_args() .map(|a| a.to_string_lossy().into_owned()) .collect() @@ -2250,7 +2247,6 @@ pub fn profile_argv(options: &BootOptions) -> Vec { fn qemu_command( boot_image: &Path, nvme_image: &Path, - uart_log: &Path, socket_dir: &Path, firmware_vars: &Path, options: &BootOptions, @@ -2518,26 +2514,21 @@ fn qemu_command( .arg("-device") .arg(format!("virtio-sound-pci,audiodev=audio0,streams=1{platform}")); } - // The clipboard's channel, just before the console's function, so it - // enumerates first and every virtio boot runs the kernel's console - // rule. Refused by name where QEMU has no `qemu-vdagent`. + // The clipboard's channel, refused by name where QEMU has no + // `qemu-vdagent`. toyos_build::vdagent::available(arch).unwrap_or_else(|why| panic!("[qemu] {why}")); qemu.args(toyos_build::vdagent::args(platform)); - qemu - // virtio-console on stdio is the primary I/O channel; UART goes to - // a temp file so early-boot logs and panic fallback still land - // somewhere when the kernel switches backends. + // The console's port on stdio, which `/system/bin/virtio-console` + // carries the log on; the UART on the socket this harness listens on + // ([`UartConsole`]), which takes the commands typed at the guest and + // the lines the console's port cannot carry. + qemu.arg("-chardev") + .arg(format!("socket,id=uart0,path={}", socket_dir.join(UART_SOCKET).display())) .arg("-serial") - .arg(format!("file:{}", uart_log.display())) + .arg("chardev:uart0") .arg("-chardev") .arg("stdio,id=cs0,signal=off") - .arg("-device") - .arg(format!( - "virtio-serial-pci-non-transitional,id=virtio-serial0,max_ports=1{}", - if shape.console_bypasses { "" } else { platform } - )) - .arg("-device") - .arg("virtconsole,chardev=cs0,id=console0"); + .args(toyos_build::vmconsole::args("cs0", platform)); } else if options.mute { qemu.arg("-serial").arg("none"); } else { @@ -2618,57 +2609,31 @@ fn spawn_and_wait_ready(mut qemu: Command, options: BootOptions, files: Files) - .stdout(Stdio::piped()) .stderr(Stdio::inherit()); + // Listening before QEMU starts, which connects as it makes the chardev. + let virtio = options.profile.shape().virtio.present(); + let listener = virtio.then(|| { + UnixListener::bind(sockets.dir.join(UART_SOCKET)).unwrap_or_else(|e| panic!("[qemu] the UART's socket: {e}")) + }); + if VERBOSE.load(Ordering::Relaxed) { eprintln!("[qemu {seq}] Launching QEMU..."); } let (mut child, tether) = toyos_build::tether::spawn(qemu).expect("Failed to launch QEMU"); - let stdin = BufWriter::new(child.stdin.take().unwrap()); let stdout = child.stdout.take().unwrap(); + let qemu_stdin = child.stdin.take().unwrap(); + let uart = listener.map(|listener| UartConsole::accept(&listener, &mut child, &uart_log)); + // Commands go to the UART, which test-runner reads; on a virtio machine + // QEMU's stdin stays open and unwritten, since the console port's chardev + // reads its end as the host going. + let (stdin, console_input): (Box, _) = match &uart { + Some(uart) => (Box::new(uart.stream.try_clone().expect("[qemu] the UART's socket")), Some(qemu_stdin)), + None => (Box::new(qemu_stdin), None), + }; + let stdin = BufWriter::new(stdin); let (tx, rx) = mpsc::channel::(); - // The virtio port starts at the kernel's first record; a 16550 on stdio has - // no other file, so it is read whole. - let mut kernel_console = options - .profile - .shape() - .virtio - .present() - .then(toyos_build::kernelconsole::KernelConsole::default); - let reader_thread = thread::spawn(move || { - let mut reader = BufReader::new(stdout); - let mut full_log = String::new(); - // Read bytes and split them, rather than `BufRead::lines`: every - // consumer below still gets whole lines and nothing else. - let mut pending: Vec = Vec::new(); - let mut chunk = [0u8; 4096]; - loop { - let read = reader.read(&mut chunk).unwrap_or(0); - if read == 0 { - // EOF, and whatever has no newline behind it is the last line — - // which is what `lines` hands over here too. - if !pending.is_empty() { - publish_line(pending, seq, &mut full_log, &tx); - } - return full_log; - } - let read = match &mut kernel_console { - Some(console) => console.pass(&chunk[..read]), - None => std::borrow::Cow::Borrowed(&chunk[..read]), - }; - pending.extend_from_slice(&read); - while let Some(at) = pending.iter().position(|&b| b == b'\n') { - let mut line: Vec = pending.drain(..=at).collect(); - line.pop(); - if line.last() == Some(&b'\r') { - line.pop(); - } - if !publish_line(line, seq, &mut full_log, &tx) { - return full_log; - } - } - } - }); + let reader_thread = thread::spawn(move || read_consoles(stdout, uart, seq, &tx)); // A muted guest has no console at all, so there is no marker to wait for: // the caller polls the framebuffer. Blocking here would only time out. @@ -2709,6 +2674,7 @@ fn spawn_and_wait_ready(mut qemu: Command, options: BootOptions, files: Files) - child, _tether: tether, stdin, + _console_input: console_input, rx, _reader_thread: reader_thread, nvme, @@ -2723,6 +2689,190 @@ fn spawn_and_wait_ready(mut qemu: Command, options: BootOptions, files: Files) - } } +/// Whether the image `config` builds carries the console on a virtio machine: +/// its console driver's row, and `logkeeper` serving it the log. Without both +/// the boot's whole log is on the UART, which a virtio machine's harness +/// reads only from the carrier's word on. +fn carries_its_console(config: &Path) -> Result<(), String> { + let text = fs::read_to_string(config).map_err(|e| e.to_string())?; + let doc: toml::Value = text.parse().map_err(|e| format!("not TOML: {e}"))?; + let row = |name: &str| doc.get("programs").and_then(|p| p.get(name)); + let names = |row: Option<&toml::Value>, key: &str, port: &str| { + row.and_then(|r| r.get(key)).and_then(|v| v.as_array()).is_some_and(|a| a.iter().any(|v| v.as_str() == Some(port))) + }; + let carrier = toyos_logstream::CARRIER; + if !names(row(toyos_logstream::CARRIER_HOLDER), "receives", carrier) { + return Err(format!("a virtio machine's image has no `{}` row receiving `{carrier}`", toyos_logstream::CARRIER_HOLDER)); + } + if !names(row(toyos_logstream::LOGKEEPER), "serves", carrier) { + return Err(format!("a virtio machine's image has no `logkeeper` serving `{carrier}`")); + } + Ok(()) +} + +/// The name of the UART's socket in a boot's socket directory. +const UART_SOCKET: &str = "uart.sock"; + +/// What the kernel puts on the UART as the console's carrier takes the log +/// (`kernel/src/log/console.rs`): every UART line before it the console port +/// carries too, and the ones after it are the UART's alone. +const CARRIED_ELSEWHERE: &str = "console: the log is carried elsewhere from here"; + +/// How long QEMU may take to connect to the UART's socket: it does as it makes +/// its chardevs, before the machine runs. +const UART_CONNECT: Duration = Duration::from_secs(30); + +/// A virtio machine's UART, on the socket this harness listens on: the +/// loader, the early boot, and the lines the console port cannot carry — the +/// stop's, a fatal path's, and the whole log again once the console's carrier +/// is gone. +/// +/// **Its lines up to [`CARRIED_ELSEWHERE`] are the console port's too**, so +/// they go only to the boot's UART file, but for a kernel's death, which no +/// other channel will carry; every line after either is published. +struct UartConsole { + stream: UnixStream, + file: fs::File, + pending: Vec, + live: bool, +} + +impl UartConsole { + /// QEMU's connection, waited for while it runs and for [`UART_CONNECT`]. + fn accept(listener: &UnixListener, child: &mut Child, file: &Path) -> Self { + listener.set_nonblocking(true).expect("[qemu] the UART's socket"); + let deadline = Instant::now() + UART_CONNECT; + let stream = loop { + match listener.accept() { + Ok((stream, _)) => break stream, + Err(e) if e.kind() == std::io::ErrorKind::WouldBlock => {} + Err(e) => panic!("[qemu] the UART's socket: {e}"), + } + if let Ok(Some(status)) = child.try_wait() { + panic!("[qemu] QEMU ended ({status}) before it connected to the UART's socket"); + } + if Instant::now() > deadline { + let _ = child.kill(); + panic!("[qemu] QEMU did not connect to the UART's socket in {} s", UART_CONNECT.as_secs()); + } + readable(listener.as_raw_fd(), 100); + }; + stream.set_nonblocking(false).expect("[qemu] the UART's socket"); + let file = fs::File::create(file).unwrap_or_else(|e| panic!("[qemu] {}: {e}", file.display())); + Self { stream, file, pending: Vec::new(), live: false } + } + + /// The UART's lines in `bytes`, each published that is to be. `false` + /// once nothing is left to read for. + fn pass(&mut self, bytes: &[u8], seq: u32, full_log: &mut String, tx: &mpsc::Sender) -> bool { + self.file.write_all(bytes).expect("[qemu] the boot's UART file"); + self.pending.extend_from_slice(bytes); + while let Some(line) = take_line(&mut self.pending) { + let text = String::from_utf8_lossy(&line); + if !self.live && text.contains(CARRIED_ELSEWHERE) { + self.live = true; + continue; + } + self.live |= super::serial::died(&text) == Some(super::serial::Died::Kernel); + if self.live && !publish_line(line, seq, full_log, tx) { + return false; + } + } + true + } +} + +/// Whether `fd` reads within `ms`, or at its end. +fn readable(fd: std::os::fd::RawFd, ms: i32) -> bool { + let mut ready = libc::pollfd { fd, events: libc::POLLIN, revents: 0 }; + // SAFETY: one `pollfd`, which lives across the call. + unsafe { libc::poll(&mut ready, 1, ms) > 0 } +} + +/// The first whole line in `pending`, without its line ending. +fn take_line(pending: &mut Vec) -> Option> { + let at = pending.iter().position(|&b| b == b'\n')?; + let mut line: Vec = pending.drain(..=at).collect(); + line.pop(); + if line.last() == Some(&b'\r') { + line.pop(); + } + Some(line) +} + +/// Publish every whole line of QEMU's stdout and, on a virtio machine, of +/// its UART, until QEMU's stdout ends; answers every line published. +/// +/// **One thread reads both, and drains stdout before each read of the UART**: +/// QEMU writes a console port's bytes to stdout before the guest is told the +/// host took them, so what the guest said there before it said a line on the +/// UART is in the pipe by the time that line can be read, and is published +/// first. +fn read_consoles( + mut stdout: std::process::ChildStdout, + mut uart: Option, + seq: u32, + tx: &mpsc::Sender, +) -> String { + let mut full_log = String::new(); + let mut pending: Vec = Vec::new(); + let mut chunk = [0u8; 4096]; + // One read of stdout: `false` at its end, or once nothing is left to read for. + let mut out = |stdout: &mut std::process::ChildStdout, full_log: &mut String| -> bool { + let read = stdout.read(&mut chunk).unwrap_or(0); + if read == 0 { + // EOF, and whatever has no newline behind it is the last line. + if !pending.is_empty() { + publish_line(std::mem::take(&mut pending), seq, full_log, tx); + } + return false; + } + pending.extend_from_slice(&chunk[..read]); + while let Some(line) = take_line(&mut pending) { + if !publish_line(line, seq, full_log, tx) { + return false; + } + } + true + }; + loop { + let Some(console) = &mut uart else { + if !out(&mut stdout, &mut full_log) { + return full_log; + } + continue; + }; + let mut fds = [ + libc::pollfd { fd: stdout.as_raw_fd(), events: libc::POLLIN, revents: 0 }, + libc::pollfd { fd: console.stream.as_raw_fd(), events: libc::POLLIN, revents: 0 }, + ]; + // SAFETY: two `pollfd`s, which live across the call. + if unsafe { libc::poll(fds.as_mut_ptr(), 2, -1) } < 0 { + continue; + } + if fds[0].revents != 0 && !out(&mut stdout, &mut full_log) { + return full_log; + } + if fds[1].revents == 0 { + continue; + } + while readable(stdout.as_raw_fd(), 0) { + if !out(&mut stdout, &mut full_log) { + return full_log; + } + } + let mut bytes = [0u8; 4096]; + match console.stream.read(&mut bytes) { + Ok(0) | Err(_) => uart = None, + Ok(n) => { + if !console.pass(&bytes[..n], seq, &mut full_log, tx) { + return full_log; + } + } + } + } +} + /// One finished console line into everything that keeps one. /// /// `false` means nothing is left to read for: the receiver has gone, or the @@ -2772,8 +2922,8 @@ fn wait_for_ready( // one failure in this harness that arrives with no evidence at all, // and "the guest printed nothing" and "the guest printed sixty // lines and then stopped" are different machines. The 16550's - // too: a kernel that dies before virtio-console arrives said why - // only there. + // too: a kernel that dies before the console port carries the log + // said why only there. let uart = fs::read_to_string(uart_log).unwrap_or_default(); panic!( "[qemu] Boot timed out waiting for {ready}; the console carried:\n{}\nuart:\n{}", @@ -2833,9 +2983,9 @@ fn wait_for_ready( continue; } // The 16550's file is never read for the marker: it carries the - // kernel's records until virtio-console arrives and replays them + // log until the console port's carrier takes it and replays it // here, so a marker found there first ends the wait with a boot - // log that holds none of them. + // log that holds none of the boot. Err(RecvTimeoutError::Timeout) => continue, Err(RecvTimeoutError::Disconnected) => { let status = child.wait(); diff --git a/tests/common/serial.rs b/tests/common/serial.rs index bd2e8b4ccdc..2d892dbebdd 100644 --- a/tests/common/serial.rs +++ b/tests/common/serial.rs @@ -225,10 +225,8 @@ impl Serial { self.text.lines().filter(|l| is_kernel_line(l)).count() } - /// A line carrying a kernel prefix somewhere other than its start, which - /// is the virtio-console's missing line atomicity showing up in the - /// capture: `log!` and a userspace `println!` interleave mid-word (see - /// `issues/`). Reported rather than repaired — a needle that went + /// A line carrying a kernel prefix somewhere other than its start: two + /// writers' lines spliced mid-word in the capture. Reported rather than repaired — a needle that went /// missing because it was split in half should say so instead of looking /// like the guest never said it. pub fn interleaved(&self) -> Option<&str> { diff --git a/tests/common/vmconsole.rs b/tests/common/vmconsole.rs new file mode 100644 index 00000000000..5bf51551b17 --- /dev/null +++ b/tests/common/vmconsole.rs @@ -0,0 +1,84 @@ +//! The VM's console on a virtio machine: `/system/bin/virtio-console` +//! carrying the log on its port, and the UART taking it back when it goes. + +use std::io::Write; +use std::path::Path; + +use super::qemu::{self, BootOptions, QemuInstance, QmpMonitor}; +use super::serial::Serial; + +/// What `logkeeper` says once the console's carrier has taken the log's first +/// bytes and the kernel has been told the UART is quiet +/// (`system/logkeeper/src/carrier.rs`). +const QUIETED: &str = "logkeeper: the console carrier's channel takes the log, so the UART is quiet"; + +/// What the kernel puts on the UART as the log comes back to it +/// (`kernel/src/log/console.rs`). +const BACK_ON_THE_UART: &str = "console: the UART carries the log again"; + +/// What the supervisor says of the console's driver ending, the first +/// program line the UART can carry once it has. +const DRIVER_ENDED: &str = "supervisor: virtio-console on PCI"; + +/// **The UART carries the log again once the console's carrier goes.** The +/// console's port is unplugged under the running guest, the one way to end +/// its driver without the authority to: the driver ends on the device's +/// `DEVICE_REMOVE`, `logkeeper` sees its carrier gone, the kernel says so on +/// the UART, and the supervisor's account of the driver's end reaches the +/// harness there. Its parts are host-tested (`logkeeper`'s `Carriage`, +/// `kernel::carrier`); what only a guest has is the kernel, `logkeeper` and +/// the driver's process doing it. q35 only: no other machine of this suite +/// carries a console port. +pub fn the_uart_carries_the_log_again_once_the_console_carrier_goes(test_config: &Path) -> Result<(), String> { + let mut qemu = QemuInstance::boot_with_options(test_config, &[], &[], BootOptions { qmp: true, ..Default::default() }); + let boot = Serial::boot(&qemu); + boot.must_be_clean()?; + let mut console = boot.text().to_string(); + qemu::await_marker(&mut qemu, &mut console, QUIETED, "logkeeper to quiet the UART for its carrier")?; + let from = console.len(); + QmpMonitor::open(qemu.qmp_socket()).human(&format!("device_del {}", toyos_build::vmconsole::PORT_DEVICE)); + qemu::await_marker_new(&mut qemu, &mut console, DRIVER_ENDED, from, "the console's driver to end")?; + let after = Serial::named("console after its port went", console[from..].to_string()); + after.must_say_after(BACK_ON_THE_UART, DRIVER_ENDED)?; + eprintln!(" [vmconsole] {}", after.must_say(DRIVER_ENDED)?.trim()); + Ok(()) +} + +/// What the fatal path says once it has armed the reboot that ends it +/// (`kernel/src/panic_reboot.rs`): a line only the UART carries. +const REBOOT_ARMED: &str = "panic: rebooting in"; + +/// **A kernel's fatal report reaches the harness on the UART while the +/// console port's carrier carries the log**, three CPUs making kernel +/// records as fast as they can — the records `klogd` advances past unwritten +/// while it does. `test_rs_panic_halts_first` goes fatal by `SYS_DEBUG` once +/// its siblings run: its line and the fatal path's after it are the UART's +/// alone, the carrier being one of the processes the fatal path stopped. +/// That a fatal path's first record is never advanced past is the ordering +/// `kernel/src/log/console.rs` states, not this test's: the halt makes the +/// same claim microseconds later. q35 only: no other machine of this suite +/// carries a console port. +pub fn a_fatal_report_reaches_the_uart_while_a_carrier_carries_the_log(test_config: &Path) -> Result<(), String> { + let fatal = qemu::build_toyos_bin( + qemu::SUITE_ARCH, + &super::compile::repo_root().join("tests/guest/rust-tests"), + "panic_halts_first", + ); + let options = BootOptions { + smp: crate::STOP_CPUS, + kernel_features: crate::ACTUATOR_KERNEL, + extra_root_files: vec![("bin/test_rs_panic_halts_first".to_string(), fatal)], + ..Default::default() + }; + let mut qemu = QemuInstance::boot_with_options(test_config, &[], &[], options); + let mut console = Serial::boot(&qemu).text().to_string(); + qemu::await_marker(&mut qemu, &mut console, QUIETED, "logkeeper to quiet the UART for its carrier")?; + let from = console.len(); + writeln!(qemu.stdin_mut(), "run test_rs_panic_halts_first").map_err(|e| e.to_string())?; + qemu.flush_stdin(); + qemu::await_marker_new(&mut qemu, &mut console, REBOOT_ARMED, from, "the fatal path to arm its reboot")?; + let after = Serial::named("console after the fatal halt", console[from..].to_string()); + let nonce = after.must_say_after(crate::FATAL_HALT_NONCE, REBOOT_ARMED)?; + eprintln!(" [vmconsole] {}", nonce.trim()); + Ok(()) +} diff --git a/tests/images/acpicase.toml b/tests/images/acpicase.toml index ff0955ee095..1fd2a05fdcd 100644 --- a/tests/images/acpicase.toml +++ b/tests/images/acpicase.toml @@ -7,6 +7,7 @@ start = ["logkeeper", "diskserver", "fileserver", "test-runner"] [programs.logkeeper] service = true syscap = ["logread"] +serves = ["log-carrier"] [programs.acpiserver] @@ -58,3 +59,8 @@ receives = ["disk"] restart = true roles = ["data", "log", "boot"] receives = ["block"] + +# The VM's console, on whichever virtio-serial function names its port. +[programs.virtio-console] +drives = ["pci:07:80:00/1af4:1043"] +receives = ["log-carrier"] diff --git a/tests/images/clipboardcase.toml b/tests/images/clipboardcase.toml index dcb0294d230..d3f91fb58d0 100644 --- a/tests/images/clipboardcase.toml +++ b/tests/images/clipboardcase.toml @@ -19,6 +19,7 @@ start = ["logkeeper", "compositor", "test-runner"] [programs.logkeeper] service = true syscap = ["logread"] +serves = ["log-carrier"] [programs.compositor] service = true @@ -26,10 +27,15 @@ serves = ["compositor", "prompt", "clipboard", "input", "display"] devices = ["pci:03:80:00/1af4:1050"] args = ["scale=2"] -# One process per virtio-serial function the kernel does not drive: the -# clipboard's channel, beside the kernel's console. +# One process per virtio-serial function: the one that names the clipboard's +# port is its own, and the console's goes to the row below. [programs.vdagent] drives = ["pci:07:80:00/1af4:1043"] receives = ["clipboard"] [programs.test-runner] + +# The VM's console, on whichever virtio-serial function names its port. +[programs.virtio-console] +drives = ["pci:07:80:00/1af4:1043"] +receives = ["log-carrier"] diff --git a/tests/images/drivescase.toml b/tests/images/drivescase.toml index e8ed7433d85..6549e616612 100644 --- a/tests/images/drivescase.toml +++ b/tests/images/drivescase.toml @@ -9,6 +9,7 @@ start = ["logkeeper", "test-runner"] [programs.logkeeper] service = true syscap = ["logread"] +serves = ["log-carrier"] # Started on each function the kernel does not drive, never by `[boot] start`. [programs.pciholder] @@ -19,3 +20,8 @@ drives = ["pci:00:ff:00/1234:11e8"] # none without it. [programs.test-runner] syscap = ["inventory", "dup"] + +# The VM's console, on whichever virtio-serial function names its port. +[programs.virtio-console] +drives = ["pci:07:80:00/1af4:1043"] +receives = ["log-carrier"] diff --git a/tests/images/hostedclangcase.toml b/tests/images/hostedclangcase.toml index 2037b54c7aa..7d791a4c657 100644 --- a/tests/images/hostedclangcase.toml +++ b/tests/images/hostedclangcase.toml @@ -7,6 +7,7 @@ start = ["logkeeper", "diskserver", "fileserver", "test-runner"] [programs.logkeeper] service = true syscap = ["logread"] +serves = ["log-carrier"] [programs.test-runner] @@ -28,3 +29,8 @@ receives = ["disk"] restart = true roles = ["data", "log", "boot"] receives = ["block"] + +# The VM's console, on whichever virtio-serial function names its port. +[programs.virtio-console] +drives = ["pci:07:80:00/1af4:1043"] +receives = ["log-carrier"] diff --git a/tests/images/jobcase.toml b/tests/images/jobcase.toml index 4ac95388b1c..736cf117861 100644 --- a/tests/images/jobcase.toml +++ b/tests/images/jobcase.toml @@ -7,6 +7,7 @@ start = ["logkeeper", "diskserver", "fileserver", "test-runner"] [programs.logkeeper] service = true syscap = ["logread"] +serves = ["log-carrier"] # One binary name per job, never a command line: the stdin path's `run ` # has the only parser. `power` because the one job ends the machine, asking @@ -49,3 +50,8 @@ receives = ["disk"] restart = true roles = ["data", "log", "boot"] receives = ["block"] + +# The VM's console, on whichever virtio-serial function names its port. +[programs.virtio-console] +drives = ["pci:07:80:00/1af4:1043"] +receives = ["log-carrier"] diff --git a/tests/images/launchcase.toml b/tests/images/launchcase.toml index f9c8e589ecf..0716b3a963f 100644 --- a/tests/images/launchcase.toml +++ b/tests/images/launchcase.toml @@ -8,6 +8,7 @@ start = ["logkeeper", "test-runner"] [programs.logkeeper] service = true syscap = ["logread"] +serves = ["log-carrier"] [programs.test-runner] starts = ["toybox"] @@ -17,3 +18,8 @@ starts = ["toybox"] [symlinks] "bin/cat" = "/system/bin/toybox" "bin/echo" = "/system/bin/toybox" + +# The VM's console, on whichever virtio-serial function names its port. +[programs.virtio-console] +drives = ["pci:07:80:00/1af4:1043"] +receives = ["log-carrier"] diff --git a/tests/images/libccase.toml b/tests/images/libccase.toml index 2c2edd1d22d..abd8faa6b0b 100644 --- a/tests/images/libccase.toml +++ b/tests/images/libccase.toml @@ -7,6 +7,7 @@ start = ["logkeeper", "test-runner"] [programs.logkeeper] service = true syscap = ["logread"] +serves = ["log-carrier"] # `roster` for `libc_read_restarts`, which orders a child's end after its # own read is parked by the kernel's roster, and `dup` to hand it on. @@ -20,3 +21,8 @@ syscap = ["dup", "roster"] [symlinks] "bin/echo" = "/system/bin/toybox" "bin/cat" = "/system/bin/toybox" + +# The VM's console, on whichever virtio-serial function names its port. +[programs.virtio-console] +drives = ["pci:07:80:00/1af4:1043"] +receives = ["log-carrier"] diff --git a/tests/images/netcase.toml b/tests/images/netcase.toml index 2b1a38d2d1c..18120340ab3 100644 --- a/tests/images/netcase.toml +++ b/tests/images/netcase.toml @@ -17,6 +17,7 @@ start = ["logkeeper", "diskserver", "fileserver", "netstack", "soundserver", "te [programs.logkeeper] service = true syscap = ["logread"] +serves = ["log-carrier"] # netstack holds the NIC's PCI function and drives it: the virtqueues, the register # window and the interrupt are its own, and the kernel keeps only the claim. @@ -67,3 +68,8 @@ receives = ["disk"] restart = true roles = ["data", "log", "boot"] receives = ["block"] + +# The VM's console, on whichever virtio-serial function names its port. +[programs.virtio-console] +drives = ["pci:07:80:00/1af4:1043"] +receives = ["log-carrier"] diff --git a/tests/images/panelcase.toml b/tests/images/panelcase.toml index 169e69f0565..a0d861a8812 100644 --- a/tests/images/panelcase.toml +++ b/tests/images/panelcase.toml @@ -13,8 +13,14 @@ start = ["logkeeper", "compositor"] [programs.logkeeper] service = true syscap = ["logread"] +serves = ["log-carrier"] [programs.compositor] service = true serves = ["compositor", "prompt", "clipboard", "input", "display"] devices = ["framebuffer"] + +# The VM's console, on whichever virtio-serial function names its port. +[programs.virtio-console] +drives = ["pci:07:80:00/1af4:1043"] +receives = ["log-carrier"] diff --git a/tests/images/proctreecase.toml b/tests/images/proctreecase.toml index 0ec846d590c..4eb02e22d52 100644 --- a/tests/images/proctreecase.toml +++ b/tests/images/proctreecase.toml @@ -27,6 +27,7 @@ folder = "read-write" [programs.logkeeper] service = true syscap = ["logread"] +serves = ["log-carrier"] # `roster` because the guest asks which of its tree still runs, and `dup` # because test-runner hands each binary a duplicate of its capability. @@ -81,3 +82,8 @@ receives = ["disk"] restart = true roles = ["data", "log", "boot"] receives = ["block"] + +# The VM's console, on whichever virtio-serial function names its port. +[programs.virtio-console] +drives = ["pci:07:80:00/1af4:1043"] +receives = ["log-carrier"] diff --git a/tests/images/scalecase.toml b/tests/images/scalecase.toml index 8bb5db7740d..4acf004be0e 100644 --- a/tests/images/scalecase.toml +++ b/tests/images/scalecase.toml @@ -15,6 +15,7 @@ start = ["logkeeper", "compositor", "ps2server", "test-runner"] [programs.logkeeper] service = true syscap = ["logread"] +serves = ["log-carrier"] [programs.compositor] service = true @@ -33,3 +34,8 @@ starts = ["toybox"] [programs.toybox] receives = ["display"] + +# The VM's console, on whichever virtio-serial function names its port. +[programs.virtio-console] +drives = ["pci:07:80:00/1af4:1043"] +receives = ["log-carrier"] diff --git a/tests/images/sessioncase.toml b/tests/images/sessioncase.toml index 90307fb1230..a52582ea381 100644 --- a/tests/images/sessioncase.toml +++ b/tests/images/sessioncase.toml @@ -21,6 +21,7 @@ start = ["logkeeper", "compositor", "ps2server", "test-runner"] [programs.logkeeper] service = true syscap = ["logread"] +serves = ["log-carrier"] # `roster`, which the job reads to see the holder outlive its shell and end # when forced; `dup`, because test-runner hands each job its capability. @@ -58,3 +59,8 @@ starts = ["hangupholder"] [programs.logout] receives = ["session-admin"] + +# The VM's console, on whichever virtio-serial function names its port. +[programs.virtio-console] +drives = ["pci:07:80:00/1af4:1043"] +receives = ["log-carrier"] diff --git a/tests/images/testcases.toml b/tests/images/testcases.toml index 55cb249e956..3b03da89f52 100644 --- a/tests/images/testcases.toml +++ b/tests/images/testcases.toml @@ -13,7 +13,7 @@ start = ["logkeeper", "diskserver", "fileserver", "soundserver", "acpiserver", " [programs.logkeeper] service = true syscap = ["logread"] -serves = ["log"] +serves = ["log", "log-carrier"] # The machine's ACPI fixed hardware, as `images/system.toml` serves it: on the T14 # every row of this image runs in ACPI mode, and on q35 the stop tests power off @@ -123,3 +123,8 @@ receives = ["disk"] restart = true roles = ["data", "log", "boot"] receives = ["block"] + +# The VM's console, on whichever virtio-serial function names its port. +[programs.virtio-console] +drives = ["pci:07:80:00/1af4:1043"] +receives = ["log-carrier"] diff --git a/tests/toyos.rs b/tests/toyos.rs index d5afe06f454..249e19ba036 100644 --- a/tests/toyos.rs +++ b/tests/toyos.rs @@ -145,7 +145,8 @@ const RUST_SKIP: &[&str] = &[ // Its product is what a diary record costs, which the `trace_record_cost` // metal row reads off the test kernel its flood needs. "trace_flood", - // It takes the machine down; `virt_fatal_halts_the_others_first` runs it. + // It takes the machine down; `virt_fatal_halts_the_others_first` and + // `a_fatal_report_reaches_the_uart_while_a_carrier_carries_the_log` run it. "panic_halts_first", // They claim QEMU's `edu` and `e1000e`, which only `virt_claim_lpi`'s // machine has. @@ -469,10 +470,17 @@ const MACHINE_TESTS: &[&str] = &[ // its DMA bypasses the unit: the supervisor refuses it to netstack by // name. Only QEMU can create such a function; q35 only. "a_virtio_function_without_access_platform_is_refused", - // The kernel's own virtio console created without `iommu_platform=on` - // on q35 with VT-d: the kernel tells it physical addresses, and the boot - // runs over it. q35 only. - "the_kernel_tells_a_bypassing_console_physical_addresses", + // The VM's console port unplugged under a running guest: its driver + // ends, logkeeper sees its carrier gone and the UART carries the log + // again. Three processes, the kernel and QEMU's hot-unplug, which no host + // build has. q35 only: no other machine carries a console port. + "the_uart_carries_the_log_again_once_the_console_carrier_goes", + // A kernel's fatal path on a guest whose console port carries the log, + // while three CPUs make records `klogd` advances past unwritten: the + // report reaches the harness on the UART. The kernel, the carrier's + // processes and the harness's two channels, which no host build has. + // q35 only: no other machine carries a console port. + "a_fatal_report_reaches_the_uart_while_a_carrier_carries_the_log", // An AMD-Vi unit left blocking every device, switched off and programmed // before the boot stick is driven behind it: the register sequence and the // DMA it lets through are no host test's, and the AMD bench is staged by @@ -4330,8 +4338,11 @@ fn run_machine_test(name: &str, test_config: &Path) -> Result<(), String> { "a_virtio_function_without_access_platform_is_refused" => { common::iommu::a_virtio_function_without_access_platform_is_refused() } - "the_kernel_tells_a_bypassing_console_physical_addresses" => { - common::iommu::the_kernel_tells_a_bypassing_console_physical_addresses() + "the_uart_carries_the_log_again_once_the_console_carrier_goes" => { + common::vmconsole::the_uart_carries_the_log_again_once_the_console_carrier_goes(test_config) + } + "a_fatal_report_reaches_the_uart_while_a_carrier_carries_the_log" => { + common::vmconsole::a_fatal_report_reaches_the_uart_while_a_carrier_carries_the_log(test_config) } "amdvi_firmware_left" => common::iommu::amdvi_firmware_left(test_config), "amdvi_selftest" => common::iommu::amdvi_selftest(test_config), @@ -5672,7 +5683,7 @@ fn pci_cap_selftest(log: &str) -> Result<(), String> { return Err(format!("the walk's self-test never ran:\n{log}")); }; // The count, not the absence of a FAILED line, which zero cases satisfy too. - if !verdict.contains("15/15") { + if !verdict.contains("selftest 6/6") { return Err(format!("not every crafted capability layout was answered: {verdict}")); } // And how those layouts *ended*, which is the split a claimed function's From a255028cc4f9821601aee87ed84cd67a94b0094b Mon Sep 17 00:00:00 2001 From: japabu Date: Sat, 24 Oct 2026 10:26:28 +0200 Subject: [PATCH 2/7] issues: the console port throttles, so QEMU no longer drops what the harness is slow to read The harness's console is now a `virtserialport`, which QEMU holds a transmit for until its chardev takes it, rather than the `virtconsole` that dropped what a full pipe refused. The record's exit named a `virtserialport` as the fix and a test of its own as the proof; that test was deleted for flaking on the very loss. In its place: the harness's stdout reader stopped for 20 s after 20 KB of a boot whose kernel flooded 380 records, twice, on q35 TCG at host loads of about 20; both boots reached the ready marker with all 380 records once each and no torn line. Co-Authored-By: Claude Opus 5.5 --- ...sole-output-the-harness-is-slow-to-read.md | 86 ------------------- 1 file changed, 86 deletions(-) delete mode 100644 issues/qemu-drops-console-output-the-harness-is-slow-to-read.md diff --git a/issues/qemu-drops-console-output-the-harness-is-slow-to-read.md b/issues/qemu-drops-console-output-the-harness-is-slow-to-read.md deleted file mode 100644 index 9981337f335..00000000000 --- a/issues/qemu-drops-console-output-the-harness-is-slow-to-read.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -status: open -kind: tooling -opened: 2026-09-26 ---- - -# QEMU drops console output that the harness is slow to read - -The harness reads each guest's virtio console from QEMU's stdout -(`-chardev stdio,id=cs0` behind a `virtconsole`, `tests/common/qemu.rs`). -QEMU makes that stdout non-blocking (`qemu_chr_open_fd`, `chardev/char-fd.c`, -v11.1.0). Its `virtconsole` then drops whatever a full pipe refuses: -`flush_buf` in `hw/char/virtio-console.c` throttles the port only -`if (!k->is_console)`, and its own comment says it is "silently dropping -console data on EAGAIN". It completes the transmit buffer either way, so the -guest cannot tell a line was lost. - -Seen on `wt/toyos-logtrack`, where `log_stream_stalled_reader` puts about a -mebibyte of program lines through the console after its flood: - -- A passing run printed `flood 001999`'s first 460 bytes followed directly by - the whole of `flood 002159`, on one console line. Every entry the guest's - queue holds is a whole line, or a 1024-byte piece of one, so a fragment of - that length is bytes the host dropped. -- A hung run printed a 555-byte fragment of `flood 005384`. After it came - nothing that `logd` put on the console between 5.6 s and 28 s, including - the runner's `===TEST_END===`, while `/log` holds every one of those lines. - The harness waited for the marker until its ceiling. The host's load - average was 40–50. - -Across 8 runs that day, 2 hung this way. Any test that waits for a line on a -virtio console under host load is exposed in the same way; the flood only -makes it likely. - -**`log_stream_stalled_reader` no longer reads that channel.** It boots with -`BootOptions::console_file`: the virtio console's output goes to a regular -file the harness follows and its input through a FIFO, so no write is refused. -Twenty runs of it after that change: 18 green; one red was the kernel's TLB -shootdown panic on a starved vCPU -(`issues/a-shootdown-panicked-on-a-cpu-the-host-starved.md`), which -killed a `logd` reader thread so it was never let go, and one had `logd` let -none of the eight stalled readers go in 90 s while every host guest slot was -held by other worktrees, green alone. Neither lost a console line. Once -`logd` synced every round the test's stimulus was too small for its readers -to stall, and its flood was made four times wider; twenty runs after that, at -host loads of 3.9 to 9.0, were 20 green. Every other test still reads the -stdio console. - -**`blockd_serves_partitions` is a sighting of the same loss.** In `650-libcllvm-whole.log` -(`wt/toyos-libcllvm`, load average 66–74) test-runner printed `blockd_io: PASS bench` at -35.962 s and the bench's process exited `code=0` at 35.978 s, and `===TEST_END -test_rs_blockd_io exit=0===` never reached the harness, which waited 3559 s until the run was -ended by hand. That boot read the stdio console. -Every userland line stops at 35.962 s while the kernel's own ten-second `sched:` and `PMM:` lines -go on, and user `mmap` held rises from 23 to 30 between 74.9 s and 3026.8 s, so a process kept -running through the silence; the capture cannot tell a dropped marker from a `logd` that stopped -forwarding. The tree moved `rust` from `aca5f527f` to `9151571ca`, which changes std's exported C -`malloc`, `free` and `realloc` in every Rust guest program, so reading it as this loss rests on -that change being off its path. Owner: the orchestrator. -That kernel's ten-second report is gone: the owner ruled on 2026-10-04, choosing "Remove it -entirely": "Delete the periodic report and its counters; hang triage uses the trace diary and -panic records instead." A later sighting tells a live kernel from a stopped one by those. - -**`crypto_instrument_prints_its_table` hangs this way, and a probe placed every -byte on the guest's side of the pipe.** On `wt/toyos-usbd-2` (host load average 10–15, -other worktrees' guests running) it hung in 9 of 12 runs beside eleven other rows, after -one of its parity jobs printed its table: about 80 KB in one burst. Each hang stopped -mid-line inside that burst. In one boot with temporary probes, logkeeper had read every -line from the runner's ring (its position equal to the ring's head, none refused or -suppressed), and it had written every rendered byte to its console handle, holding none -back. The kernel's console queue was empty and `klogd` owed no record. The runner then -exited by its own watchdog, which proved that it had printed `===TEST_END===` and waited -30 s for a command that never came. The bytes were lost after the guest's virtio -transmit, in QEMU's `virtconsole`. The same hang happened with no `usbd` in the image, and at the branch's base `e07354fb1`, which -has none of its changes: `STALLED: 973s of guard expired` with the same idle guest. - -## Exit condition - -The console chardev the harness reads cannot refuse a write — for example a -file chardev the harness follows, or a `virtserialport`, which QEMU throttles -instead of dropping. Shown by `log_stream_stalled_reader` green in 20 of 20 -runs beside a full suite. - -**`log_stream_stalled_reader` is deleted**, as a flaky test is, on the two -reds recorded here: `96763794e` took it out, and `cc291947e` then deleted -`BootOptions::console_file`, which only it set. From 4e1431405d9bc3ccb91a59001dd3343f9426dbbe Mon Sep 17 00:00:00 2001 From: japabu Date: Sat, 24 Oct 2026 11:06:31 +0200 Subject: [PATCH 3/7] review r1 of #1062: every fatal path gives the UART back first, and the carrier goes with the console's last handle - An IOMMU fault on a function this kernel drives halts the machine, and its line was logged before `halt_all_cpus` took the log back: `klogd` could advance past the one line that explains the halt. The drain now takes it back where it decides `Owner::Kernel`, before the line. - The console is a `deferred` object, and its carrier is released in `on_zero_handles` rather than on the last `Arc`: userland-visible lifetime rides the handle count, and a dying holder's releases run before its end is published, so logkeeper's death cannot leave the UART quiet behind a stranded reference. - The lines a carrier had not delivered when it went are recorded in `issues/the-console-loses-what-its-carrier-had-not-delivered.md`. - QEMU's source is no longer cited; the port's throttling is cited as measured (two 20 s stalls of the host's reader, no byte lost). - `images/system.toml` states the probe window as it is: a process holds the function it probes beside its own row's port until its verdict. - Both drivers share one `identify` in `toyos-virtio-serial-claim`. - `a_fatal_report_reaches_the_uart_while_a_carrier_carries_the_log` is deleted: its negative control showed it cannot see the race it was written for, and the restore test holds its harness claim. Co-Authored-By: Claude Opus 5.5 --- build/vmconsole.rs | 10 ++-- images/system.toml | 15 +++--- ...oses-what-its-carrier-had-not-delivered.md | 36 +++++++++++++ kernel/src/iommu/fault.rs | 5 ++ kernel/src/object/device.rs | 11 ++-- kernel/src/object/mod.rs | 3 +- lib/virtio-serial/claim/src/lib.rs | 51 ++++++++++++++++++- lib/virtio-serial/src/lib.rs | 3 +- system/logkeeper/src/carrier.rs | 16 +++--- system/vdagent/src/main.rs | 42 ++++----------- system/virtio-console/src/main.rs | 40 ++++----------- tests/common/vmconsole.rs | 39 -------------- tests/toyos.rs | 12 +---- 13 files changed, 144 insertions(+), 139 deletions(-) create mode 100644 issues/the-console-loses-what-its-carrier-had-not-delivered.md diff --git a/build/vmconsole.rs b/build/vmconsole.rs index a035a692ded..0f3703c7f8e 100644 --- a/build/vmconsole.rs +++ b/build/vmconsole.rs @@ -4,11 +4,11 @@ //! The bytes the host reads are never coloured. //! //! **A `virtserialport`, never a `virtconsole`**: QEMU 11.1.1 holds a -//! `virtserialport`'s transmit until its chardev takes it, and drops what a -//! `virtconsole`'s chardev refuses (`hw/char/virtio-console.c`, `flush_buf`). -//! Port 0 is a `virtconsole`'s alone, so the port is [`PORT_ID`] of a -//! multiport function. Firmware writes nothing on it, so its first byte is -//! the boot log's. +//! `virtserialport`'s transmit until its chardev takes it, measured — a host +//! reader stopped for 20 s lost no byte — where a `virtconsole` dropped what +//! a slow reader left. The port is [`PORT_ID`] of a multiport function. +//! Firmware writes nothing on it, measured, so its first byte is the boot +//! log's. use std::io::{self, Read, Write}; use std::os::fd::{AsFd, AsRawFd, BorrowedFd}; diff --git a/images/system.toml b/images/system.toml index b8cf58a6733..43394b525f7 100644 --- a/images/system.toml +++ b/images/system.toml @@ -130,12 +130,15 @@ serves = ["netstack"] devices = ["pci:02:00:00/1af4:1041"] # The shared clipboard with QEMU's host, and the VM's console: two drivers of -# one device, `1af4:1043`, each started on every virtio-serial function and -# keeping the one whose port carries its name; on another, it ends with exit 0 -# and the function goes to the next row. Each holds only its own authority: -# vdagent the compositor's clipboard port, virtio-console the log's carrier -# port, which no other row may receive. A machine with no such function, the -# T14 among them, starts neither. +# one device, `1af4:1043`, each started on every virtio-serial function in +# turn and keeping the one whose port carries its name; on another, it ends +# with exit 0 and the function goes to the next row. Each row's authority is +# its own — vdagent the compositor's clipboard port, virtio-console the log's +# carrier port, which no other row may receive — but while it probes a +# function that is not its own, a process holds that function's claim beside +# its row's port, until its verdict or its probe's bound: vdagent the +# console's function, which comes to it first. A machine with no such +# function, the T14 among them, starts neither. [programs.vdagent] drives = ["pci:07:80:00/1af4:1043"] receives = ["clipboard"] diff --git a/issues/the-console-loses-what-its-carrier-had-not-delivered.md b/issues/the-console-loses-what-its-carrier-had-not-delivered.md new file mode 100644 index 00000000000..bebafa261de --- /dev/null +++ b/issues/the-console-loses-what-its-carrier-had-not-delivered.md @@ -0,0 +1,36 @@ +--- +status: open +kind: defect +opened: 2026-10-24 +--- + +# The console loses what its carrier had not delivered when it went + +While `/system/bin/virtio-console` carries the log, the UART takes none of +it (`system/logkeeper/src/carrier.rs`, `kernel/src/log/console.rs`). When the +carrier goes, two runs of lines reach neither the console port nor the UART: + +- the bytes logkeeper sent the carrier that its channel had not taken, + `[delivered, sent)` in `Carrier`'s state; +- the kernel records `klogd` advanced past between the carrier's end and + logkeeper's word that the UART carries the log again — among them the + kernel's own record of a driver that crashed, the line that explains the + event. + +The same holds across logkeeper's own death, whose console's last handle +gives the log back: what it had read and not yet written, and the records +`klogd` advanced past before that handle went. `/log` keeps the first run +where logkeeper lived to write it, and the kernel's ring and black box the +second. + +**Evidence:** the code; `the_uart_carries_the_log_again_once_the_console_carrier_goes` +shows the restore itself (PR #1062). + +**Owner:** unheld; whoever next changes `carrier.rs`. + +**Exit:** a carrier that goes mid-stream leaves every line on one of the two +channels: logkeeper, which holds the replay and knows `delivered`, puts the +undelivered tail on the UART as it gives the log back, and the records +`klogd` advanced past since the carrier last delivered reach the UART too. +Shown by a guest test that unplugs the port with lines in flight and finds +each of them once on the merged capture. diff --git a/kernel/src/iommu/fault.rs b/kernel/src/iommu/fault.rs index 210cb5d6b96..9e502035710 100644 --- a/kernel/src/iommu/fault.rs +++ b/kernel/src/iommu/fault.rs @@ -203,6 +203,11 @@ pub fn report(unit: usize, count: &AtomicU32, fault: Fault) -> bool { Some(function) if crate::pcidev::note_fault(function.who.requester()) => Owner::Process, Some(_) => Owner::Kernel, }; + // A fault this kernel's own function made halts the machine: before its + // line, as every fatal path's first record. + if matches!(owner, Owner::Kernel) { + crate::log::console::on_the_uart_for_good(); + } let count = count.fetch_add(1, Ordering::Relaxed) + 1; // A stray's line is bounded by its count, never by the device: the next // line it writes says how many were read in between. diff --git a/kernel/src/object/device.rs b/kernel/src/object/device.rs index 8932bdc84cf..98fae6fd483 100644 --- a/kernel/src/object/device.rs +++ b/kernel/src/object/device.rs @@ -174,7 +174,7 @@ impl ConsoleObject { } /// Whether this holder carries the log off the machine, so the UART does - /// not (`log::console::carried_by`); its end lets go of it. + /// not (`log::console::carried_by`); its last handle lets go of it. pub fn carry(&self, elsewhere: bool) -> Result<(), kernel::carrier::Refused> { match elsewhere { true => crate::log::console::carried_by(self.holder()), @@ -190,10 +190,11 @@ impl ConsoleObject { } } -// Flushes the partial line on last handle instead of dropping it, and gives -// the log back to the UART if this holder carried it. -impl Drop for ConsoleObject { - fn drop(&mut self) { +// Flushes the partial line and gives the log back to the UART if this holder +// carried it, on last handle: a dying holder's releases run before its end is +// published, so its end never leaves the UART quiet. +impl ZeroHandles for ConsoleObject { + fn on_zero_handles(&self) { self.line.lock().finish(); crate::log::console::released_by(self.holder()); } diff --git a/kernel/src/object/mod.rs b/kernel/src/object/mod.rs index 47c664fdb5b..b59c059ef01 100644 --- a/kernel/src/object/mod.rs +++ b/kernel/src/object/mod.rs @@ -257,7 +257,8 @@ kobject! { immediate Namespace => namespace::Namespace, // A file's flush and cache reference ride the last `Arc`; `read`/`write` on a file never park. immediate File => file::FileObject, - immediate Console => device::ConsoleObject, + // Its last handle lets go of the log's carrier, which a stranded `Arc` must not hold. + deferred Console => device::ConsoleObject, // The authority is the rights on the handle; a handle going away *is* the whole event. immediate SysCap => syscap::SysCap, // The last handle's loss is the loss of the *ability to wait*, not proof the process should stop. diff --git a/lib/virtio-serial/claim/src/lib.rs b/lib/virtio-serial/claim/src/lib.rs index 5f9d1665158..f79f908c247 100644 --- a/lib/virtio-serial/claim/src/lib.rs +++ b/lib/virtio-serial/claim/src/lib.rs @@ -19,8 +19,9 @@ //! refusal's own name: every message after it is one that silently never //! arrives. -use std::time::Instant; +use std::time::{Duration, Instant}; +use toyos::poller::{Poller, READABLE}; use toyos::shm::SharedMemory; use toyos::{DmaRegion, PciDev}; use toyos_abi::syscall::SyscallError; @@ -31,6 +32,7 @@ use toyos_virtio::pci::Live; use toyos_virtio::queue::{Buffer, Parts, Used, Virtqueue}; use toyos_virtio_serial::control::{self, Control}; +use toyos_virtio_serial::probe::{Probe, Step, Verdict}; use toyos_virtio_serial::{CONTROL_BUFFERS, MAX_PORTS}; /// §5.3.3: the device has control queues and more than one port. @@ -372,3 +374,50 @@ impl Channel { len } } + +/// How long the device may take to name its ports. Policy, far past what QEMU +/// 11.1.1 takes, measured: it adds and names them at the driver's first +/// messages. +pub const PROBE_BOUND: Duration = Duration::from_secs(5); + +/// What the device said its ports are. +pub enum Identified { + /// A port carries the driver's name; the control messages about known + /// ports that came with the names, the driver's to answer. + Ours(Vec), + /// Every port the device added is named, by these names, and none is the + /// driver's. + Foreign(Vec), + /// The device named no port of the driver's within [`PROBE_BOUND`]. + Unnamed, +} + +/// Read the device's control messages, answering each `DEVICE_ADD`, until +/// `probe` has a verdict or [`PROBE_BOUND`] is spent, parking on `poller`. +/// A device that misnames its ports ends the program, `who`'s, by name. +pub fn identify(channel: &mut Channel, probe: &mut Probe<'_>, poller: &Poller, who: &str) -> Identified { + let deadline = Instant::now() + PROBE_BOUND; + let mut held = Vec::new(); + loop { + while let Some(received) = channel.control() { + match probe.take(received.bytes()) { + Ok(Step::Reply(reply)) => channel.send_control(reply), + Ok(Step::Driver(control)) => held.push(control), + Ok(Step::Nothing) => {} + Err(why) => panic!("{who}: {why}"), + } + } + match probe.verdict() { + Verdict::Ours => return Identified::Ours(held), + Verdict::Foreign(names) => return Identified::Foreign(names), + Verdict::Probing => {} + } + let left = deadline.saturating_duration_since(Instant::now()); + if left.is_zero() { + return Identified::Unnamed; + } + poller.watch(channel.claim(), READABLE, 0); + poller.wait(1, u64::try_from(left.as_nanos()).unwrap_or(u64::MAX), |_| {}); + channel.take_interrupt(); + } +} diff --git a/lib/virtio-serial/src/lib.rs b/lib/virtio-serial/src/lib.rs index 1f1c40a69e4..2e1abed36de 100644 --- a/lib/virtio-serial/src/lib.rs +++ b/lib/virtio-serial/src/lib.rs @@ -28,8 +28,7 @@ pub mod probe; pub const CONSOLE_PORT: &str = "org.toyos.console"; /// Every virtio-serial function the launch makes has `max_ports` this: port 0, -/// which QEMU 11.1.1 reserves for a `virtconsole` (`hw/char/virtio-console.c`) -/// and nothing here uses, and the function's one named port. +/// which nothing here uses, and the function's one named port. pub const MAX_PORTS: u32 = 2; /// The id of a function's one named port, `nr=` at the launch. diff --git a/system/logkeeper/src/carrier.rs b/system/logkeeper/src/carrier.rs index dfa48892399..b485b707d18 100644 --- a/system/logkeeper/src/carrier.rs +++ b/system/logkeeper/src/carrier.rs @@ -14,7 +14,7 @@ //! back to the UART with it, so a crash here cannot leave the UART quiet. //! //! What the carrier's channel had not taken when the carrier went is on -//! neither: `/log` has it. +//! neither (`issues/the-console-loses-what-its-carrier-had-not-delivered.md`). use std::sync::{Arc, Condvar, Mutex, MutexGuard}; use std::time::{Duration, Instant}; @@ -234,13 +234,13 @@ impl Carrier { let mut ready = [false; 2]; poller.wait(1, u64::MAX, |token| ready[token as usize] = true); - if ready[ACCEPT as usize] { - if let Ok(conn) = acceptor.accept() { - if admitted.is_some() || asking.is_some() { - say!("logkeeper: refusing a second console carrier: one is served"); - } else { - asking = Some((conn, FrameRx::new())); - } + if ready[ACCEPT as usize] + && let Ok(conn) = acceptor.accept() + { + if admitted.is_some() || asking.is_some() { + say!("logkeeper: refusing a second console carrier: one is served"); + } else { + asking = Some((conn, FrameRx::new())); } } if !ready[CONNECTION as usize] { diff --git a/system/vdagent/src/main.rs b/system/vdagent/src/main.rs index 9ade8b2dc47..ce1af06f99a 100644 --- a/system/vdagent/src/main.rs +++ b/system/vdagent/src/main.rs @@ -42,9 +42,9 @@ use toyos_vdagent::{LARGEST_TRANSMIT, PORT_NAME}; use toyos_vdagent_bridge::sets::Sets; use toyos_vdagent_bridge::{Bridge, Effect}; use toyos_virtio_serial::control::{Control, Event}; -use toyos_virtio_serial::probe::{Probe, Step, Verdict}; +use toyos_virtio_serial::probe::{Probe, Step}; use toyos_virtio_serial::PORT_ID; -use toyos_virtio_serial_claim::{Channel, Shape}; +use toyos_virtio_serial_claim::{Channel, Identified, Shape, PROBE_BOUND}; use window::watcher::{Watched, Watcher}; /// One declaration of the clipboard's bound, read by the codec and the @@ -64,9 +64,6 @@ const TRANSMIT_BOUND: Duration = Duration::from_secs(5); /// every message owed, in one element. const SHAPE: Shape = Shape { receive: Some((16, 4096)), transmit: (1, LARGEST_TRANSMIT) }; -/// How long the device may take to name its ports. Policy, far past what QEMU -/// 11.1.1 takes: it adds and names them at the driver's first messages. -const PROBE_BOUND: Duration = Duration::from_secs(5); const TOKEN_CLAIM: u64 = 0; const TOKEN_WATCHER: u64 = 1; @@ -231,36 +228,19 @@ fn main() { } } -/// Read the device's control messages until a port carries the agent's name, -/// and answer the messages for it that came with them; or end with exit 0 -/// where none does: the function is another driver's. +/// The agent's function's ports identified, and the messages about them that +/// came first; or the end with exit 0 where none is the agent's: the +/// function is another driver's. fn identify(channel: &mut Channel, probe: &mut Probe<'_>, poller: &Poller) -> Vec { - let deadline = Instant::now() + PROBE_BOUND; - let mut held = Vec::new(); - loop { - while let Some(received) = channel.control() { - match probe.take(received.bytes()) { - Ok(Step::Reply(reply)) => channel.send_control(reply), - Ok(Step::Driver(control)) => held.push(control), - Ok(Step::Nothing) => {} - Err(why) => panic!("vdagent: {why}"), - } - } - match probe.verdict() { - Verdict::Ours => return held, - Verdict::Foreign(names) => { - say!("vdagent: the function's ports are {names:?}, none {PORT_NAME}; another row drives it"); - std::process::exit(0); - } - Verdict::Probing => {} + match toyos_virtio_serial_claim::identify(channel, probe, poller, "vdagent") { + Identified::Ours(held) => held, + Identified::Foreign(names) => { + say!("vdagent: the function's ports are {names:?}, none {PORT_NAME}; another row drives it"); + std::process::exit(0); } - let left = deadline.saturating_duration_since(Instant::now()); - if left.is_zero() { + Identified::Unnamed => { say!("vdagent: the function named no port {PORT_NAME} in {PROBE_BOUND:?}; another row drives it"); std::process::exit(0); } - poller.watch(channel.claim(), READABLE, TOKEN_CLAIM); - poller.wait(1, u64::try_from(left.as_nanos()).unwrap_or(u64::MAX), |_| {}); - channel.take_interrupt(); } } diff --git a/system/virtio-console/src/main.rs b/system/virtio-console/src/main.rs index 5ef9637c535..ada676b3036 100644 --- a/system/virtio-console/src/main.rs +++ b/system/virtio-console/src/main.rs @@ -30,9 +30,9 @@ use toyos::{PciDev, Pipe}; use toyos_abi::syscall::{PciId, SyscallError}; use toyos_manifest::device::PciMatch; use toyos_virtio_serial::control::{Control, Event}; -use toyos_virtio_serial::probe::{Probe, Step, Verdict}; +use toyos_virtio_serial::probe::{Probe, Step}; use toyos_virtio_serial::{CONSOLE_PORT, PORT_ID}; -use toyos_virtio_serial_claim::{Channel, Shape}; +use toyos_virtio_serial_claim::{Channel, Identified, Shape, PROBE_BOUND}; /// A communication controller of class `07:80:00`, which virtio 1.2 §5.3.1 /// numbers `0x1040 + 3`: the matcher its row drives, and the label its claim @@ -43,10 +43,6 @@ const VIRTIO_SERIAL: PciMatch = PciMatch::new([0x07, 0x80, 0x00], Some(PciId { v /// one out, of a size that makes each a few hundred lines. const SHAPE: Shape = Shape { receive: None, transmit: (4, 32 * 1024) }; -/// How long the device may take to name its ports. Policy, far past what QEMU -/// 11.1.1 takes: it adds and names them at the driver's first messages. -const PROBE_BOUND: Duration = Duration::from_secs(5); - /// How long the host may take nothing before that is said. const SILENT: Duration = Duration::from_secs(10); @@ -93,35 +89,19 @@ fn main() { } } -/// Read the device's control messages until a port carries the console's -/// name, or end with exit 0 where none does: the function is another -/// driver's. +/// The console's function identified; or the end with exit 0 where none of +/// its ports is the console's: the function is another driver's. fn identify(channel: &mut Channel, probe: &mut Probe<'_>, poller: &Poller) { - let deadline = Instant::now() + PROBE_BOUND; - loop { - while let Some(received) = channel.control() { - match probe.take(received.bytes()) { - Ok(Step::Reply(reply)) => channel.send_control(reply), - Ok(Step::Driver(_) | Step::Nothing) => {} - Err(why) => panic!("virtio-console: {why}"), - } - } - match probe.verdict() { - Verdict::Ours => return, - Verdict::Foreign(names) => { - say!("virtio-console: the function's ports are {names:?}, none {CONSOLE_PORT}; another row drives it"); - std::process::exit(0); - } - Verdict::Probing => {} + match toyos_virtio_serial_claim::identify(channel, probe, poller, "virtio-console") { + Identified::Ours(_) => {} + Identified::Foreign(names) => { + say!("virtio-console: the function's ports are {names:?}, none {CONSOLE_PORT}; another row drives it"); + std::process::exit(0); } - let left = deadline.saturating_duration_since(Instant::now()); - if left.is_zero() { + Identified::Unnamed => { say!("virtio-console: the function named no port {CONSOLE_PORT} in {PROBE_BOUND:?}; another row drives it"); std::process::exit(0); } - poller.watch(channel.claim(), READABLE, TOKEN_CLAIM); - poller.wait(1, u64::try_from(left.as_nanos()).unwrap_or(u64::MAX), |_| {}); - channel.take_interrupt(); } } diff --git a/tests/common/vmconsole.rs b/tests/common/vmconsole.rs index 5bf51551b17..1d72aea95c8 100644 --- a/tests/common/vmconsole.rs +++ b/tests/common/vmconsole.rs @@ -1,7 +1,6 @@ //! The VM's console on a virtio machine: `/system/bin/virtio-console` //! carrying the log on its port, and the UART taking it back when it goes. -use std::io::Write; use std::path::Path; use super::qemu::{self, BootOptions, QemuInstance, QmpMonitor}; @@ -44,41 +43,3 @@ pub fn the_uart_carries_the_log_again_once_the_console_carrier_goes(test_config: Ok(()) } -/// What the fatal path says once it has armed the reboot that ends it -/// (`kernel/src/panic_reboot.rs`): a line only the UART carries. -const REBOOT_ARMED: &str = "panic: rebooting in"; - -/// **A kernel's fatal report reaches the harness on the UART while the -/// console port's carrier carries the log**, three CPUs making kernel -/// records as fast as they can — the records `klogd` advances past unwritten -/// while it does. `test_rs_panic_halts_first` goes fatal by `SYS_DEBUG` once -/// its siblings run: its line and the fatal path's after it are the UART's -/// alone, the carrier being one of the processes the fatal path stopped. -/// That a fatal path's first record is never advanced past is the ordering -/// `kernel/src/log/console.rs` states, not this test's: the halt makes the -/// same claim microseconds later. q35 only: no other machine of this suite -/// carries a console port. -pub fn a_fatal_report_reaches_the_uart_while_a_carrier_carries_the_log(test_config: &Path) -> Result<(), String> { - let fatal = qemu::build_toyos_bin( - qemu::SUITE_ARCH, - &super::compile::repo_root().join("tests/guest/rust-tests"), - "panic_halts_first", - ); - let options = BootOptions { - smp: crate::STOP_CPUS, - kernel_features: crate::ACTUATOR_KERNEL, - extra_root_files: vec![("bin/test_rs_panic_halts_first".to_string(), fatal)], - ..Default::default() - }; - let mut qemu = QemuInstance::boot_with_options(test_config, &[], &[], options); - let mut console = Serial::boot(&qemu).text().to_string(); - qemu::await_marker(&mut qemu, &mut console, QUIETED, "logkeeper to quiet the UART for its carrier")?; - let from = console.len(); - writeln!(qemu.stdin_mut(), "run test_rs_panic_halts_first").map_err(|e| e.to_string())?; - qemu.flush_stdin(); - qemu::await_marker_new(&mut qemu, &mut console, REBOOT_ARMED, from, "the fatal path to arm its reboot")?; - let after = Serial::named("console after the fatal halt", console[from..].to_string()); - let nonce = after.must_say_after(crate::FATAL_HALT_NONCE, REBOOT_ARMED)?; - eprintln!(" [vmconsole] {}", nonce.trim()); - Ok(()) -} diff --git a/tests/toyos.rs b/tests/toyos.rs index 249e19ba036..cbaa8adcd54 100644 --- a/tests/toyos.rs +++ b/tests/toyos.rs @@ -145,8 +145,7 @@ const RUST_SKIP: &[&str] = &[ // Its product is what a diary record costs, which the `trace_record_cost` // metal row reads off the test kernel its flood needs. "trace_flood", - // It takes the machine down; `virt_fatal_halts_the_others_first` and - // `a_fatal_report_reaches_the_uart_while_a_carrier_carries_the_log` run it. + // It takes the machine down; `virt_fatal_halts_the_others_first` runs it. "panic_halts_first", // They claim QEMU's `edu` and `e1000e`, which only `virt_claim_lpi`'s // machine has. @@ -475,12 +474,6 @@ const MACHINE_TESTS: &[&str] = &[ // again. Three processes, the kernel and QEMU's hot-unplug, which no host // build has. q35 only: no other machine carries a console port. "the_uart_carries_the_log_again_once_the_console_carrier_goes", - // A kernel's fatal path on a guest whose console port carries the log, - // while three CPUs make records `klogd` advances past unwritten: the - // report reaches the harness on the UART. The kernel, the carrier's - // processes and the harness's two channels, which no host build has. - // q35 only: no other machine carries a console port. - "a_fatal_report_reaches_the_uart_while_a_carrier_carries_the_log", // An AMD-Vi unit left blocking every device, switched off and programmed // before the boot stick is driven behind it: the register sequence and the // DMA it lets through are no host test's, and the AMD bench is staged by @@ -4341,9 +4334,6 @@ fn run_machine_test(name: &str, test_config: &Path) -> Result<(), String> { "the_uart_carries_the_log_again_once_the_console_carrier_goes" => { common::vmconsole::the_uart_carries_the_log_again_once_the_console_carrier_goes(test_config) } - "a_fatal_report_reaches_the_uart_while_a_carrier_carries_the_log" => { - common::vmconsole::a_fatal_report_reaches_the_uart_while_a_carrier_carries_the_log(test_config) - } "amdvi_firmware_left" => common::iommu::amdvi_firmware_left(test_config), "amdvi_selftest" => common::iommu::amdvi_selftest(test_config), "nested_nmi_is_loud" => faults::nested_nmi_is_loud(test_config), From c5a64029afeffb793ae4bc53157bc87f935c2612 Mon Sep 17 00:00:00 2001 From: japabu Date: Sat, 24 Oct 2026 11:55:29 +0200 Subject: [PATCH 4/7] review r1 of #1062: a machine that refuses every claim keeps its UART as the console The full q35 suite at the head found three rows the rewired harness broke: - `usbd_refused_without_smi_filter` and `usbd_refused_without_amdvi_remapping` boot machines whose every claim the kernel refuses. The console's driver never runs there, so the UART never goes quiet and the harness, which publishes UART lines only from the quiet line on, saw nothing. A shape now says whether its machine has the console's port: not `HeadlessNoIommu` nor the two AMD-Vi machines, whose UART is their console on stdio as metal-sim's is. - `iommu_virtio_platform` counted the virtio functions the kernel drives, which is now none: it asserts that, and the no-unit half also expects the console row's refused claim on the clipboard's function. The fourth red, `netstack_holders_exported_to_children`, hangs at the same step on `origin/main` (`2a5b446b0`); it is recorded in `issues/socket-export-hangs-at-its-race.md`. Co-Authored-By: Claude Opus 5.5 --- issues/socket-export-hangs-at-its-race.md | 29 +++++++++ tests/common/iommu.rs | 76 ++++++----------------- tests/common/qemu.rs | 23 +++++-- 3 files changed, 64 insertions(+), 64 deletions(-) create mode 100644 issues/socket-export-hangs-at-its-race.md diff --git a/issues/socket-export-hangs-at-its-race.md b/issues/socket-export-hangs-at-its-race.md new file mode 100644 index 00000000000..ef2c01826bc --- /dev/null +++ b/issues/socket-export-hangs-at-its-race.md @@ -0,0 +1,29 @@ +--- +status: open +kind: defect +opened: 2026-10-24 +--- + +# `test_rs_socket_export` hangs at its race + +`netstack_holders_exported_to_children` (`tests/toyos.rs`) runs +`test_rs_socket_export`, which says its first claim — "a listener and a +datagram socket exported to a child: it accepted and spoke on them, and so +did the parent after it" — spawns its racer, and then neither the job nor the +guest says anything more: the run's guard expires with the guest silent for +its whole span (`STALLED`). The host's peer is waiting on the two `racing` +datagrams of `sockets_exported_to_a_child`'s race. + +**Evidence** (q35 TCG, host load averages 20 to 28): on `origin/main` +`2a5b446b0` it hung 1 run of 1 (965 s, silent throughout); on PR #1062's +branch it hung 2 runs of 3 and passed once. The capture of the base run is in +PR #1062's evidence (`pr-1062`, `guest/socket-export-hang-base.log`). Nothing +in either says which side is waiting: the datagram, the racer's spawn or its +stdout line. + +**Owner:** unheld; whoever next holds the socket-sharing track +(`issues/a-socket-libc-or-socket2-has-not-bound-is-not-shared-with-a-child.md`). + +**Exit:** the race is read to its end — which party waits on what at the +hang — and `netstack_holders_exported_to_children` passes 20 of 20 under the +load it hung at. diff --git a/tests/common/iommu.rs b/tests/common/iommu.rs index 3ff1be77bb0..302358d7935 100644 --- a/tests/common/iommu.rs +++ b/tests/common/iommu.rs @@ -25,12 +25,8 @@ fn netcase() -> std::path::PathBuf { /// whatever the tables say — unless it is created with `iommu_platform=on` /// (`hw/virtio/virtio-bus.c:86-99`, `hw/virtio/virtio-pci.c:1400-1405` at /// v11.1.1), and under identity mapping the two are indistinguishable. So the -/// argv says which functions were created behind a unit and the kernel's -/// records say which of those it drives negotiated `VIRTIO_F_ACCESS_PLATFORM`; -/// the NIC's and the sound function's drivers are processes', whose -/// negotiation is `toyos-virtio`'s and host-tested there. On [`Profile::HeadlessNoIommu`] -/// the guest reports `n` because QEMU never *offers* the bit -/// (`hw/virtio/virtio-bus.c:87-94`), not because the driver declined. +/// argv says which functions were created behind a unit; every driver is a +/// process's, whose negotiation is `toyos-virtio`'s and host-tested there. pub fn iommu_virtio_platform() -> Result<(), String> { for profile in [Profile::Headless, Profile::HeadlessNoIommu] { let name = if profile.iommu().is_some() { "headless" } else { "headless-no-iommu" }; @@ -104,47 +100,14 @@ pub fn iommu_virtio_platform() -> Result<(), String> { no_unit_is_no_claim(&log)?; } - // Every function the kernel drives itself: all QEMU created but the - // NIC, the sound function and the clipboard's channel, whose drivers - // are processes. - let expected = created.len() - 3; - let mut negotiated = Vec::new(); - for line in log.text().lines().filter(|line| is_kernel_line(line)) { - let Some(rest) = line.split("VirtIO: PCI ").nth(1) else { continue }; - let Some((who, _)) = rest.split_once(' ') else { - return Err(format!("{name}: unreadable feature line: {line:?}")); - }; - let fields = unit_fields(line); - let Some(accepted) = fields.get("access_platform") else { continue }; - negotiated.push((who.to_string(), accepted == "y")); - } - if negotiated.len() != expected { - return Err(format!( - "{name}: QEMU created {} virtio function(s), {expected} of them for the kernel's \ - own drivers, and the kernel negotiated features with {} — \ - {negotiated:?} against {created:?}", - created.len(), - negotiated.len() - )); - } - let enumerated = enumerated_functions(&log); - for (who, accepted) in &negotiated { - if *accepted != behind_unit { - return Err(format!( - "{name}: {who} negotiated VIRTIO_F_ACCESS_PLATFORM = {accepted} where \ - {behind_unit} is owed — the unit exists = {behind_unit}" - )); - } - if !enumerated.contains(who) { - return Err(format!( - "{name}: {who} negotiated features and the PCI walk enumerated \ - {enumerated:?} — the driver is naming a function this machine does not have" - )); - } + // The kernel drives no virtio function: every one is a process's, whose + // negotiation is `toyos-virtio`'s and host-tested there. + if let Some(line) = log.text().lines().find(|l| is_kernel_line(l) && l.contains("VirtIO: PCI ")) { + return Err(format!("{name}: the kernel negotiated a virtio function's features: {line:?}")); } eprintln!( - " [iommu] {name}: the kernel's {} virtio function(s) negotiated, behind a unit = {behind_unit}; {}", - negotiated.len(), + " [iommu] {name}: {} virtio function(s), behind a unit = {behind_unit}; {}", + created.len(), if behind_unit { "the NIC's claim handed over" } else { "every claim refused for want of a unit" } ); } @@ -163,6 +126,11 @@ const XHCI_AT: &str = "00:01.0"; /// `tests/images/netcase.toml`'s soundserver row claims. const SOUND_AT: &str = "00:04.0"; +/// The slot QEMU's `-device` order puts the clipboard's virtio-serial +/// function on, which `tests/images/netcase.toml`'s virtio-console row is +/// started on in turn. +const AGENT_AT: &str = "00:05.0"; + /// Why a machine with no unit hands no function over, in the kernel's words. const NOT_REMAPPED: &str = "its interrupts would not be remapped on this machine"; @@ -184,7 +152,7 @@ fn not_handed_over(at: &str, why: &str) -> String { /// would fail. The NVMe controller and the virtio-sound function are refused /// the same. fn no_unit_is_no_claim(log: &Serial) -> Result<(), String> { - refused_claim(log, NETSTACK_CLAIMS, NOT_REMAPPED, &[XHCI_AT, NVME_AT, SOUND_AT])?; + refused_claim(log, NETSTACK_CLAIMS, NOT_REMAPPED, &[XHCI_AT, NVME_AT, SOUND_AT, AGENT_AT])?; log.must_say(¬_handed_over(XHCI_AT, NOT_REMAPPED))?; log.must_say(&format!( "supervisor: usbd was not started on PCI {XHCI_AT}: pci:0c:03:30/1033:0194 is on this machine and could not be handed over" @@ -195,6 +163,10 @@ fn no_unit_is_no_claim(log: &Serial) -> Result<(), String> { log.must_say(&format!( "supervisor: nvme was not started on PCI {NVME_AT}: pci:01:08:02 is on this machine and could not be handed over" ))?; + log.must_say(¬_handed_over(AGENT_AT, NOT_REMAPPED))?; + log.must_say(&format!( + "supervisor: virtio-console was not started on PCI {AGENT_AT}: pci:07:80:00/1af4:1043 is on this machine and could not be handed over" + ))?; // And this machine handed *nothing* over, which is more than the claim's // own refusal says: with no unit there is no function any process could be // given an address space for. @@ -312,18 +284,6 @@ pub(crate) fn unit_fields(line: &str) -> BTreeMap { .collect() } -/// Every function `pci::enumerate` printed. Anchored on the class field that -/// follows the address, so `pcidev: PCI 00:02.0 [...]` is not one of them. -fn enumerated_functions(log: &Serial) -> BTreeSet { - log.text() - .lines() - .filter_map(|line| { - let (bdf, tail) = line.split("PCI ").nth(1)?.split_once(' ')?; - tail.starts_with('[').then(|| bdf.to_string()) - }) - .collect() -} - /// What the kernel says of the one unit QEMU's IVRS describes, as an 11h over /// a superseded 10h (`lib/acpi/tests/ivrs.rs` decodes the same table). const AMDVI_UNIT: &str = "iommu: amdvi unit0 @0xfed80000 seg=0 dev=00:02.0 ivhd=0x11"; diff --git a/tests/common/qemu.rs b/tests/common/qemu.rs index a6ce68f6c39..eec53decc5a 100644 --- a/tests/common/qemu.rs +++ b/tests/common/qemu.rs @@ -946,6 +946,12 @@ struct Shape { /// virtio-sound, the console on virtio-serial, and the clipboard's /// channel on a virtio-serial function of its own. virtio: Virtio, + /// The console's port, on its own virtio-serial function, which + /// `/system/bin/virtio-console` carries the log on. A machine whose every + /// claim the kernel refuses — no unit, or one that cannot remap or filter + /// a claim's messages — has none: its driver could never run, so its + /// UART is its console. + console_port: bool, nic: Nic, /// The `-device` argument for each xHCI controller, port and slot counts /// included. A list because a machine can have more than one and the T14 @@ -1063,6 +1069,7 @@ impl Profile { vga: "std", panel: None, virtio: Virtio::Absent, + console_port: false, nic: Nic::Absent, xhci: &[XHCI_DEFAULT], usb: &[], @@ -1078,6 +1085,7 @@ impl Profile { vga: "none", panel: None, virtio: Virtio::Present, + console_port: true, nic: Nic::Virtio, xhci: &[XHCI_DEFAULT], usb: &["usb-kbd,bus=xhci.0"], @@ -1097,6 +1105,7 @@ impl Profile { // expresses. panel: Some((1920, 1080)), virtio: Virtio::Absent, + console_port: false, nic: Nic::Absent, xhci: &[XHCI_DEFAULT], usb: &[], @@ -1114,13 +1123,13 @@ impl Profile { Self::MetalBesideAHostDisk => Shape { storage: Storage::Stick { beside: Beside::HostDisk }, ..Self::Metal.shape() }, Self::MetalAmdVi => Shape { iommu: Some(Unit::AmdVi { intremap: false }), ..Self::Metal.shape() }, Self::HeadlessUsbSpareAmdVi => { - Shape { iommu: Some(Unit::AmdVi { intremap: true }), ..Self::HeadlessUsbSpare.shape() } + Shape { iommu: Some(Unit::AmdVi { intremap: true }), console_port: false, ..Self::HeadlessUsbSpare.shape() } } Self::HeadlessUsbSpareAmdViNoRemap => { Shape { iommu: Some(Unit::AmdVi { intremap: false }), ..Self::HeadlessUsbSpareAmdVi.shape() } } Self::MetalAmdViTestdev => Shape { probes: Probes::Testdevs, ..Self::MetalAmdVi.shape() }, - Self::HeadlessNoIommu => Shape { iommu: None, ..Self::Headless.shape() }, + Self::HeadlessNoIommu => Shape { iommu: None, console_port: false, ..Self::Headless.shape() }, Self::HeadlessNoUsb => Shape { xhci: &[], usb: &[], ..Self::Headless.shape() }, Self::HeadlessVirtioGpu => Shape { virtio_gpu: true, ..Self::Headless.shape() }, Self::HeadlessNicBehindPort => Shape { nic: Nic::BehindPort, ..Self::Headless.shape() }, @@ -1459,7 +1468,7 @@ impl QemuInstance { rust_tests: &[(String, Vec)], options: BootOptions, ) -> Self { - if options.profile.shape().virtio.present() { + if options.profile.shape().console_port { carries_its_console(config).unwrap_or_else(|why| panic!("[qemu] {}: {why}", config.display())); } let mut features: Vec<&str> = kernel_of(&options); @@ -2518,6 +2527,8 @@ fn qemu_command( // `qemu-vdagent`. toyos_build::vdagent::available(arch).unwrap_or_else(|why| panic!("[qemu] {why}")); qemu.args(toyos_build::vdagent::args(platform)); + } + if shape.console_port { // The console's port on stdio, which `/system/bin/virtio-console` // carries the log on; the UART on the socket this harness listens on // ([`UartConsole`]), which takes the commands typed at the guest and @@ -2532,9 +2543,9 @@ fn qemu_command( } else if options.mute { qemu.arg("-serial").arg("none"); } else { - // The 16550 *is* the console here: no virtio-serial exists, so the + // The 16550 *is* the console here: no console port exists, so the // kernel's log ring drains to it and the guest reads its commands - // off it. signal=off matches the virtio console above, so a ^C in + // off it. signal=off matches the console port above, so a ^C in // the stream reaches the guest rather than killing QEMU. qemu.arg("-chardev") .arg("stdio,id=uart0,signal=off") @@ -2610,7 +2621,7 @@ fn spawn_and_wait_ready(mut qemu: Command, options: BootOptions, files: Files) - .stderr(Stdio::inherit()); // Listening before QEMU starts, which connects as it makes the chardev. - let virtio = options.profile.shape().virtio.present(); + let virtio = options.profile.shape().console_port; let listener = virtio.then(|| { UnixListener::bind(sockets.dir.join(UART_SOCKET)).unwrap_or_else(|e| panic!("[qemu] the UART's socket: {e}")) }); From 779e0d4703d0fdfe6ae37612848d879863521661 Mon Sep 17 00:00:00 2001 From: japabu Date: Sat, 24 Oct 2026 12:27:25 +0200 Subject: [PATCH 5/7] issues: the socket-export stall is the record #1060 filed The hang review r1's suite run met at test_rs_socket_export's race is `issues/two-holders-racing-on-an-exported-socket-can-both-stall.md`, which PR #1060 filed: its own record goes, and its observation (two of three at this branch, one of one on main) joins that one. Co-Authored-By: Claude Opus 5.5 --- issues/socket-export-hangs-at-its-race.md | 29 ------------------- ...ng-on-an-exported-socket-can-both-stall.md | 5 ++++ 2 files changed, 5 insertions(+), 29 deletions(-) delete mode 100644 issues/socket-export-hangs-at-its-race.md diff --git a/issues/socket-export-hangs-at-its-race.md b/issues/socket-export-hangs-at-its-race.md deleted file mode 100644 index ef2c01826bc..00000000000 --- a/issues/socket-export-hangs-at-its-race.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -status: open -kind: defect -opened: 2026-10-24 ---- - -# `test_rs_socket_export` hangs at its race - -`netstack_holders_exported_to_children` (`tests/toyos.rs`) runs -`test_rs_socket_export`, which says its first claim — "a listener and a -datagram socket exported to a child: it accepted and spoke on them, and so -did the parent after it" — spawns its racer, and then neither the job nor the -guest says anything more: the run's guard expires with the guest silent for -its whole span (`STALLED`). The host's peer is waiting on the two `racing` -datagrams of `sockets_exported_to_a_child`'s race. - -**Evidence** (q35 TCG, host load averages 20 to 28): on `origin/main` -`2a5b446b0` it hung 1 run of 1 (965 s, silent throughout); on PR #1062's -branch it hung 2 runs of 3 and passed once. The capture of the base run is in -PR #1062's evidence (`pr-1062`, `guest/socket-export-hang-base.log`). Nothing -in either says which side is waiting: the datagram, the racer's spawn or its -stdout line. - -**Owner:** unheld; whoever next holds the socket-sharing track -(`issues/a-socket-libc-or-socket2-has-not-bound-is-not-shared-with-a-child.md`). - -**Exit:** the race is read to its end — which party waits on what at the -hang — and `netstack_holders_exported_to_children` passes 20 of 20 under the -load it hung at. diff --git a/issues/two-holders-racing-on-an-exported-socket-can-both-stall.md b/issues/two-holders-racing-on-an-exported-socket-can-both-stall.md index 1dc15e0ff62..872c75eb586 100644 --- a/issues/two-holders-racing-on-an-exported-socket-can-both-stall.md +++ b/issues/two-holders-racing-on-an-exported-socket-can-both-stall.md @@ -20,6 +20,11 @@ change touches no socket or netstack code. Each stalled run's last guest lines are the racer's spawn and netstack's mDNS line. Logs: PR #1060's evidence, `qemu/netstack-*`. +PR #1062 saw the same stall, the same last lines, at load averages of 20 to +28: two of three runs at its branch (`ebac17a93`), whose change touches no +socket or netstack code, and one of one at `2a5b446b0` (main). Logs: PR +#1062's evidence, `guest/socket-export-*`. + What blocks is not established: no guest line says whether either racer sent its `racing` datagram, which the harness waits for before it connects. From ff2af0a4dd97854768fb3fb79697ce512d91b718 Mon Sep 17 00:00:00 2001 From: japabu Date: Sat, 24 Oct 2026 12:37:13 +0200 Subject: [PATCH 6/7] review r2 of #1062: a kernel #PF and an aarch64 kernel exception give the UART back first Two fatal entries wrote their reports before the log came back to the UART: x86's page fault from Ring 0, whose SMAP backtrace and `#PF` line precede `fatal_exception`, and aarch64's `exception()`, whose registers, page walk and backtrace precede its `panic!`. Both now take it back before their first record; a Ring 3 fault, which ends only its process, does not. A carry that lands after its console's last handle has gone, and so after the release that handle ran, is given back at once. The iommu tests' header and the harness shape's `virtio` field say what they check now, and the check for a kernel virtio driver that no longer exists is gone. Co-Authored-By: Claude Opus 5.5 --- kernel/src/arch/aarch64/trap.rs | 1 + kernel/src/arch/x86_64/idt/exceptions.rs | 4 ++++ kernel/src/object/device.rs | 11 ++++++++++- tests/common/iommu.rs | 14 ++++---------- tests/common/qemu.rs | 4 ++-- 5 files changed, 21 insertions(+), 13 deletions(-) diff --git a/kernel/src/arch/aarch64/trap.rs b/kernel/src/arch/aarch64/trap.rs index 594e38bc82a..1aa14ff2599 100644 --- a/kernel/src/arch/aarch64/trap.rs +++ b/kernel/src/arch/aarch64/trap.rs @@ -137,6 +137,7 @@ extern "C" fn dispatch(frame: &mut Frame, entry: u64) { /// EL1, or an FIQ or SError from anywhere. The panic handler's branch puts it /// on the console and the panel. fn exception(frame: &Frame, entry: u64) -> ! { + crate::log::console::on_the_uart_for_good(); let entry = Entry(entry); log!("KERNEL PANIC: {entry}: {} (ESR={:#010x})", class_name(frame.esr), frame.esr); log!(" elr={:#018x} far={:#018x} spsr={:#010x}", frame.elr, frame.far, frame.spsr); diff --git a/kernel/src/arch/x86_64/idt/exceptions.rs b/kernel/src/arch/x86_64/idt/exceptions.rs index 7de3b76138c..684a6334e5d 100644 --- a/kernel/src/arch/x86_64/idt/exceptions.rs +++ b/kernel/src/arch/x86_64/idt/exceptions.rs @@ -470,6 +470,10 @@ pub(super) fn page_fault_handler(frame: &TrapFrame) { } let fault_addr = cpu::read_cr2(); + // A Ring 0 fault halts below: before its first record. + if !Ring::of_cs(frame.cs).is_user() { + crate::log::console::on_the_uart_for_good(); + } if frame.error_code & PF_PRESENT != 0 && !Ring::of_cs(frame.cs).is_user() && mm::is_kernel_addr(fault_addr) diff --git a/kernel/src/object/device.rs b/kernel/src/object/device.rs index 98fae6fd483..9753c0c4c9d 100644 --- a/kernel/src/object/device.rs +++ b/kernel/src/object/device.rs @@ -177,7 +177,16 @@ impl ConsoleObject { /// not (`log::console::carried_by`); its last handle lets go of it. pub fn carry(&self, elsewhere: bool) -> Result<(), kernel::carrier::Refused> { match elsewhere { - true => crate::log::console::carried_by(self.holder()), + true => { + crate::log::console::carried_by(self.holder())?; + // A last handle closed beside this call ran its release + // before the carry landed: the carry is given back here. + core::sync::atomic::fence(Ordering::SeqCst); + if self.core.retired() { + crate::log::console::released_by(self.holder()); + } + Ok(()) + } false => { crate::log::console::released_by(self.holder()); Ok(()) diff --git a/tests/common/iommu.rs b/tests/common/iommu.rs index 302358d7935..37803a3b812 100644 --- a/tests/common/iommu.rs +++ b/tests/common/iommu.rs @@ -1,14 +1,13 @@ -//! `iommu_virtio_platform`: whether each virtio function the kernel drives -//! itself negotiated `VIRTIO_F_ACCESS_PLATFORM` behind QEMU's emulated VT-d -//! unit and none without one, and that a process's claim on one is handed -//! over behind the unit and refused where no unit is. +//! `iommu_virtio_platform`: whether every virtio function QEMU creates +//! carries `iommu_platform=on` exactly where a unit is, and a process's claim +//! on one is handed over behind the unit and refused where no unit is. //! `amdvi_firmware_left`: whether an AMD-Vi unit firmware left blocking is //! switched off before the boot stick is driven. use std::collections::{BTreeMap, BTreeSet}; use std::path::Path; -use super::qemu::{self, is_kernel_line, BootOptions, Profile, QemuInstance}; +use super::qemu::{self, BootOptions, Profile, QemuInstance}; use super::serial::Serial; /// The function `tests/images/netcase.toml`'s netstack claims, as its `devices` row spells it. @@ -100,11 +99,6 @@ pub fn iommu_virtio_platform() -> Result<(), String> { no_unit_is_no_claim(&log)?; } - // The kernel drives no virtio function: every one is a process's, whose - // negotiation is `toyos-virtio`'s and host-tested there. - if let Some(line) = log.text().lines().find(|l| is_kernel_line(l) && l.contains("VirtIO: PCI ")) { - return Err(format!("{name}: the kernel negotiated a virtio function's features: {line:?}")); - } eprintln!( " [iommu] {name}: {} virtio function(s), behind a unit = {behind_unit}; {}", created.len(), diff --git a/tests/common/qemu.rs b/tests/common/qemu.rs index eec53decc5a..df483274177 100644 --- a/tests/common/qemu.rs +++ b/tests/common/qemu.rs @@ -943,8 +943,8 @@ struct Shape { /// *size* is a shape dimension exactly as a disk's is, and the tests that /// read pixels were all blind to the remainder until one profile had one. panel: Option<(u32, u32)>, - /// virtio-sound, the console on virtio-serial, and the clipboard's - /// channel on a virtio-serial function of its own. + /// virtio-sound, and the clipboard's channel on a virtio-serial function + /// of its own. virtio: Virtio, /// The console's port, on its own virtio-serial function, which /// `/system/bin/virtio-console` carries the log on. A machine whose every From cdbfc36490853aab59faccc5b073cb38d7127947 Mon Sep 17 00:00:00 2001 From: japabu Date: Sat, 24 Oct 2026 13:14:38 +0200 Subject: [PATCH 7/7] issues: the T14 i219 row met a quiet wire again, at #1062's head The row's only red reading was the receive under traffic, with the MAC counting no frame seen, as the record's 2026-10-21 boots did. Co-Authored-By: Claude Opus 5.5 --- ...19-msi-passes-only-when-the-network-sends-it-a-frame.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/issues/i219-msi-passes-only-when-the-network-sends-it-a-frame.md b/issues/i219-msi-passes-only-when-the-network-sends-it-a-frame.md index aa00f754ff7..19f75d09e46 100644 --- a/issues/i219-msi-passes-only-when-the-network-sends-it-a-frame.md +++ b/issues/i219-msi-passes-only-when-the-network-sends-it-a-frame.md @@ -20,6 +20,13 @@ frame(s), took 0 good, missed 0, 0 with CRC errors", while 65 frames were sent and written back and the link came up after 2.7 s: https://github.com/ToyOSOrg/evidence/tree/e240b6db1696f7e5c805f1146578d0451e88673a/t14-i219 +It failed alike on 2026-10-24 in the T14 `testcases` boot of PR #1062 at +`ff2af0a4d`, whose change touches no NIC, interrupt or network code: every +interrupt reading held, the link came up after 2.8 s, 65 frames were sent and +written back, and the MAC saw 0. #1058's boot at `4ce692580` had passed it +earlier the same day: +https://github.com/ToyOSOrg/evidence/tree/617bbbe20535d92ccbc262a8a0f88a8d7df3e6c3/metal-r2 + Exit: the job makes the traffic it receives, by MAC or PHY loopback or a frame addressed to itself, so its receive claim no longer depends on the network sending anything. Its verdict is then the same on a quiet wire and a