[Japabu]

Small-kernel track, stages K4 and K6, by the owner's ruling that the kernel creates no thread but each CPU's idle loop; and, in the same thread-lifecycle area, a yield that yields and detached threads that free themselves.

Problem

klogd (kernel/src/log/console.rs) was the last kernel thread: the console UART's one writer between the boot and the stop, draining the record ring and the lines logkeeper queued on the kernel console. Every kernel-thread provision existed for it: sched/kthread.rs and its row table, is_kernel_task/current_is_kernel_thread, quiesce's exemption, the scheduler's zero baseline, a kernel-space task and its reserved PCID/ASID. std's yield_now was a spin hint, and a detached thread's stack and table place waited for the process's next spawn.

Now

The kernel writes the UART only where no process can: inline as each record commits until log::console::hand_over (before userland can claim the UART), from the machine's stop once userland is stopped, and on every fatal path. Between them the UART is /system/bin/uart's.

The UART is a claim like any device: a new uart device class (toyos_abi::uart). On x86 it is an isa row, COM1's eight ports bound to the reading process and IRQ 4 as records; COM1 leaves pio::FIXED and is that row's (the ACPI mediation now answers ClaimedPort for it). On AArch64 the row is the SPCR's PL011: its register frame installed as an uncached aperture with the description, and its interrupt a level SPI routed to the row (irqchip::route_spi/set_spi_masked, arch::aarch64::pio::Wire), masked by the handler until the holder acknowledges.

One carrier serving, two channels (system/logkeeper/src/carrier.rs). The UART's driver asks on log-wire exactly as the virtio console's asks on log-carrier: admitted alone, fed from where its channel starts, says DELIVERED. The port's Carriage decides the UART's quiet as before; the UART's feed writes CARRIED_ELSEWHERE/BACK_ON_THE_UART on itself (the harness's merge keys on them, now toyos_logstream's) and skips what the port carries. Lines said while the machine stops reach the UART's carrier, held back from the file as before.

What the console has spoken is a read position per log shard, the kernel's drain position (log::console). The inline drain moves it. SYS_LOG_SPOKEN (replacing SYS_CONSOLE_CARRIED) moves it to at least the positions logkeeper's own read reached through the rounds a channel delivered whole, taking a cursor as SYS_LOG_READ does. It needs the new Rights::CONSOLE (the console syscap), which the build lets only logkeeper hold, because what it moves is left off every fatal path's output. logkeeper holds a round's positions back to each shard's first record still waiting for a later round, so a round speaks only for what it wrote, and a channel handed a gap in the replay speaks for nothing after it. The stop and every fatal path write every record past the mark: they may repeat what a channel delivered and had not said, and miss none of the kernel's records, whatever their stamps. The UART's first carrier is handed every line before the hand-over except the kernel records the kernel put on the UART itself (decided by position), so the boot is on the UART once. A carrier started again resumes where its channel left off, or where it passed over while the port carried the log.

The hand-over takes the boot drains' claim on the wire for good: a drain still walking on another CPU finishes first, and none starts after, so there is one writer by construction (bounded by a 60 s tripwire).

The supervisor's end is the machine's. The kernel ends the machine on the supervisor's end through its panic path, naming how it ended (the supervisor ended, code=101, and nothing starts it again). Before that, the supervisor's panic hook says the panic's message as its last words through SYS_LOG_LAST_WORDS. The call writes one alert record, accepts only the boot's first call (AlreadyExists after), needs the new Rights::LAST_WORDS, which the kernel mints on the supervisor's system capability and no manifest syscap name maps to, and takes one line of at most 512 bytes of UTF-8 with no control character, checked at the boundary (toyos_abi::log::last_words; the SDK fits text to it first). The fatal path puts both on the UART, the panel and the black box. A supervisor that fails before logkeeper runs therefore says why again, and a machine whose supervisor dies no longer idles.

The kernel console object is gone, with its line queue, ConsoleLine, FileType::Serial and the 10 ms console repoll. The supervisor makes the console's input a pipe: its read end is every program's stdin (test-runner's commands arrive through it), and its write end is endowed to the UART's driver (toyos_logstream::CONSOLE_INPUT), which writes what the UART receives into it and refuses to start without it.

The log's origins and the console's input are grants, not names: a [programs] row says holds = ["log-origins"] or holds = ["console-input"] (the manifest's hold line), and the supervisor endows the row that holds each. The build refuses a row that holds either but its one holder (HOLDINGS, in held_by_their_holders_alone, on every rendered manifest), and every boot config that runs logkeeper must give it the origins and its UART driver the input.

The log's readiness without a thread: emit owes it (log::user::owe_readiness, one release store) and the next syscall's end or scheduler pass posts it.

K6: sched/kthread.rs, kthread::spawn, the row table, is_kernel_task, current_is_kernel_thread, kernel_start/kernel_thread_returned, Start::Kernel, the user-less kernel stack, Runs::Kernel, mm::space::kernel with PcidHandle::Kernel/Asid::Kernel, ElfInfo::none, quiesce's two exemptions and the scheduler's blocking_baseline are deleted; a task's image is no longer an Option. git grep -n 'kthread::spawn(' -- kernel/src finds nothing, and no kernel code creates a schedulable task but the idle loop.

Thread lifecycle: SYS_YIELD gives the CPU to another runnable thread of the band and is std's yield_now; osbench's time.yield now times it. SYS_THREAD_EXIT's detached form (thread_exit_detached) releases the thread's stack once it no longer runs on it, and proclife::leave_detached forgets the thread's place as it leaves (a teardown in progress keeps it). std's and libc's REAPED lists are gone.

Deleted with klogd: log/handoff.rs, kernel::carrier, the log waiter, the sleep lock (its one user), the three wire-holding actuators and the five tests that staged them (machine_shutdown_wire_held/_kept/_at_the_seal, virt_reboot_wire_held/_kept), the loom models console_handoff, log_wake and sleep_lock with their negative controls, and the metal row klogd_hosted, replaced by console_uart_driven.

Decisions to review

• The kernel shares the UART with its driver, by time: before the hand-over, and after the stop or a fatal path. A holder the stop did not stop within its budget may write over the stop's lines, which the stop's own record names. A fatal path and the stop begin on a line of their own that says the kernel takes the UART, since the driver can be stopped mid-line or behind the log.
• The spoken mark is exact by position, and its granularity is a round: the stop and a fatal path may repeat the kernel records of rounds a channel had not delivered whole, never miss one.
• A kernel panic on the supervisor's end (by the owner's decision through the orchestrator): nothing restarts the supervisor, so the machine is over either way, and the fatal path is the one channel that reaches the UART, the panel and the black box.
• AArch64 routes every SPI to the boot CPU (recorded at irqchip::init), as x86 routes every device line to one destination.
• The console's input is a pipe the supervisor owns, endowed to the row that holds it, as the log's origins now are; bytes received with no reader's room are dropped and said once per spell.
• AArch64 gets the PL011 claim now, because the virt job tests read their job output off the PL011.

Checks

Head 152cae5d1 on main 783569506. Every log, patch and command, masked, with its revision, host and exit: round 1, round 3, rounds 4 and 5 round 6 and round 7 (README in each).

• Round 7, at 152cae5d1: main's start plan (Supervisor times PCI claims: the declaration in the device set, and storage and homes before the rows that need them #1164) had put the UART's driver after the declaration, storage and the homes, leaving a UART-only machine silent from userland's start until storage was up. The plan now starts first the rows that carry the log, by what they hold (the log's origins and the console's input), beside the declaration's holder. Pinned by the_uart_starts_with_the_log_before_the_declaration, which the plan's old rule reds (exit 101). --ci host exit 0, 108 green. On the x86 guest host, 6 rows passed: the new supervisor row, both UART rows, machine_shutdown_short_stop, console_image_boots and session_lifecycle. The UART's claim is now pid 4's, beside logkeeper. Where nothing can declare, both still start with every row in main's storage order.

• Round 6, review r2's fixes rebased onto main 783569506, at d4c43fd35: --ci host exit 0, 108 green; --build-only x86 and AArch64 exit 0; a_supervisor_that_fails_early_says_why_on_the_uart on the x86 guest host passed, with last words now gated by Rights::LAST_WORDS. The rebase resolved tests/images/acpicase.toml (main's storage rows beside this change's UART row) and spawn_thread (main's near placement beside the image expect); the r5 rows and metal at 8ac4f2187 cover what r6 leaves untouched.

• Host, at 8ac4f2187: cargo run -- --ci host exit 0, 108 steps green; --build-only x86 and AArch64 exit 0. New host tests: the ABI's last-words rule and fitting; logkeeper's a_round_is_through_the_cursor_but_what_waits, what_the_uart_carried_is_told_by_position, a_uart_carrier_started_again_resumes_where_its_channel_left_off and the hub's positions; the build's console syscap gate; the harness filter's the_kernels_taking_the_uart_is_shown_from.

• x86 guests on the x86 guest host (KVM), at 8ac4f2187: 10 passed: a_supervisor_that_fails_early_says_why_on_the_uart (new), the_uart_carries_the_boot_once_across_the_hand_over, the_uart_carries_the_log_again_once_the_console_carrier_goes, machine_shutdown_short_stop, std_threads_count_the_cpus_and_give_back_ended_threads, session_lifecycle, console_image_boots, acpi_lock_given_back_on_one_cpu, screen_fatal_halt_composited, nested_nmi_is_loud.

• That host found a defect at 8678b19c7: machine_shutdown_short_stop stalled there twice and passed on main. The UART capture held the stop's lines and Shutting down., but they came after the UART driver's boot backlog and before the driver ever said the port carried the log. The harness shows a virtio machine's UART only from that line, so it never saw them. 8ac4f2187 has the kernel start its stop and fatal lines with console: the kernel writes the UART from here (toyos_abi::log::KERNEL_TAKES_THE_UART), and the harness filter shows the UART from it on. The failing runs are the control for that commit.

• AArch64 guests: virt_jobs_at_el2 and virt_fatal_halts_the_others_first passed at 8678b19c7, on this Mac while it was loaded and before the guest-host rule took effect. They have not run at 8ac4f2187. That commit changes only the shared console's takeover line, which these rows exercise. They will run on a quiet Mac, and that run is pending.

• Controls, each red as it should be:

  • the supervisor's panic saying no last words (the new row, exit 1);
  • the kernel letting a supervisor that exited with a code end nothing (the new row, exit 1: the boot never ends);
  • a round speaking through records still waiting (exit 101);
  • a restarted UART carrier beginning at the hand-over (exit 101).

• Metal, the readbacks of 08e4bb8f0 judged. Both benches' testcases-watchdog boots are green: loader_watchdog_arms and watchdog_fed, the log whole to its stop. On both testcases boots std_threads exited 0. console_uart_driven held its own claim on both (uart: no console UART claim: neither bench has a COM1, so the 16550 and IRQ 4 paths reach no hardware we have). It is red only on must_be_clean finding another row's panic: on the AMD bench, counters_metal's wait for an EC query that machine never raises (metal: the ACPI rows wait from their own start, and counters admits the machine's own events #1163 fixes it); on the T14, usbd_spare's (issues/a-metal-row-names-no-machine-so-another-benchs-boot-carries-it.md). The other reds are the other bench's rows riding the boot, i219_msi's own record and, on the T14, rows needing its DATA, which an unrelated incident damaged.

• Metal staged at 8ac4f2187, not run: boot:testcases per bench, which brings testcases-watchdog.

• Earlier rounds' rows (rounds 1 and 3) stand for what later commits did not touch. The detached-thread, yield, stop and fatal-path controls of round 1 are unchanged in subject.

Kernel size, kernel/src: 233 files, 60,378 lines on the original base → 230 files, 58,979 lines at 8ac4f2187. All of kernel/ (pure, loom, proclife, sim): 380 files, 112,027 lines → 373 files, 110,056 lines.

Risks and gaps

• kernel: user copies go through the process's own mapping, and none runs under a lock (S0-a's lock half) #1150 conflicts: it edits ConsoleObject and the console read path, both deleted here, and used kernel/src/sleeplock.rs, which this deletes as dead. By the landing order (Consolidate storage around native bcachefs and userland filesystems #1152, this, then kernel: user copies go through the process's own mapping, and none runs under a lock (S0-a's lock half) #1150), kernel: user copies go through the process's own mapping, and none runs under a lock (S0-a's lock half) #1150 adapts.
• Lines a carrier had not delivered at its end, and program lines the UART had not sent at the stop, are on neither channel (issues/the-console-loses-what-its-carrier-had-not-delivered.md). At the stop the kernel now says so on the UART.
• A refused stop's held lines reach the UART a second time.
• A panic on one of the supervisor's sync-* threads ends only that thread: its last words are said, a stray alert record on a machine that carries on, and a later end of the supervisor finds the slot spent. Its other threads' panics become a panic of its main thread, whose reason is the first panic's.
• time.yield against Linux on both benches is owed: issues/toyos-yield-is-not-yet-measured-against-linux.md.
• No bench has a COM1 or an AArch64 PL011, so the UART driver's hardware paths are QEMU's alone.

🤖 Generated with Claude Code

[Japabu]

Review r1 of #1165 at 08e4bb8f0 (K4, K6)

Architecture

The ownership fits the ruling. The kernel writes the UART only where no process can: before the hand-over, from the stop, and on fatal paths. Between those points the UART is an ordinary device claim held by a ToyOS-only userland driver, which logkeeper feeds through the same carrier serving the virtio console uses. A per-holder kernel console with a line queue, a sleep lock and a stop handshake is replaced by one shared mark, SYS_LOG_SPOKEN. Kernel threads, the kernel address space object, its PCID/ASID and every kernel-thread special case are gone. kernel/src shrinks by about 1,450 lines; the PR as a whole is +2,585/-3,731.

Each kernel addition has a job userland cannot do:

• the spoken mark, which keeps fatal-path output from repeating;
• the uart class and its row;
• AArch64 SPI routing;
• SYS_YIELD;
• the detached SYS_THREAD_EXIT.

Granting by holds instead of matching on the program's name is the right authority model.

The weak point is the spoken mark's representation. It is a stamp compared across shards. The fatal path compares it against records the reader had not yet seen, so in a narrow window the mark claims more than was delivered (finding 1).

What I checked and found sound

• K6 residue. git grep finds no kthread, is_kernel_task, kernel_start, Runs::Kernel, space::kernel or ElfInfo::none. The idle loop runs on kernel_root() (PCID/ASID 0), and tlb.rs's "tag 0 is no space" assertion is now exactly true. The AP path (ap_entry then ap_idle) never used the kernel space object. Parkable and yield_now assert the flat trap baseline.
• Detached exit. The stack is released through vm::release before leave_as. leave_detached keeps the zombie while a teardown is claimed, so the teardown collects it. A join racing the exit wakes on the thread watch post, finds no thread and answers NotFound, which std_threads checks. Tids are never reused (IdMap), so forgetting a still-running thread's place cannot alias a new thread. The std and libc state CAS (JOINABLE/DETACHED/EXITED) is unchanged and correct.
• uart claim authority. On x86 the row is COM1's eight ports. isa::claim refuses selector claims on rows with empty irqs, so only the class claims this row. Ports bind on first read to the reading process and stay with it until its teardown. The ACPI mediation now answers ClaimedPort for them. On AArch64 the PL011 frame is a 4 KiB Device-memory aperture without DUP or TRANSFER. bind also blocks a re-claim until the holder process ends. log-wire is in HELD_ALONE.
• Stdin. It is one supervisor-made pipe. The supervisor keeps the write end, so readers never see EOF, as with the old console, and a restarted driver is handed it again. Bytes are dropped when the pipe is full, and this is said once per spell. The harness path is covered by every guest row that uses test-runner.
• Deleted loom models. Each model's subject is deleted (handoff, waiter flag, sleep lock). The new OWED flag needs no model and no fence. Commit, then store(Release), then swap(Acquire), then the watch post under its lock, then the reader's arm and read: that sequence is a happens-before chain. A stale relaxed load only defers the post to the next pass or syscall end on the producer's CPU, and logkeeper's cadence bounds that.
• Inline drain in interrupt context. The phase and its cost are unchanged from main: until hand-over, at the same point klogd used to start. After the hand-over, no interrupt-context emit touches the UART.

Required

1. The spoken stamp can hide a committed kernel record from a fatal path, contrary to the stated contract.

   • Where: kernel/src/log/console.rs:9-14 and :95-98, system/logkeeper/src/carrier.rs:18-21, issues/the-console-loses-what-its-carrier-had-not-delivered.md:20 ("so no kernel record is lost this way"), and the PR body ("never lose one").
   • Mechanism: emit stamps inside its IRQ-off bracket and commits later. drain_ordered (used by SYS_LOG_READ) skips a ring whose next slot is reserved but not yet committed. So logkeeper can read and deliver a round whose newest stamp S was taken on CPU B while CPU A's record, stamped T < S, is still in flight. logkeeper speaks S. Every fatal path and the stop then drop A's record (Unspoken::put, at_ns > spoken) until a later round delivers it.
   • Consequence: the record most likely to be in flight is the one just before a crash on the same CPU, which is exactly what fatal output exists to show. Cross-CPU clock skew widens the same window. The window is short on metal and unbounded under a descheduled vCPU.
   • Fix: make the mark exact, or correct every claim and record the window with its exit in the issue. Two exact options:
     • speak a per-shard cursor (the read positions of the last delivered round) rather than a stamp, so the fatal path drains from it by position, as the inline phase does;
     • have speak clamp to the oldest stamp a CPU has taken and not yet committed (a per-CPU word set in emit before the stamp and cleared at commit).

2. A restarted UART driver is fed the whole replay again from the hand-over (system/logkeeper/src/carrier.rs:507: let mut at = if wire { self.hub.wire_from() } ...).

   • Scope: every image runs uart with restart = true (images/system.toml:83, images/console.toml:44, tests/images/testcases.toml:23).
   • Consequence: after any driver panic, up to REPLAY_BYTES (16 MiB) of lines already shown are replayed. That is roughly 24 minutes at 115200 baud, during which no new line reaches the UART, and every stop meanwhile waits out STOP_WAIT for the wire. Nothing documents or tests this.
   • Fix: a re-admitted wire carrier should resume where the previous one's channel stopped, or at the current end with one line saying what it skipped. Keep that offset in the carriers' state across admissions, and cover it with a host test on Carriers/Hub.

3. AArch64: the comments say the UART SPI goes to the boot CPU, but it goes to whichever CPU runs the claim.

   • Where: kernel/src/arch/aarch64/pio.rs:2 and :25.
   • Mechanism: irqchip::route_spi (irqchip.rs:343) writes GICD_IROUTER with cpu::hardware_id(). isa::claim_row calls it from the claimer's syscall, on an arbitrary CPU.
   • Consequence: the comments are false. The UART's interrupt target also depends on scheduling, while x86 sends every device line to one fixed MSG_DEST.
   • Fix: route to a fixed device-interrupt target (for example the boot CPU's affinity, as route_iommu_events gets at boot), or correct both comments.

4. kernel/src/process.rs:693: let image = proc.image.clone()?; makes an internal invariant a silent spawn failure.

   • Invariant: the caller is a live thread of the process and has passed admit_thread_insert. image becomes None only in teardown_bookkeeping, which the last thread out runs, so None here is a kernel bug.
   • Consequence: the path returns None after building ThreadData around the published TLS allocation. The code's own comment says the table owns that allocation's release past admission, so the TLS mapping leaks without a word.
   • Fix: use expect.

5. kernel/proclife/src/table.rs:43: "Only a join does this" is now false. teardown::leave_detached also calls forget_thread. Fix the contract line.

6. issues/a-threads-kernel-stack-has-no-guard-page.md:25 cites the deleted issues/the-kernel-still-creates-threads.md. Its hold ("Held until ... K6") is now lifted. Update the record in this merge (issues/README.md: search for the slug before deleting).

7. The record for the supervisor's pre-logkeeper failure says the shape is undecided. issues/a-supervisor-that-fails-before-logkeeper-runs-says-why-nowhere.md ends with "the shape is the owner's to rule (#1165)", but per the brief the follow-up is already decided: the supervisor's reason goes out through the kernel fatal path. Before landing, either land that follow-up on this branch, or have the issue record the decided exit and its owner. Until then main loses a working report (bootstrap_fails used to reach the UART). That loss is acceptable mid-migration only if it is recorded accurately.

8. Metal evidence is outstanding. No 16550 driver, IRQ 4 path or stop path has run on hardware. Landing waits for the staged boot:testcases and testcases-watchdog runs on the T14 and the AMD bench: console_uart_driven, std_threads, and watchdog_fed, which judges the log whole to the stop on the path this change rewrites. QEMU's 16550 has no baud limit, so the drain-speed behaviour in finding 2 and in suggestion 1 cannot appear there.

The host, build and QEMU evidence I checked (r3 README, commands, exits and controls) matches the PR body. Reusing the r1 rows after the r3 commit is justified: r3 touches no kernel code, and its rows rebuilt and booted the changed userland.

Optional

1. hand_over does not wait for an inline drain already running on another CPU, and that drain does not stop at HANDED (console.rs:57-74). If an AP holds WIRE, for example a late AP past the roster's echo wait, hand_over returns at once. The AP's drain_ordered then keeps writing every new commit after the hand-over, and moves SPOKEN past what logkeeper read at its start. The window is very unlikely, but the module's "one writer at a time" is not established by the code. Fix: spin for WIRE in hand_over, and stop putting records once HANDED.
2. SYS_LOG_SPOKEN is gated by Rights::LOG alone. Any log reader can therefore mute fatal-path UART output up to "now"; test-runner holds logread today. Consider tying it to a narrower grant than reading.
3. console_uart::init/fill_row ignore the SPCR's Interrupt Type field. They take any GSIV ≥ 32 and hard-code level triggering. Check the GIC bit and take the trigger mode from the table.
4. x86: when fill_uart_row declines (no I/O APIC carries IRQ 4), COM1 is neither FIXED nor a row. AML then sees it as Standing::Free, while the kernel still writes it at the stop and on fatal paths. Keep it Kept when no row is filled.
5. console.rs:119-120: "the holder's last line ended" reads as if the line had already ended. It means the stop writes a newline because that line may not have ended.

SEND BACK

[Japabu]

Response to review r1 (at 8678b19c7 and 8ac4f2187)

Required:

1. The spoken mark is exact. It is now the kernel's drain position, one per shard, not a stamp. SYS_LOG_SPOKEN takes a cursor (read_on_cursor, refused past a shard's head) and moves each position up with fetch_max. The stop and the fatal paths drain every record past it whatever its stamp, so the Unspoken stamp filter is gone. logkeeper speaks the cursor of the rounds a channel took whole. Each shard's position is held back to that shard's first record still waiting for a later round (written_through), and a channel handed a replay gap speaks for nothing after it. The UART carrier's start is decided by position too: it is owed every line before the hand-over except the records the kernel itself put on the UART. Host tests cover both rules, and a control (a round speaking through waiting records) reds.
2. A restarted UART carrier resumes where its channel left off, or where it passed over the log while the port carried it (Feed::resumed, kept across admissions; Hub::wire_start). It does not restart from the hand-over. Covered by a_uart_carrier_started_again_resumes_where_its_channel_left_off, and its control reds.
3. Every SPI is routed to the boot CPU, whose affinity irqchip::init records, so the comments are now true.
4. proc.image.clone()? is now expect.
5. The forget_thread contract now names the detached leave.
6. The guard-page record's hold is lifted.
7. Option A has landed. The kernel ends the machine through the panic path on the supervisor's end, and the supervisor's panic hook says its reason first through SYS_LOG_LAST_WORDS. That call writes one alert record, accepts the boot's first call only, needs Rights::LOG, and takes 1 to 512 bytes of UTF-8 with no control character, checked at the boundary. The record is deleted. The new QEMU row a_supervisor_that_fails_early_says_why_on_the_uart boots a ROOT without its release file (Plan::root_omits). Two controls, the hook unsaid and the exit ignored, red it.
8. Metal. Both benches' readbacks of 08e4bb8f0 were judged; the PR body has the detail. In short, the watchdog boots are green and std_threads passed on both. console_uart_driven held its own claim on both, but neither bench has a COM1, so the 16550 and IRQ 4 paths reach no hardware we have. It is red only on another row's recorded panic. The final head 8ac4f2187 is staged.

Optional:

1. Taken. The hand-over takes the boot drains' wire claim and never gives it back; a drain on another CPU finishes first (60 s tripwire).
2. Taken. SYS_LOG_SPOKEN needs the new Rights::CONSOLE (the console syscap), which the build gives only logkeeper.
3. Taken in part. The SPCR's Interrupt Type must carry the GIC bit, or the UART gets no row. The SPCR carries no trigger mode, so level stays, from the PL011 TRM.
4. Fixed. COM1 is declared Kept when no row is filled.
5. Fixed. The comment is rewritten.

New since r1: under KVM on the x86 guest host, machine_shutdown_short_stop stalled at 8678b19c7. The UART driver was still sending its boot backlog when the stop took the UART, and it never said that the port carried the log. The harness shows a virtio machine's UART only from that line, so the stop's lines went unseen. 8ac4f2187 makes the kernel start its own lines with KERNEL_TAKES_THE_UART, and the harness filter shows the UART from it on. The failing runs are its control; the PR body has the evidence.

[Japabu]

Review r2 of #1165 at 8ac4f2187 (K4, K6)

This review covers the fixes since r1 (0638624c0..8ac4f2187, +651/−297 in 59 files; 25 of those files are one-line image syscap additions) and the new kernel interfaces that came with them.

Architecture

The spoken mark is now kernel positions, one per shard, read by logkeeper and validated by read_on_cursor/Cursor::from_reader. This is the right representation. It removes the stamp filter (Unspoken) and the SPOKEN word instead of adding a correction on top of them. Rights::CONSOLE separates speaking from reading, the kernel enforces it, and the build checks who holds it (held_by_their_holders_alone on every rendered manifest), the same way HELD_ALONE is handled. Option A is the owner's decision; the code implements it in one place (process::teardown, keyed on the tree's only parentless node, and tree.rs says nothing reparents). The takeover line is a reasonable kernel-owned marker. Its text is a false claim, though (finding 2).

The one interface that needs another look is SYS_LOG_LAST_WORDS. A kernel call is justified: the supervisor's ring has no reader before logkeeper runs, and the fatal path is the only channel that reaches the UART, the panel and the black box. But it is gated by the wrong right (finding 1).

r1's required items

1. Exact mark: fixed. drain_ordered stops a ring at its first reserved-but-uncommitted slot, so logkeeper's cursor never passes a record still in flight. written_through holds a shard back to its first record still waiting. Fed::Gap stops a carrier speaking after an eviction. Published::advance is fetch_max per ring. The stop and fatal drains are plain position drains. I traced the r1 race (stamp T < S still in flight on CPU A) and it can no longer hide the record. Host tests and the control-waiting-ignored control back this.
2. Restarted UART driver: fixed. Feed::resumed is advanced from delivered rounds and while quiet. admit does not reset it (at, written and gap are reset). Hub::wire_start resumes from it, or re-owes the pre-hand-over lines to a channel that took less. Test and control are in r4.
3. AArch64 routing: fixed. BOOT_CPU is stored in irqchip::init on the boot CPU, and route_spi writes it into GICD_IROUTER. The comments in pio.rs are now true. The SPCR GIC bit is at offset 52, bit 3, which matches the SPCR layout.
4. Missing-image panic: fixed (process.rs:693, expect with the invariant).
   5–7. Comments and records: fixed. The forget_thread contract names the detached leave. The guard-page record's hold is lifted. The pre-logkeeper issue is deleted because option A landed. No references remain to the deleted slugs or to SYS_CONSOLE_CARRIED, stamp_through, wire_from(), log::console::spoken or Unspoken.
5. Metal: satisfied at 8ac4f2187. I read the readbacks under metal-r5/runs/{t14,amd}: each boot's revision is 8ac4f218…, and all four verdict.txt (testcases and testcases-watchdog on both benches) say passed. console_uart_driven reports uart: no console UART claim — nothing to drive on both. The 16550 and IRQ 4 paths therefore remain QEMU-only, as the PR body says.

r1's optional items were all taken (COM1 Kept, Rights::CONSOLE, the hand-over's wire claim, the SPCR bit, the stop comment).

New interfaces

• SYS_LOG_SPOKEN / Rights::CONSOLE. The surface is minimal. The cursor is bounded by the shard count and each position by the ring's head. Rights can only be narrowed: the supervisor's root capability carries CONSOLE, and only logkeeper may ask for it. A holder could mark a reserved-but-uncommitted slot as spoken. That is within what the right grants, and the only holder is trusted.
• SYS_LOG_LAST_WORDS. Its inputs are validated at the boundary: length is checked before the copy, last_words is applied to the copy, and SAID is set only after validation, so a refused call does not use up the slot. The authority is wrong (finding 1).
• Takeover line. It is needed because the harness cannot otherwise tell the driver's backlog from the kernel's own lines, and it also marks the break for a person reading the UART. That is not machinery that exists only for tests. Its text is wrong (finding 2).
• Option A. The panic is the last thing teardown does, after publish_climb, with no lock held. The stop parks threads and does not tear them down, so a shutdown or reboot cannot trigger it. Nothing else to raise.

Required

1. kernel/src/syscall/machine.rs:47 (and sdk/abi/src/syscall.rs:2418): last words are gated by Rights::LOG, a read right, and the slot is global and single-use.
   • Who holds LOG: logkeeper and test-runner (logread). Rights::LOG's contract (handle.rs:170) is "read the whole machine's kernel log".
   • What a holder can do: with this change, any LOG holder can write an alert record of its own text into the kernel ring, which the fatal path shows on the UART, the panel and the black box. It can also use up the boot's one slot, after which the supervisor's real reason gets AlreadyExists, which is the failure this call exists to report.
   • Why it is wrong: this is the same conflation r1's optional 2 found for SYS_LOG_SPOKEN, now in a new interface. It silently widens what a reader's right allows.
   • Fix: gate the call on its own right, which no SYSCAP_RIGHTS name maps to, so only the supervisor's root capability carries it. Alternatively, have the build hold it to the supervisor, as it does console. Update LOG's holders and the new docs to match.
2. sdk/abi/src/log.rs:26: the takeover line's "what its driver had not sent is in /log" is false in the cases where it is printed.
   • Stop: program lines said while the machine stops are held back from /log and owed only to the UART carrier (Log::write, Hub::append_stopping). The record the-console-loses-what-its-carrier-had-not-delivered.md says those lines are lost when the UART had not taken them.
   • Fatal paths: the line is printed whenever HANDED is set. That includes the new row's boot, where /system failed and no logkeeper or /log ever existed, and machines with no DATA.
   • Consequence: an operator is told that missing output is recoverable when it may not exist.
   • Fix: keep only the takeover statement (for example, console: the kernel writes the UART from here). The harness keys on the constant, so it is unaffected.
3. system/logkeeper/src/serve.rs:394-395: the test's contract says a refused stop's lines reach the UART "through the replay, once", but its own assertions show stopping\n delivered twice (first as Fed::Said, then again in the resumed replay). The comment is false, and the duplicate is debt that is recorded only in the PR body. Correct the comment, and record the duplicate in issues/the-console-loses-what-its-carrier-had-not-delivered.md, which owns this accounting, with an exit (for example, Hub::resume skipping the bytes the wire already took as Said).

The implementer's two risks

• A refused stop's held lines reach the UART twice. The behaviour is acceptable to land. It is a rare path, a bounded handful of lines, and /log holds them once. It needs the record and the comment fix in required item 3.
• A non-fatal supervisor-thread panic uses up the last words. The behaviour is acceptable. The hook speaks for the first panic, which is the root cause in the paths I traced: an apps/files worker panic becomes main's failed send/expect, and the bootstrap reader's becomes main's join().expect. One case is different: a sync-* thread's panic shows up only as recv_timeout disconnecting, and the stop carries on. That leaves a stray "last words" alert on a machine that keeps going. The PR body's "the supervisor treats any panic as its end" is therefore not quite true. That correction is for the orchestrator. No code change needed.

Suggestions

1. kernel/src/arch/aarch64/irqchip.rs:314: route_iommu_events still says "Take SPI intid on this CPU". It now goes to the boot CPU through route_spi. The text is true only because smmu::init runs there. Say "the boot CPU".
2. issues/a-threads-kernel-stack-has-no-guard-page.md:25: the rewritten line is unwrapped and carries history ("kernel: no kernel thread but the idle loop; the console UART is a userland driver's (K4, K6) #1165 deleted the kernel threads"). "Every kernel stack is a user thread's or a CPU's idle loop's" is enough.

Evidence

The r4/r5 README matches the PR body. The four control patches are real inversions of the claims they guard: I read control-resume-ignored and control-waiting-ignored. The settled items (host CI, the remote x86 rows, AArch64 owed on a quiet host) are not re-asked. Fixing findings 1–3 needs a compile and the host checks. Finding 1 also changes what the new QEMU row exercises, so one run of a_supervisor_that_fails_early_says_why_on_the_uart should follow. Finding 2 only changes a constant's text and needs no guest rerun.

SEND BACK

[Japabu]

Response to review r2 (at d4c43fd35, rebased onto main 783569506)

1. Fixed. Last words need their own right, Rights::LAST_WORDS. The kernel mints it only on the supervisor's system capability, and no syscap name maps to it, as with PAGER. A manifest test (no_syscap_name_is_the_last_words_right) holds that, so logread holders can no longer write the record or spend the slot.
2. Fixed. KERNEL_TAKES_THE_UART now reads console: the kernel writes the UART from here.
3. Fixed. The test comment now says a refused stop's lines reach the UART twice. issues/the-console-loses-what-its-carrier-had-not-delivered.md records the duplicate, with the exit you suggested: Hub::resume moves the UART carrier past what it already took.

Both suggestions are taken. The IOMMU event SPI's comment now names the boot CPU, and the guard-page record no longer carries the history. The PR body's risk line now describes a sync-* thread's panic correctly.

Checks: --ci host exit 0; --build-only x86 and AArch64 exit 0; a_supervisor_that_fails_early_says_why_on_the_uart passed on the x86 guest host (evidence). The two AArch64 rows are waiting for a quiet Mac.

[Japabu]

Review r3 of #1165 at d4c43fd35 (K4, K6)

This round checks r2's three required items, both suggestions and the rebase onto main 783569506. I did not reopen the settled items: r5 metal, the jan-pc run of a_supervisor_that_fails_early_says_why_on_the_uart, host CI and the owed AArch64 rows. The r2 fix commit is +44/−16 across 10 files. Nearly all of it is doc and record text, plus one new right and its manifest test.

Architecture

The r2 fixes are minimal and in the right place. Rights::LAST_WORDS is a right of its own, built the same way as PAGER. Nothing changes structurally. The rebase does have a semantic interaction that the textual conflict resolution misses: main's #1164 start plan now runs this PR's UART driver after storage (finding 1).

r2's required items

1. Rights::LAST_WORDS: fixed.
   • Kernel boundary. sys_log_last_words calls demand_syscap(syscap, Rights::LAST_WORDS) first (kernel/src/syscall/machine.rs:48). The length is checked in dispatch before the copy, and SAID is set only after last_words accepts the text.
   • Who gets it. The kernel mints it in exactly one place: the supervisor's system capability (kernel/src/loader/mod.rs:481), the only SysCap::new call site. ops::initial_rights gives a SysCap NONE.
   • Narrowing. The supervisor hands children only syscap.narrowed(syscap_rights(..)) (system/supervisor/src/main.rs:3242-3247). syscap_rights starts from TRANSFER and adds only SYSCAP_RIGHTS entries. no_syscap_name_is_the_last_words_right checks that the union of every name excludes the bit.
   • The pager path. command.pager(syscap.as_handle()) is the only place the full capability is named, and the kernel only checks Rights::PAGER there (syscall/proc.rs:92). It does not endow the handle. compositor's SYSCAP_LABEL take receives a narrowed copy.
   • Duplicating. The supervisor holds DUP/TRANSFER, so only the supervisor could pass the right on, and its code never hands out an unnarrowed copy. That is the same trust as for PAGER and CONSOLE, and LAST_WORDS is not widenable or reachable from a manifest.
   • Docs. Rights::LOG's contract (handle.rs:170) again describes reading only, which is true now.
2. Takeover line: fixed. KERNEL_TAKES_THE_UART is console: the kernel writes the UART from here. Its consumers are the constant itself (build/vmconsole.rs:110,517,521; kernel/src/log/console.rs:150), and the old text appears nowhere else.
3. Refused-stop comment and record: fixed. The comment no longer claims "once". the-console-loses-what-its-carrier-had-not-delivered.md records the duplicate, and its exit (Hub::resume moves the UART carrier past what it took) is concrete. It sits in the record that owns this accounting. Suggestion 1 covers a small overstatement.

Both r2 suggestions were taken (irqchip.rs:314, the guard-page record).

Rebase

• spawn_thread: both sides kept. Main's mask/siblings/homes and near: Some(Near { .. }) are there, beside the PR's image expect (kernel/src/process.rs:695-715). spawn keeps main's near: None (loader/mod.rs:424). The near: None that main added to sched/kthread.rs went with the deleted file. git range-diff shows no other change to commits 1, 4 and 5 beyond these resolutions.
• tests/images/acpicase.toml: both sides kept. Main's storage rows (diskserver, nvme, usbd, fileserver, virtio-console, serves = ["log-carrier"], acpiserver's devices) and the PR's uart row, console syscap, log-wire and the log-origins hold are all present. The start order matches every other image.

Required

1. system/supervisor/lifecycle/src/declaration.rs:68 (plan): after the rebase, the console UART's driver starts after the declaration and storage, so the UART is silent from userland's start until storage is up.
   • Where the row lands. The uart row (toyos_logstream::WIRE_HOLDER) is not logkeeper, does not hold acpi, does not mint, and is not storage. plan therefore puts it in rest, which storage_up starts only after the storage rows, make_session_home, the state directories and load_grants (system/supervisor/src/main.rs:1124-1140). On a boot that waits for the declaration, that is after the declaration, or after BOUND (30 s) through start_without_storage.
   • Why the UART is silent meanwhile. The kernel stops writing the UART at the hand-over, before userland runs (kernel/src/log/console.rs:1-7), and only the uart row writes it after that.
   • What changed from each side. Before the rebase, this PR started uart right after logkeeper. On main, the kernel writes its records to the UART inline and logkeeper writes program lines to it from logkeeper's start. The rebased branch has neither: no kernel record and no supervisor line reaches the UART during the declaration wait, storage bring-up or a home being made. A hang there, such as a disk driver or file server that never answers, leaves a UART-only machine with a silent console until a fatal path. That is exactly the failure the console exists to show. The lines are replayed later only if storage finishes.
   • What does not justify it. The uart claim is a kernel-described non-PCI device (DeviceType::Uart), so it needs no root-bridge declaration, and the row uses no HOME.
   • Evidence gap. The PR body says "the r5 rows and metal at 8ac4f2187 cover what r6 leaves untouched". This ordering did not exist at 8ac4f2187, so that evidence does not cover it.
   • Fix. Start the wire holder with logkeeper in plan (beside toyos_logstream::LOGKEEPER in first_here), and cover it with a plan host test next to the existing ones (declaration.rs:221 onward): the uart row is in first when a declaration comes. Then --ci host. With nothing to declare, first is empty and everything starts in storage_up, which is main's design for logkeeper too. Deciding whether uart should be ahead of the homes in that case can follow the same reasoning.

Suggestions

1. system/logkeeper/src/serve.rs:396 and the record's new paragraph say a refused stop's lines reach the UART twice. The test shows that only the lines the carrier already took as Said repeat (stopping). again, said after that, arrives once. Something like "those it took as they were said reach it again through the replay" would be exact.
2. issues/the-console-loses-what-its-carrier-had-not-delivered.md:51 is an unwrapped line, about 130 columns, left over from the reflow of the exit paragraph.

Evidence

I accepted items 1–3 by inspection. They are type, rights and text changes, and the implementer's --ci host and x86 guest row at d4c43fd35 cover them. Finding 1 needs only a host unit test of plan plus --ci host. A guest row is optional; any row whose UART capture shows supervisor lines before storage would show the change.

SEND BACK

[Japabu]

Response to review r3 (at 152cae5d1)

Required item 1: fixed. plan now starts the rows that carry the log first, by what they hold: the log's origins (logkeeper) and the console's input (the UART's driver). This replaces the match on logkeeper's name, and these rows sit beside the declaration's holder. the_uart_starts_with_the_log_before_the_declaration pins the UART's driver in first, and the old rule (logkeeper only) reds it with exit 101.

Where nothing can declare, both rows still start in main's storage order, as logkeeper does. Changing that is main's design to revisit, so I left it.

Both suggestions are taken. Only the lines the UART had already taken repeat on a refused stop, in the comment and in the record, and the record's exit paragraph is wrapped.

Checks: --ci host exit 0. On the x86 guest host, one dispatch of 6 rows passed: a_supervisor_that_fails_early_says_why_on_the_uart, both UART rows, machine_shutdown_short_stop, console_image_boots and session_lifecycle. The UART's claim is pid 4's, started beside logkeeper. Evidence.

The PR's head still reads 8ac4f2187 here, although the branch is at 152cae5d1.

[Japabu]

Superseded by #1175, which has the same branch no-kernel-threads at 152cae5d1. GitHub's PR view had stuck at 8ac4f2187, so I closed and reopened this PR to resync it. GitHub then refused the reopen because the branch had been rebased. Review rounds r1–r3 stay here for the record.