Skip to content

boot: the kernel hands the supervisor its images and parses no filesystem - #994

Draft
Japabu wants to merge 4 commits into
mainfrom
wt/toyos-boothandoff
Draft

Japabu wants to merge 4 commits into
mainfrom
wt/toyos-boothandoff

Conversation

@Japabu

@Japabu Japabu commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Draft. Top of a stack: #988 (landed) → #998 (dlopen from an image handle) → this, rebased on #998. Review round 1 is answered in its own commit (25d787482).

Problem

The kernel mounted ROOT itself with bcachefs, started the supervisor by path, and served /system to every process. The supervisor then read /system/etc/system.manifest through that mount. The owner's ruling of 2026-10-21, "Removing the boot filesystem exception", removes this: the supervisor and the file server become authenticated boot payloads, and the file server serves ROOT from memory before anything reads a file.

Resulting behaviour

  • One signed release binds every boot payload. The header format is now 2, with sections kernel, cmdline, supervisor, fileserver, root. Each section carries its role (its name and place), its length and its SHA-256, all under one version and one signature.

    • The two programs sit beside kernel.elf on the slot's FAT volume (toyos/supervisor.elf, toyos/fileserver.elf), and ROOT no longer carries them.
    • The update image and system/update's installer carry them too. An install whose supervisor bytes don't match is refused by section name before anything is written.
    • The cmdline no longer names ROOT's filesystem: root= is gone, and toyos_abi::boot::tokens replaces actuators.
  • The loader reads both programs into page-aligned LoaderData pages (loader/src/payload.rs) and checks them against their hashes before ExitBootServices.

    • A refusal names the section, e.g. Slot A: REFUSED, its supervisor is not the bytes its signed header names.
    • The existing fallback to the other slot applies.
  • The kernel receives three typed BootImage extents in KernelArgs (kernel/src/bootimage.rs).

    • It refuses the boot by name if an image is empty, unaligned or not whole pages inside one LoaderData descriptor.
    • It also refuses if any two reserved loader regions overlap (toyos_rootimage::handoff::overlapping, host-tested).
    • The pages stay out of the allocator, are charged to no process, and live for the machine's life.
    • Each image becomes a SharedMemObject with Memory::Boot. Every mapping of it is read-only, it is never lendable to a device, and fstat reports it as FileType::ImmutableMemory with the payload's exact length.
    • The supervisor is spawned from its image and endowed boot:fileserver and boot:root.
  • Deleted from the kernel:

    • rootfs.rs and bcachefs_adapter.rs;
    • the ROOT file backing;
    • the /system mount and UserAccess;
    • SUPERVISOR_PATH;
    • the bcachefs dependency.

    The kernel VFS now serves only /tmp.

  • The supervisor's bootstrap sets up authority before any file is read:

    1. It creates one port per role in toyos_manifest::ROLES, mints its own grants, and creates the launcher and grants ports.
    2. It starts the file server's system role from boot:fileserver, endowed with a duplicate of boot:root.
    3. It reads os-release and the manifest through std over the file protocol, with a bounded wait.
    4. Any role no manifest row serves has its port closed before anything asks it.

    Other behaviour around SYSTEM:

    • A bootstrap failure is written to the console, because no log ring is read that early.
    • SYSTEM restarts from the kept images under the usual budget. This is established by inspection only: no run ends SYSTEM, because nothing in a guest can kill it.
    • Every fileserver role starts from the boot image. The manifest row's source is typed (toyos_manifest::Source::BootFileServer, rendered boot-program), and a swap never targets SYSTEM.
  • fileserver's RootVolume (system/fileserver/src/root.rs) serves ROOT read-only from the mapped bytes:

    • every block read is bounds-checked;
    • every change is refused ReadOnly;
    • it uses the same protocol, resolver and rights as the other roles.

    Every view now holds /system read-only.

  • SDK and std:

    • toyos::shm::Immutable maps a region only after the kernel says nobody can write it.
    • Image::boot wraps an Immutable.
    • CommandExt::boot_image spawns from one.
  • A pre-existing kernel leak, fixed in its own commit: a shared-memory object held its mappers' address spaces strongly. A child that mapped its parent's log ring therefore kept its whole address space alive: tables, PCID and backings. Once every program is spawned from an image, that meant 2 MiB more in use per spawn. The list now holds those address spaces weakly.

Decisions

  • Owner (2026-10-21): the direction this implements, quoted in the brief.
  • Orchestrator: option A, i.e. do the prerequisites, not accept the gaps. files: one client of the file servers, in toyos, for std and libc alike #988 is the shared file-server client, and dlopen from an image handle comes below this PR. I record the loader's own bcachefs read of ROOT for microcode (issues/the-loader-parses-roots-filesystem-to-find-microcode.md).
  • Mine:
    • The boot images are files on the slot volume, read the way the kernel is.
    • KernelArgs grows by two extents at its end; root_image keeps its offset.
    • SYSTEM is started from a row the supervisor builds itself, because no manifest row can describe what serves the manifest.
    • The ROOT UUID is still derived. The harness now holds the file server's ROOT is filesystem <uuid> line where it held the kernel's root=.

Checks (head 25d787482, on #998 6ae79ccaa)

Host-only (macOS on Apple silicon); QEMU 11 per build/ci/qemu-version.

check exit
cargo run -- --ci host (87 steps, clippy included) 0
cargo run -- --build-only (x86_64) 0
one cargo test --test toyos-build -- console_image_boots virt_user_mode virt_smp nvme_disk_boots a_launch_gives_each_unsent_stdio_slot_a_null the_log_lands_on_the_boot_stick a_bent_supervisor a_spawn_from_an_image libc_children 1: eight PASS (x86_64, plus virt_* on aarch64; the bent-supervisor test boots both). libc_children red: spawn_children_cap gets ENOMEM (Risks)
scratch QEMU run of every dlopen guest test and C case over /system served by the file server (patch in the evidence) 0
scratch ps after spawns: the supervisor has no Z thread 0

Negative controls were run at ff9b4b1c5. The code each one mutates (loader, updater, rootimage, update header, shm) is unchanged since, so the results still hold:

control result
loader skips the program hash the bent-supervisor test is red
the shm fix reverted the image-lifetime test is red (20 MiB more in use)
installer skips the supervisor hash the updater's copies test is red
overlap check misses a later start the rootimage host test is red
header takes an empty program the update host test is red

Memory and image cost, measured at ff9b4b1c5: QEMU x86_64, tests/images/testcases.toml, toybox free after the boot, one boot each in the same session.

  • 330 MiB in use at b9e4c5862, 384 MiB here: 54 MiB more. That is the running services' images (each rounded up to 2 MiB) and the SYSTEM server.
  • ROOT carries 3.5 MB less: the two programs, 1,910,584 and 1,584,864 bytes in this x86_64 build.
  • No size saving is claimed.

Metal:

  • control_regs passed on the T14 at ff9b4b1c5.
  • bent_supervisor_falls_back was not flashed: toyos-metal required exactly one ROOT. It now admits an image by its slot table instead, and both rows are restaged at this head.

Evidence: ff1189541, with the first round at 07e0808d2.

Risks

  • libc_children is red. spawn_children_cap holds 255 children, and each was spawned from its own 2 MiB image of its program, so it runs out of memory partway; the cause is inferred, not measured.
    • This is the per-spawn image cost of issues/root-in-memory-costs-its-whole-size.md, now visible as lost capacity.
    • A zero-copy proposal is with the orchestrator: the file server hands out an immutable slice of ROOT for a contiguous file.
  • 54 MiB more memory on the test image, for the same reason.
  • The loader's own bcachefs read of ROOT for microcode is accepted for landing and recorded.
  • No metal run was made by the implementer.

🤖 Generated with Claude Code

@Japabu

Japabu commented Oct 22, 2026

Copy link
Copy Markdown
Collaborator Author

Review of #994, round 1 (head ff9b4b1c5, base b9e4c5862)

I read both commits, the whole diff, every changed file, and the callers that matter: the kernel's shm users and SharedImage, paging::AddressSpace and its PCID reclaim, the supervisor's start, restart and swap paths, and std's Command spawn path. I also read the owner record (claude-questions.md, "Removing the boot filesystem exception", 2026-10-21), the handoff, the plan and the evidence under agents/boothandoff/. I ran nothing.

Architecture

The design matches the owner's ruling.

  • One signed header (format 2) binds the kernel, cmdline, supervisor, fileserver and ROOT, each by role, length and hash.
  • The loader verifies every section before ExitBootServices.
  • The kernel no longer depends on bcachefs. It no longer mounts ROOT, opens a /system path or spawns anything by path. It validates three typed extents and starts the supervisor from its image.
  • The supervisor sets up ports and grants before any read. It starts the existing fileserver in a system role over the read-only ROOT bytes, then reads os-release and the manifest over the ordinary protocol.
  • ROOT no longer carries the two programs.
  • Net production is about +420 lines once in-file tests are excluded (+1707/−1033 including them; tests +319/−49). The additions are justified: RootVolume (about 270), the bootstrap, payload.rs and bootimage.rs. Against them, rootfs.rs, bcachefs_adapter.rs, ReadOnlyBacking and UserAccess are deleted.

The temporary gaps (libc path calls, posix_spawn, dlopen of /system) are correctly scoped to the stack. This PR must not land before #988 and the dlopen-by-handle PR; libc_children is red at this head.

The loader's own bcachefs read of ROOT for microcode is acceptable for landing:

  • The ruling is explicitly about the kernel ("from the kernel"; "no filesystem implementation remains there").
  • The loader reads ROOT only after ROOT's hash has verified.
  • issues/the-loader-parses-roots-filesystem-to-find-microcode.md records it, with an owner and a candidate exit.
  • Whether to make microcode a signed section is a later decision that touches a-new-microcode-patch-reaches-a-machine-only-in-a-new-image.md.

What I verified

  • Verification happens before any use. slot::verify reads and hashes each section in order: kernel, cmdline, then supervisor and fileserver via program(), which hashes the pages actually handed over rather than the read buffer, then ROOT. Chosen exists only after all five match. Microcode reads chosen.root.pages only after that. Every early return frees the pages allocated so far, including on ROOT read failure and ROOT hash failure. The fallback to the other slot goes through the unchanged choose loop.
  • Role, length and hash binding.
    • A section's name is checked as its whole padded field at its fixed index, so swapped program entries are refused SectionName(2) (tested).
    • The length is in the signed entry, and read_section refuses a file whose size differs.
    • Empty and oversized programs are refused by name (tested).
  • Downgrade resistance. A format-1 header is refused, and the existing floor and death record still apply.
  • Mix-and-match resistance. Sections from different slots fail their hashes under the one signed header.
  • Kernel extents.
    • bootimage::init asserts each extent is non-empty and that checked_next_multiple_of succeeds.
    • held checks alignment and checked_add, and that the extent lies inside one LoaderData descriptor.
    • kernel_main then runs overlapping across every reserved region, including arch::boot::reserved().
    • The unchecked arithmetic in pages() is reached only after init passed.
  • Immutability.
    • Memory::Boot maps Prot::Read.
    • ram() is None, so dma_map refuses the region and so does SharedImage::over, which now uses is_ordinary.
    • No other kernel path writes through phys(): phys_before_mapping is reached only for inbox pages the kernel created.
  • RootVolume. Bytes::block uses checked multiply and add plus get. read slices only within one block, and a block past ROOT's end returns Io. Every mutator answers ReadOnly. The host tests cover a read back, a symlink, a listing, every mutator, and both non-volume bytes and an out-of-range block.
  • The shm leak fix (1e2095fe0, which compiles alone on both kernel targets according to the shm-alone logs).
    • The weak list is sound because AddressSpace owns only its table pages and its PCID or ASID.
    • The PCID pool flushes before it reissues a tag, so a dead space's TLB entries cannot be used before its pages are freed.
    • map_into identifies the space by pointer after pruning dead entries. That closes the pid-reuse case the commit names.
    • The guest test plus its control (the fix reverted is red, about 20 MiB) establishes the claim.
  • Evidence.
    • final-head.txt is ff9b4b1c5.
    • The host suite and --build-only both exit 0.
    • QEMU: 8 PASS, plus the expected libc_children red.
    • The loader control ran at 01df4d7, and git diff 01df4d7 ff9b4b1c5 -- loader/ lib/update tests/common/slots.rs is empty, so its reuse is valid.

Findings

BLOCKER

  1. kernel/src/file_backing.rs:63-69: the SAFETY comment in SharedImage::read_page is now false.

    • It says the object is "one physically contiguous run the kernel allocated (ram) and keeps while object lives".
    • A Memory::Boot object is loader-allocated LoaderData that the kernel keeps reserved for the machine's life. It is neither ram() nor kernel-allocated.
    • The comment is an unsafe precondition, so it must state the real one: the object's pages are direct-mapped ordinary memory (is_ordinary), alive while object lives.
  2. lib/manifest / build/build.rs:791 / system/supervisor/src/main.rs:779 / sdk/abi/src/syscall.rs:396-401: a kernel ABI constant is the manifest's vocabulary, overloaded into a path field.

    • The build writes toyos_abi::syscall::FILESERVER_IMAGE_LABEL ("boot:fileserver") as a manifest row's path.
    • The supervisor selects the boot image with path == FILESERVER_IMAGE_LABEL.
    • The ABI doc now also describes the manifest ("also the path its manifest row names").
    • The manifest is a userland protocol between the build and the supervisor. Root's rule is that such protocols do not take their definitions from toyos-abi. The result is also an untyped either-a-path-or-a-label field that every program.path consumer has to know about (the launcher's resolve, swap's installed_digest, the image test's filter).
    • Fix: the manifest owns the representation, preferably typed (for example a program source of Boot or Path, or "a storage row runs the boot file server"). The ABI label stays only the kernel-to-supervisor endowment label, and the doc clause goes.
  3. system/supervisor/src/main.rs:3317-3335: the bootstrap reader threads are unexplained debt.

    • The PR's own Risks section says they stay Z in ps and were not investigated, and nothing in issues/ records it.
    • The cheapest fix is to keep the JoinHandle and join it once recv_timeout has answered. Then check ps once.
    • If a joined thread still shows Z, or a detached thread that ended shows Z in any program, that is a std or kernel defect. It needs an issue with its evidence, per "No silent debt".
  4. Issue records that this change directly falsifies. Each one cites deleted code or deleted behaviour as current:

    • issues/sys-debug-actions-and-two-loader-words-that-nothing-calls.md:23 lists boot::WITHHOLD_ROOT_PARAM as an unused name. This PR deletes it, which meets that name's exit, so the line goes.
    • issues/the-guest-suite-runs-only-what-no-cheaper-tier-reaches.md:37-39:
      • root_from_memory says "ROOT mounts from the loader's image"; the kernel no longer mounts it.
      • root_withheld_refused "brings back loader-withholds-root", which no longer exists.
      • Both rows need to be restated against the new handoff: the kernel's refusal of an empty or unheld extent, and SYSTEM serving ROOT with no storage command.
    • issues/a-supervisor-panic-before-logkeeper-starts-reaches-no-console.md: bootstrap_fails now puts the os-release and manifest read failures on the console. The record still says those refusals reach no console. The os-release parse panic (say_release, line 3302) and a manifest parse panic still go only to the ring. Either route them through bootstrap_fails too (one line each), or record precisely what is left.
    • issues/libc-answers-einval-for-every-error-it-does-not-name.md:18 cites kernel/src/bcachefs_adapter.rs's write as a current EINVAL source; that file is deleted.
  5. sdk/abi/src/boot.rs:68-70: the rewritten cmdline contract is false.

    • "Every shipping image carries none" is wrong because the loader appends boot-slot= to every boot.
    • "Refuses any token" is wrong because params::claims tokens are accepted. The actuator table refuses only unclaimed tokens (kernel/src/actuator.rs:218).
    • State it as: every token is a kernel parameter (params) or a declared actuator, and a kernel without actuators refuses any other.
  6. The evidence gap on SYSTEM's restart. The ruling requires "restart access to the bootstrap images". The PR states as behaviour that "SYSTEM restarts from the kept images under the usual budget".

    • The plan's guest check (kill SYSTEM, read /system again) was dropped without explanation, and no run exercises the restart.
    • By inspection the path is plausible: the generic ended branch respawns service.path, which is "boot:fileserver", and storage_endowment duplicates ROOT afresh on every start.
    • Either run one QEMU check that ends SYSTEM and reads a /system file afterwards, or label the claim in the PR and handoff as established by inspection only.

SUGGESTION

  • kernel/src/bootimage.rs:38: HANDED is a static that both of its call sites, in kernel_main, could avoid. init could return the Handed value next to the regions, and objects(handed) could take it. That removes one lock, one Option state and the "taken once" assertion.
  • system/supervisor/src/main.rs:3359-3367: Boot::take maps all of ROOT into the supervisor, which never reads it; it only duplicates the handle. Checking fstat == ImmutableMemory and keeping the OwnedHandle would avoid the page tables for a ROOT-sized mapping in the supervisor.
  • sdk/toyos/src/shm.rs:179: Immutable::adopt refuses with Lent::NotRam, whose message is "it is not memory the kernel allocated". That is not the reason (a boot image is not kernel-allocated either), so the supervisor's and fileserver's panics would print a misleading cause. A dedicated variant or message would be accurate.
  • system/supervisor/src/main.rs:1504: a swap picks the first service named fileserver, and that is now SYSTEM (services[0]), not the DATA, LOG or BOOT server it used to hit. The ambiguity existed before, but its consequence changed. Consider whether a swap should name a role.
  • sdk/abi/src/boot.rs a_boot_parameter_is_its_non_empty_tokens tests one split(',').filter, which inspection already establishes. It could be deleted.
  • loader/src/slot.rs program() reads into a Vec and then copies into pages. Reading straight into the Payload would save one allocation and one copy per program. This is minor.

Notes (not findings)

  • I am not judging metal (control_regs, bent_supervisor_falls_back); that run is the orchestrator's. The judge's lines are the loader's Refusal::Hash(SUPERVISOR) text and SLOT_RECORD, which are the right oracles.
  • The +54 MiB is measured, attributed by inference (the running services' images rounded to 2 MiB, plus SYSTEM), and recorded with an exit in root-in-memory-costs-its-whole-size.md. That is what the ruling asked for.
  • Before landing, rebase onto files: one client of the file servers, in toyos, for std and libc alike #988 and dlopen-by-handle, then rerun libc_children and the dlopen rows. The c-programs issue's /system text must be accurate at that point.

SEND BACK — blockers 1–6.

SYS_DLOPEN opened a path through the kernel's VFS, so no library a file
server holds could load, and the kernel kept a cross-process cache of
every library it loaded that never evicted. The small-kernel track's
step 5 asks for dlopen from an image handle, and the orchestrator ruled,
under the memory design, that the cache goes and each process holds its
own library pages; that design's later stages bring sharing back through
memory objects.

- ABI: SYS_DLOPEN takes a DlopenArgs: the name the module goes by, the
  image handle and its length, and init_out. The kernel opens nothing by
  the name; it is what SYS_QUERY_MODULES answers and what a second load
  of the same name is known by, which answers the module already loaded.
- Kernel: sys_dlopen loads from the SharedImage; elf/cache.rs, its
  budget and identity (vfs::BackingId, open_backing_identified) and
  LibMemory's shared arm are deleted, and a LoadedLib owns its one
  allocation.
- SDK: toyos::files::read_image reads a file into an image wherever it is
  served, and toyos::files::dl_open loads one under its absolute path.
  std's read_image is that helper; libc's dlopen calls dl_open, and its
  posix_spawn of a served program reads it into an image as std's
  Command does (ENOEXEC for an empty file).
- The libloading fork's ToyOS backend calls toyos::files::dl_open
  (ToyOSOrg/rust_libloading toyos-sdk-0.12 da187608c), pinned here.
- Guest tests call the SDK's dl_open; abuse_kernel_addr builds its
  DlopenArgs around an image read once.
- Issues: step 5 is done; the shared-object cache's residual record is
  deleted with the cache; the refusal-log record and the parsing track no
  longer cite it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Japabu and others added 3 commits October 22, 2026 03:53
A shared-memory object kept a strong reference to the page tables of
every process that mapped it, released only when the object's last
handle went. A process that ended while another still held the region
(a child mapping its parent's log ring, a client's lent window) kept its
whole address space alive: its tables, its PCID and every backing its
regions held. With every /system program spawned from an image object,
that was the image too: QEMU, tests/images/testcases.toml, 2 MiB more in
use for every spawn of toybox.

The list now holds each address space weakly. A dead entry is passed
over by the zero-handle teardown, by unmap_from and by map_into, which
also drops dead entries first, so a reused pid no longer finds the
mapping of the process it replaced.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…stem

The kernel mounted ROOT with bcachefs and started the supervisor by
path, and the supervisor read the manifest through the kernel's VFS.
Owner ruling 2026-10-21 ("Removing the boot filesystem exception")
removes that exception: the supervisor and the file server are signed
boot sections beside the kernel and ROOT, the kernel starts the
supervisor from its image and hands it the other two, and the file
server serves /system out of ROOT's memory before anything reads a file.

- Signed header format 2: kernel, cmdline, supervisor, fileserver, root,
  each bound by role, length and hash under the one version. The two
  programs sit beside kernel.elf on the slot's FAT volume, and ROOT no
  longer carries them. The update image and installer carry them too.
  The cmdline no longer names ROOT's filesystem.
- The loader reads both programs into page-aligned LoaderData pages and
  holds them to their hashes; a refusal names the section and happens
  before ExitBootServices, with the usual fallback to the other slot.
- KernelArgs carries three typed BootImage extents. The kernel refuses
  by name an empty, unaligned or unheld image and any two reserved
  regions that overlap, keeps the pages out of its allocator for the
  machine's life, and makes each a shared-memory object every mapping of
  which is read-only (Memory::Boot, FileType::ImmutableMemory).
- Deleted from the kernel: rootfs.rs, bcachefs_adapter.rs, the ROOT file
  backing, the /system mount and its access flag, SUPERVISOR_PATH and
  the bcachefs dependency. The kernel VFS serves /tmp alone.
- The supervisor makes every role's port and its own grants before it
  reads a file, starts the file server's SYSTEM role from its boot image
  with ROOT's image, then reads os-release and the manifest over the
  file protocol. A bootstrap failure is said on the console, since no
  log ring is read yet. SYSTEM restarts from the kept images; every
  fileserver role starts from the boot image; a role no row serves is
  closed before anything asks it.
- fileserver's RootVolume serves ROOT read-only from the mapped bytes,
  bounds-checking every block; every view holds /system read-only.
- std's CommandExt::boot_image spawns from an immutable region;
  toyos::shm::Immutable and Image::boot hold one.
- The harness reads the file server's ROOT line where it read the
  kernel's root=; a QEMU test and a metal row boot an image whose marked
  slot's supervisor is bent and judge the refusal and the fallback.
- Raw-syscall guest tests read their programs into images and open
  copies in /tmp, since the kernel opens no file of /system.
- A QEMU test holds that eight spawns from an image leave memory in use
  as it was, which the previous commit's lifetime fix is what keeps.
- Memory: 54 MiB more in use after a testcases boot on QEMU (330 to 384
  MiB), the running services' images and the SYSTEM server; recorded in
  issues/root-in-memory-costs-its-whole-size.md.
- Issues: the storage, small-kernel, parsing, isolation and update
  tracks and the ROOT memory finding are reconciled; the loader's own
  bcachefs read for microcode is recorded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A manifest row's source is typed (toyos_manifest::Source): a path, or
  the file server's boot image, rendered `boot-program <name>`. The kernel
  ABI's label is the endowment's alone, and its doc no longer speaks for
  the manifest. A swap never targets SYSTEM, which runs the signed image.
- SharedImage's SAFETY comment states the precondition boot images meet:
  ordinary memory alive while the object lives.
- The supervisor joins each bootstrap reader once it has answered, and
  keeps the boot images as handles it checked are immutable rather than
  mapping ROOT it never reads; an os-release the build did not write is
  said on the console too.
- KernelArgs' cmdline contract names the loader's words and the kernel's
  parameters beside the actuators; bootimage::init hands the images back
  rather than through a static; Immutable::adopt refuses with its own
  word.
- toyos-metal admits an image by its one slot table, so a two-slot image
  (the bent-supervisor row) is flashable.
- Issue records that cited deleted code are restated: the unused loader
  words, the guest suite's ROOT rows, the supervisor's console record, the
  libc EINVAL record and the SMP record.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu
Japabu force-pushed the wt/toyos-boothandoff branch from ff9b4b1 to 25d7874 Compare October 22, 2026 02:47
@Japabu

Japabu commented Oct 22, 2026

Copy link
Copy Markdown
Collaborator Author

Round 1 answered at 25d787482. The branch is rebased on #998 (dlopen from an image handle), which sits on #988.

Blockers:

  1. SAFETY comment. SharedImage::read_page now states the real precondition: one contiguous run of ordinary memory (is_ordinary, which covers kernel-allocated pages and boot images kept for the machine's life), alive while object lives.
  2. Manifest ownership. toyos_manifest::Source is either Path(String) or BootFileServer, and it renders as boot-program <name>.
    • Every program.path consumer now matches on the source: the service starts, swap's installed digest, the launcher's declared and resolve, the drives start, and the image test's filter.
    • FILESERVER_IMAGE_LABEL is the endowment label only, and the ABI doc's manifest clause is gone.
    • A swap no longer picks SYSTEM: SYSTEM always runs the signed image, so a fileserver swap now hits the role it used to.
  3. Bootstrap threads.
    • read_system joins its reader once the channel has answered.
    • At this head, ps after spawns shows no Z thread in the supervisor (r2-ps.log).
    • Boot now keeps the two images as handles, checked ImmutableMemory by fstat, and no longer maps ROOT.
  4. Issue records.
    • The sys-debug record drops WITHHOLD_ROOT_PARAM.
    • The guest-suite rows are restated against the handoff.
    • The supervisor-console record now says what bootstrap_fails covers, including the os-release parse, which this round routes through it. What is left is a manifest that parse panics on, and any other panic before logkeeper.
    • The libc EINVAL record drops the deleted adapter.
    • smp-ap-hole… no longer cites rootfs.rs.
  5. cmdline contract. It now says: the slot's tokens plus the loader's words; each token is a claimed kernel parameter or a declared actuator; a kernel without actuators refuses any other.
  6. SYSTEM restart. This is by inspection only: ended respawns service.source (BootFileServer), and storage_endowment duplicates ROOT on every start. No run ends SYSTEM, because nothing in a guest can kill it. The PR body now says so.

Suggestions:

  • Taken: bootimage no longer has the static; Boot keeps handles instead of mapping ROOT; Immutable::adopt refuses with its own NotImmutable; the trivial token test is deleted.
  • Left: program() still reads into a Vec and then copies into pages. It is one 2 MB copy before ExitBootServices.

Metal: toyos-metal now admits an image by its one TOYOS-SLOTS table rather than by exactly one ROOT, so the two-slot image of bent_supervisor_falls_back is flashable (unit-tested in an_image_admits_only_the_table_the_installed_rule_names). Both rows are restaged at this head.

New red at this head: libc_children.

  • Its dev_null case now passes, because posix_spawn reads served programs into images (dlopen: the kernel loads a library from an image and opens no path #998).
  • spawn_children_cap answers ENOMEM partway through its 255 held children.
  • Each child is spawned from its own copy of its program (a 2 MiB image), so 255 children hold about 510 MiB more than at base, on a 4 GiB guest. That cause is inferred, not measured.
  • This is the same per-spawn image cost recorded in root-in-memory-costs-its-whole-size.md. A proposal is with the orchestrator.

Evidence: https://github.com/ToyOSOrg/evidence/tree/ff11895416ea92090e15f698f7862442daa871a7/head-25d787482

@Japabu

Japabu commented Oct 22, 2026

Copy link
Copy Markdown
Collaborator Author

Review of #994, round 2 (head 25d787482, on #998 6ae79ccaa)

This round is focused. I checked:

I ran nothing. As directed, libc_children is out of scope here (the per-spawn 2 MiB images; the PR is held until the backing-memory stages land, and no ROOT-slice mechanism is to be added), and I judge everything else. The worktree has uncommitted edits to tests/common/libcchildren.rs and tests/guest/libc/spawn_children_cap.c. They are not part of this head and I did not review them.

Round-1 blockers

  1. SAFETY comment. Resolved. file_backing.rs:65-70 now states the real precondition: contiguous ordinary memory (is_ordinary, covering kernel-allocated pages or a boot image kept for the machine's life), alive while the object lives.
  2. Manifest vocabulary. Resolved.
    • toyos_manifest::Source { Path, BootFileServer } is owned by the manifest crate, and it renders and parses as boot-program <name>.
    • Every former path consumer now matches on the source: service start, swap's installed digest, declared/resolve, the drives start, the grant digest and the image test.
    • A launch of a boot-image row is refused by name.
    • The ABI label is now only the endowment label, and its manifest clause is gone.
    • Swap now excludes the SYSTEM role, so a fileserver swap reaches the role it used to.
  3. Bootstrap threads. Resolved for the supervisor. read_system joins its reader once it has answered, and r2-ps.log shows no Z thread. Boot keeps fstat-checked handles and no longer maps ROOT. One residue remains; see the first required item below.
  4. Issue records. Resolved. The sys-debug, guest-suite, supervisor-console, libc-EINVAL and SMP records are each now accurate. The os-release parse failure is routed through bootstrap_fails, and what is left, a manifest-parse panic, is recorded.
  5. cmdline contract. Resolved. The doc now names the slot's tokens and the loader's words (SLOT_PARAM), and says that each token is a claimed parameter or a declared actuator.
  6. SYSTEM restart. Resolved by honest labelling. The PR body says "established by inspection only", with the reason that nothing in a guest can end SYSTEM. My inspection agrees: ended respawns service.source == BootFileServer, and storage_endowment duplicates ROOT on every start.

New in this round

toyos-metal admission (tests/metal/src/lib.rs:902-909). An image is now admitted by exactly one ESP and one TOYOS-LOG at the indices the installed rule names, plus exactly one TOYOS-SLOTS, instead of exactly one TOYOS-ROOT. That is the right key: the slot table is what the loader boots by, and the ESP and LOG indices, which carry the safety the bench's installed rule depends on, are still checked. The unit test covers a missing table, a two-slot image and the single-slot built shape.

Findings

BLOCKER

  1. A record is still owed for ended detached threads that stay Z.

    • At ff9b4b1c5, the supervisor's two detached bootstrap threads showed Z after they ended. Joining them made it disappear.
    • That is the condition round 1 named. A detached std thread that has ended is never reaped on ToyOS, so every detached thread that ends leaks its record for the life of its process.
    • The supervisor itself still detaches others, for example sync-{role} at main.rs:1774.
    • Joining fixed the symptom here but not the defect.
    • The fix is cheap: an issue (kind: defect) carrying the ff9b4b1c5 ps evidence, with an exit stating that std's detach releases the thread, or that the kernel reaps an ended thread nobody will join.
  2. tests/metal/src/lib.rs:902-904: the doc says the slot table "names the slots the loader boots — one or two ROOTs", but admit counts no ROOT. An image with a table and zero, or three, ROOTs is admitted. Either check that there are one or two TOYOS-ROOT partitions, which is one more count beside one_partition, or drop the clause. This is a bench-flash gate, so its doc must say what it checks.

SUGGESTION

  • The per-spawn cost now has a measured consequence beyond the +54 MiB: spawn_children_cap gets ENOMEM at 255 children (cause inferred). Recording that in issues/root-in-memory-costs-its-whole-size.md would keep the record complete while the PR is held. The hold itself is the orchestrator's decision, which I am not reopening.
  • toyos_manifest::Source's Default is Path(String::new()), a state no parsed row can have. It is harmless with ..Program::default(), but a row built without a source renders as program <name> and is refused only by check.

Verdict

Round 1 is resolved except for the record in blocker 1. The admission change is sound apart from its doc.

SEND BACK for blockers 1–2, both small. Independently of these, landing remains held by the orchestrator until the backing-memory stages land and libc_children is green. Metal (control_regs, bent_supervisor_falls_back) is still the orchestrator's to run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant