Repository navigation
Conversation
Review of #994, round 1 (head
|
SYS_DLOPEN opened a path through the kernel's VFS, so no library a file server holds could load, and the kernel kept a cross-process cache of every library it loaded that never evicted. The small-kernel track's step 5 asks for dlopen from an image handle, and the orchestrator ruled, under the memory design, that the cache goes and each process holds its own library pages; that design's later stages bring sharing back through memory objects. - ABI: SYS_DLOPEN takes a DlopenArgs: the name the module goes by, the image handle and its length, and init_out. The kernel opens nothing by the name; it is what SYS_QUERY_MODULES answers and what a second load of the same name is known by, which answers the module already loaded. - Kernel: sys_dlopen loads from the SharedImage; elf/cache.rs, its budget and identity (vfs::BackingId, open_backing_identified) and LibMemory's shared arm are deleted, and a LoadedLib owns its one allocation. - SDK: toyos::files::read_image reads a file into an image wherever it is served, and toyos::files::dl_open loads one under its absolute path. std's read_image is that helper; libc's dlopen calls dl_open, and its posix_spawn of a served program reads it into an image as std's Command does (ENOEXEC for an empty file). - The libloading fork's ToyOS backend calls toyos::files::dl_open (ToyOSOrg/rust_libloading toyos-sdk-0.12 da187608c), pinned here. - Guest tests call the SDK's dl_open; abuse_kernel_addr builds its DlopenArgs around an image read once. - Issues: step 5 is done; the shared-object cache's residual record is deleted with the cache; the refusal-log record and the parsing track no longer cite it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A shared-memory object kept a strong reference to the page tables of every process that mapped it, released only when the object's last handle went. A process that ended while another still held the region (a child mapping its parent's log ring, a client's lent window) kept its whole address space alive: its tables, its PCID and every backing its regions held. With every /system program spawned from an image object, that was the image too: QEMU, tests/images/testcases.toml, 2 MiB more in use for every spawn of toybox. The list now holds each address space weakly. A dead entry is passed over by the zero-handle teardown, by unmap_from and by map_into, which also drops dead entries first, so a reused pid no longer finds the mapping of the process it replaced. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…stem
The kernel mounted ROOT with bcachefs and started the supervisor by
path, and the supervisor read the manifest through the kernel's VFS.
Owner ruling 2026-10-21 ("Removing the boot filesystem exception")
removes that exception: the supervisor and the file server are signed
boot sections beside the kernel and ROOT, the kernel starts the
supervisor from its image and hands it the other two, and the file
server serves /system out of ROOT's memory before anything reads a file.
- Signed header format 2: kernel, cmdline, supervisor, fileserver, root,
each bound by role, length and hash under the one version. The two
programs sit beside kernel.elf on the slot's FAT volume, and ROOT no
longer carries them. The update image and installer carry them too.
The cmdline no longer names ROOT's filesystem.
- The loader reads both programs into page-aligned LoaderData pages and
holds them to their hashes; a refusal names the section and happens
before ExitBootServices, with the usual fallback to the other slot.
- KernelArgs carries three typed BootImage extents. The kernel refuses
by name an empty, unaligned or unheld image and any two reserved
regions that overlap, keeps the pages out of its allocator for the
machine's life, and makes each a shared-memory object every mapping of
which is read-only (Memory::Boot, FileType::ImmutableMemory).
- Deleted from the kernel: rootfs.rs, bcachefs_adapter.rs, the ROOT file
backing, the /system mount and its access flag, SUPERVISOR_PATH and
the bcachefs dependency. The kernel VFS serves /tmp alone.
- The supervisor makes every role's port and its own grants before it
reads a file, starts the file server's SYSTEM role from its boot image
with ROOT's image, then reads os-release and the manifest over the
file protocol. A bootstrap failure is said on the console, since no
log ring is read yet. SYSTEM restarts from the kept images; every
fileserver role starts from the boot image; a role no row serves is
closed before anything asks it.
- fileserver's RootVolume serves ROOT read-only from the mapped bytes,
bounds-checking every block; every view holds /system read-only.
- std's CommandExt::boot_image spawns from an immutable region;
toyos::shm::Immutable and Image::boot hold one.
- The harness reads the file server's ROOT line where it read the
kernel's root=; a QEMU test and a metal row boot an image whose marked
slot's supervisor is bent and judge the refusal and the fallback.
- Raw-syscall guest tests read their programs into images and open
copies in /tmp, since the kernel opens no file of /system.
- A QEMU test holds that eight spawns from an image leave memory in use
as it was, which the previous commit's lifetime fix is what keeps.
- Memory: 54 MiB more in use after a testcases boot on QEMU (330 to 384
MiB), the running services' images and the SYSTEM server; recorded in
issues/root-in-memory-costs-its-whole-size.md.
- Issues: the storage, small-kernel, parsing, isolation and update
tracks and the ROOT memory finding are reconciled; the loader's own
bcachefs read for microcode is recorded.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A manifest row's source is typed (toyos_manifest::Source): a path, or the file server's boot image, rendered `boot-program <name>`. The kernel ABI's label is the endowment's alone, and its doc no longer speaks for the manifest. A swap never targets SYSTEM, which runs the signed image. - SharedImage's SAFETY comment states the precondition boot images meet: ordinary memory alive while the object lives. - The supervisor joins each bootstrap reader once it has answered, and keeps the boot images as handles it checked are immutable rather than mapping ROOT it never reads; an os-release the build did not write is said on the console too. - KernelArgs' cmdline contract names the loader's words and the kernel's parameters beside the actuators; bootimage::init hands the images back rather than through a static; Immutable::adopt refuses with its own word. - toyos-metal admits an image by its one slot table, so a two-slot image (the bent-supervisor row) is flashable. - Issue records that cited deleted code are restated: the unused loader words, the guest suite's ROOT rows, the supervisor's console record, the libc EINVAL record and the SMP record. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ff9b4b1 to
25d7874
Compare
|
Round 1 answered at Blockers:
Suggestions:
Metal: New red at this head:
Evidence: https://github.com/ToyOSOrg/evidence/tree/ff11895416ea92090e15f698f7862442daa871a7/head-25d787482 |
Review of #994, round 2 (head
|
Draft. Top of a stack: #988 (landed) → #998 (dlopen from an image handle) → this, rebased on #998. Review round 1 is answered in its own commit (
25d787482).Problem
The kernel mounted ROOT itself with bcachefs, started the supervisor by path, and served
/systemto every process. The supervisor then read/system/etc/system.manifestthrough that mount. The owner's ruling of 2026-10-21, "Removing the boot filesystem exception", removes this: the supervisor and the file server become authenticated boot payloads, and the file server serves ROOT from memory before anything reads a file.Resulting behaviour
One signed release binds every boot payload. The header format is now 2, with sections
kernel, cmdline, supervisor, fileserver, root. Each section carries its role (its name and place), its length and its SHA-256, all under one version and one signature.kernel.elfon the slot's FAT volume (toyos/supervisor.elf,toyos/fileserver.elf), and ROOT no longer carries them.system/update's installer carry them too. An install whose supervisor bytes don't match is refused by section name before anything is written.root=is gone, andtoyos_abi::boot::tokensreplacesactuators.The loader reads both programs into page-aligned
LoaderDatapages (loader/src/payload.rs) and checks them against their hashes before ExitBootServices.Slot A: REFUSED, its supervisor is not the bytes its signed header names.The kernel receives three typed
BootImageextents inKernelArgs(kernel/src/bootimage.rs).LoaderDatadescriptor.toyos_rootimage::handoff::overlapping, host-tested).SharedMemObjectwithMemory::Boot. Every mapping of it is read-only, it is never lendable to a device, andfstatreports it asFileType::ImmutableMemorywith the payload's exact length.boot:fileserverandboot:root.Deleted from the kernel:
rootfs.rsandbcachefs_adapter.rs;/systemmount andUserAccess;SUPERVISOR_PATH;The kernel VFS now serves only
/tmp.The supervisor's bootstrap sets up authority before any file is read:
toyos_manifest::ROLES, mints its own grants, and creates the launcher and grants ports.systemrole fromboot:fileserver, endowed with a duplicate ofboot:root.Other behaviour around SYSTEM:
toyos_manifest::Source::BootFileServer, renderedboot-program), and a swap never targets SYSTEM.fileserver's
RootVolume(system/fileserver/src/root.rs) serves ROOT read-only from the mapped bytes:ReadOnly;Every view now holds
/systemread-only.SDK and std:
toyos::shm::Immutablemaps a region only after the kernel says nobody can write it.Image::bootwraps anImmutable.CommandExt::boot_imagespawns from one.A pre-existing kernel leak, fixed in its own commit: a shared-memory object held its mappers' address spaces strongly. A child that mapped its parent's log ring therefore kept its whole address space alive: tables, PCID and backings. Once every program is spawned from an image, that meant 2 MiB more in use per spawn. The list now holds those address spaces weakly.
Decisions
issues/the-loader-parses-roots-filesystem-to-find-microcode.md).KernelArgsgrows by two extents at its end;root_imagekeeps its offset.ROOT is filesystem <uuid>line where it held the kernel'sroot=.Checks (head
25d787482, on #9986ae79ccaa)Host-only (macOS on Apple silicon); QEMU 11 per
build/ci/qemu-version.cargo run -- --ci host(87 steps, clippy included)cargo run -- --build-only(x86_64)cargo test --test toyos-build -- console_image_boots virt_user_mode virt_smp nvme_disk_boots a_launch_gives_each_unsent_stdio_slot_a_null the_log_lands_on_the_boot_stick a_bent_supervisor a_spawn_from_an_image libc_childrenvirt_*on aarch64; the bent-supervisor test boots both).libc_childrenred:spawn_children_capgetsENOMEM(Risks)/systemserved by the file server (patch in the evidence)psafter spawns: the supervisor has noZthreadNegative controls were run at
ff9b4b1c5. The code each one mutates (loader, updater, rootimage, update header, shm) is unchanged since, so the results still hold:Memory and image cost, measured at
ff9b4b1c5: QEMU x86_64,tests/images/testcases.toml, toyboxfreeafter the boot, one boot each in the same session.b9e4c5862, 384 MiB here: 54 MiB more. That is the running services' images (each rounded up to 2 MiB) and the SYSTEM server.Metal:
control_regspassed on the T14 atff9b4b1c5.bent_supervisor_falls_backwas not flashed:toyos-metalrequired exactly one ROOT. It now admits an image by its slot table instead, and both rows are restaged at this head.Evidence:
ff1189541, with the first round at07e0808d2.Risks
libc_childrenis red.spawn_children_capholds 255 children, and each was spawned from its own 2 MiB image of its program, so it runs out of memory partway; the cause is inferred, not measured.issues/root-in-memory-costs-its-whole-size.md, now visible as lost capacity.🤖 Generated with Claude Code