[Japabu]

Draft. The middle of a stack: #988 (landed) → this → #994 (the boot handoff). It is based on main, and #994 rebases onto it.

Problem

SYS_DLOPEN opened its path through the kernel's VFS, so no library on a file server could load. That covered /apps packages, and /system once #994 moves it onto the file protocol. The kernel also kept a cross-process cache of every library it loaded; nothing was ever evicted, and it was keyed by the spelling of a path. This is step 5 of the small-kernel track, whose exit is "SYS_DLOPEN takes no path".

Resulting behaviour

• ABI:
  • SYS_DLOPEN takes a DlopenArgs: the name the module goes by, an image handle carrying MAP and the image's length, and init_out.
  • The kernel opens nothing by the name. The name is what SYS_QUERY_MODULES and dladdr answer, and a second load under the same name answers the module already loaded without running its initialisers again.
  • toyos_abi::syscall::dl_open(name, image, len) runs the module's .init_array.
  • dl_open_args makes the bare call.
• Kernel:
  • sys_dlopen loads from the SharedImage the image handle names.
  • Deleted: kernel/src/elf/cache.rs (the cache, its byte budget and its stale-file refusal), vfs::BackingId / open_backing_identified, and LibMemory's shared arm.
  • A LoadedLib now owns its one allocation, so each process holds its own library pages.
• SDK:
  • toyos::files::read_image(path) reads a file into an image wherever it is served, on a file server or in the kernel's /tmp.
  • toyos::files::dl_open(path) loads a library under its absolute path.
  • std's read_image is now that same helper.
• libc:
  • dlopen goes through toyos::files::dl_open and sets errno on a refusal.
  • posix_spawn of a program on a file server reads it into an image, the way std's Command does (ENOEXEC for an empty file).
• Fork: ToyOSOrg/rust_libloading toyos-sdk-0.12 at da187608c. Its ToyOS backend calls toyos::files::dl_open, and the fork now depends on toyos by version range. Pinned in tests/guest/rust-tests/Cargo.lock. Nothing else in the tree or the forks calls dl_open.

Decisions

• Orchestrator, under the memory design: delete the kernel's cross-process library cache and let each process hold its own library pages. The design's stage 2 deletes LibMemory anyway, and sharing comes back through memory objects in its stages 4 and 6. The backing-memory implementer (agents/backing) is aware that this PR changes LibMemory first.
• Mine:
  • The module's name is the caller's, like a spawn's argv[0]. Deduplication by that name stays in the kernel, which keeps a dlopen loop from growing the address space; the cost of a misleading name falls on the caller alone.
  • read_image and dl_open live in toyos::files, the client that std and libc already share (files: one client of the file servers, in toyos, for std and libc alike #988).

Checks (head 6ae79ccaa, base 5e01f8b87)

check	exit
cargo run -- --ci host (87 steps)	0
cargo run -- --build-only (x86_64)	0
QEMU libc_children	0
QEMU, uncommitted scratch machine test (patch in the evidence): dlopen_dedup, abuse_dlopen_ledger, std_tls_multi_crate, std_tls_dlopen, std_unwind_so, the C cases 204_dl_iterate_phdr and 207_libc_names on the shipping kernel, and abuse_kernel_addr on the test kernel	0, all eight exit 0

The scratch run built the guest tests with libloading patched to a local worktree of the fork. That worktree had the same tree as the pushed da187608c; only the commit was made afterwards.

The dlopen guest tests run only in metal's shared boot, which is why the QEMU evidence is a scratch test rather than a committed row.

Evidence at 9c1851478: the host suite, the build, libc_children and the scratch dlopen run, with the patch that run used.

Risks

• Libraries are no longer shared between processes. Nothing shipped dlopens today.

🤖 Generated with Claude Code

[Japabu]

Review of #998, round 1 (head 6ae79ccaa, base 5e01f8b87)

I read the commit and the whole diff, along with:

• sys_dlopen and load_shared_lib at head, and SharedImage;
• the dispatch arm and toyos::files;
• libc's dlopen and posix_spawn;
• the fork compare bad8d494d…da187608c on ToyOSOrg/rust_libloading;
• both lockfiles;
• the evidence at 9c1851478/head-6ae79ccaa.

I ran nothing.

Architecture

This is the right shape, and it is step 5's exit:

• SYS_DLOPEN takes no path.
• The kernel loads the library from a memory object the caller read, through the same SharedImage that spawn uses.
• The never-evicting, path-keyed cross-process cache goes, along with its budget, its BackingId identity, LibMemory::Shared and that arm's extra shootdown.
• One helper, toyos::files::read_image / dl_open, serves std, libc's dlopen, libc's posix_spawn and the libloading fork.

The diff is +197/−599. Most of the deletion is production code, and that is the point of the change.

Losing cross-process library sharing is the orchestrator's decision under the approved memory design, and the track record says that sharing returns through memory objects. Nothing shipped dlopens, so I accept the loss.

Authority.

• The kernel opens nothing by name.
• The image handle must carry MAP, and SharedImage::over refuses device memory.
• Which libraries a process can load is now decided entirely by the file grants in its own namespace. Before this change, SYS_DLOPEN read any path the kernel VFS served, ambiently. Now the per-program views apply to dlopen.
• The name carries no authority. It is recorded per process and answered only to that same process (SYS_QUERY_MODULES, dedup), so a misleading name misleads only its caller.

Dedup semantics.

• They are unchanged in substance. Dedup is per process, keyed on the lexically normalised absolute spelling: files::absolute resolves . and .. against the cwd and follows no symlink, as the kernel's resolve_absolute did.
• The name is also the path that read_image reads, so the name and the bytes agree for every SDK caller.
• The re-check under the registering lock (vm.rs:341-352) still makes concurrent loads of one name a single module.
• A raw caller can pair a name with different bytes. The second load then answers the first module, which is documented and affects only that caller.

Fork pin.

• da187608c is exactly one commit ahead of the old pin bad8d494d: a toyos >=0.18, <1 target dependency, plus toyos::files::dl_open in place of toyos_abi::syscall::dl_open.
• That is upstream-quality and minimal.
• The fork branch head equals the pin, and tests/guest/rust-tests/Cargo.lock carries it.
• [patch.crates-io] toyos maps the version range to sdk/toyos (0.18.0).
• The root workspace uses crates.io libloading for hosts only, and nothing else calls dl_open.
• The fork change and its consumer are therefore coherent.

Loading from a caller-owned image. load_shared_lib still parses the layout from one header read. It then validates every later structure against its own copy, using checked slices, and the section-header count it reads a second time is clamped. A caller rewriting its object mid-load is the same exposure that tmpfs-backed loads already had, and it reaches only the caller's own module.

Findings

BLOCKER

1. sdk/libc/src/errcode.rs:75-76: the new of_image was inserted between of_files's doc comment and of_files itself.

   • of_image now carries two doc lines, and the first is false for it ("The errno a refusal of toyos::files is answered with").
   • of_files has lost its doc.
   • Move the line back onto of_files.

2. Comments and records that still describe the deleted cache:

   • kernel/src/elf/mod.rs:305-306: load_shared_lib "returning the module plus its 2 MiB-aligned writable window (must match what rw_alloc later covers)". It now returns only the LoadedLib, and rw_alloc no longer exists.
   • kernel/src/invalidation.rs:5-6: "Dlopen (a Shared window or rollback unmap)". Only the rollback remains.
   • tests/guest/rust-tests/src/bin/abuse_elf_segments.rs:247-249: "once the module is cached the write lands in a smaller private allocation". No module is cached any more. Keep the bounds-check rationale without the cache.
   • issues/dlopen-write-at-is-not-the-sole-writer-of-a-mapped-module.md:18-20 still argues from the Shared module's rw_alloc. Restate it for the single PageAlloc; its exit is unchanged.
   • issues/the-capability-end-state-is-twelve-answers.md:155-163 lists SYS_DLOPEN among the calls that reach past the process "over the same paths", as what "the tree now enforces". That is no longer true: dlopen reaches only what the caller's namespace serves it.

3. The evidence does not cover the ELF refusal tests that this load path changes, and the claim about the fork source is unverified.

   • The load error changed from Unknown to InvalidArgument, and the backing is now a SharedImage.
   • The tests that exercise exactly this path were not run: abuse_elf_loader and abuse_elf_segments, which feed malformed .so images through libloading::Library::new, and std_tls_cranelift. All three are libloading consumers.
   • The PR says the scratch build used libloading patched to a local worktree identical to da187608c. The only line about that patch in dl-scratch.log is warning: patch libloading … was not used in the crate graph. That warning comes from the host toyos-build graph, and the guest crate's build output is not in the log. So the log establishes neither which libloading the guests used nor that the patch was applied.
   • The fork is now pushed, so the cheapest sufficient check is one scratch run with no patch, using exactly what the lockfile pins. Run the existing eight, plus abuse_elf_loader, abuse_elf_segments and std_tls_cranelift.
   • Running a scratch QEMU test is otherwise adequate for this PR. These tests normally run only in metal's shared boot, and what they establish (loader refusals, dedup identity, TLS module ids, unwinding across a module) is ordering and contents rather than timing, which QEMU can establish.

SUGGESTION

• toyos::files::dl_open reads the whole library into a fresh image before the kernel looks up the name. A dlopen loop of a name already loaded therefore pays a full file read, plus a 2 MiB object, on every call. Before this change, the kernel's dedup ran before it opened anything. A per-process name table in toyos::files checked first, or a kernel lookup by name, would restore that.
• dl_open returns ImageRefused<Error> and reports a kernel refusal of the load as ImageRefused::Read(Error::Kernel(..)). abuse_dlopen_ledger then asserts that the library cap is a "read" failure. A small DlOpenRefused { Image(ImageRefused<Error>), Load(SyscallError) } would say what happened.
• kernel/src/arch/x86_64/paging.rs:512: remap's example of a caller whose mapping is live elsewhere is "sys_dlopen". That was true only of the deleted Shared arm.

SEND BACK for blockers 1–3. All three are small. Item 3 needs one scratch QEMU run.