From 9a6f3927e2eb059a6232630fc1f1a151fb3af0ea Mon Sep 17 00:00:00 2001 From: japabu Date: Thu, 22 Oct 2026 00:03:32 +0200 Subject: [PATCH] issues: self-hosting is one ordered list from today to M4 The owner asked whether everything self-hosting needs is written down. toyos-builds-itself.md held it in five lists in an order it called open: what to build, what other tracks block, what stops M2, what M3 adds and what else is owed. It is now one list in dependency order: each item has its status, the issue that owns it and what blocks it, and the evidence it rests on is below it, with libc's revision for the compiles. The status words say only what is true: "direction set" is the owner's ruling of what an item must do, with its design still engineering's. The memory and socket designs, which are outside the tree and under review, are named as such, and so is the sequencing the socket design proposes for the descriptor work's spawn. Folded in: - C reaches served files (#975, #978, #988); - #970's toolchain package; - the memory direction; - the process track's stages 4 to 7; - socket sharing; - the boot handoff, beside the path; - the database feasibility study's libc findings; - the issues that name this track as their owner, among them pread, libc++'s std::filesystem, and the two that keep a build from reproducing. Each owner ruling of 2026-10-21 is recorded once, in its owning issue: - the signal scope in the process track; - sharing rather than moving, in the socket issue, whose exit described a move; - the database rulings in a new track, split from the planning dispositions made under delegation. Four findings get a home: - unlink-while-open; - libc's uid and gid; - sigaction's false success; - st_nlink 0. Co-Authored-By: Claude Opus 5.5 --- ...nt-and-a-parent-takes-its-children-down.md | 9 + .../a-stream-cannot-move-between-processes.md | 26 ++- issues/an-unlinked-open-file-answers-stale.md | 30 +++ issues/libc-answers-st-nlink-0.md | 18 ++ .../libc-answers-uid-0-for-every-process.md | 25 ++ ...ction-answers-success-and-keeps-nothing.md | 19 ++ issues/no-guest-case-reads-std-filesystem.md | 2 +- ...-all-follows-a-link-swapped-in-mid-walk.md | 4 +- issues/sqlite-and-postgresql-run-on-toyos.md | 65 ++++++ issues/toyos-builds-itself.md | 221 +++++++++--------- 10 files changed, 300 insertions(+), 119 deletions(-) create mode 100644 issues/an-unlinked-open-file-answers-stale.md create mode 100644 issues/libc-answers-st-nlink-0.md create mode 100644 issues/libc-answers-uid-0-for-every-process.md create mode 100644 issues/libc-sigaction-answers-success-and-keeps-nothing.md create mode 100644 issues/sqlite-and-postgresql-run-on-toyos.md diff --git a/issues/a-childs-end-is-an-event-and-a-parent-takes-its-children-down.md b/issues/a-childs-end-is-an-event-and-a-parent-takes-its-children-down.md index 484f5119fd..82004b60f7 100644 --- a/issues/a-childs-end-is-an-event-and-a-parent-takes-its-children-down.md +++ b/issues/a-childs-end-is-an-event-and-a-parent-takes-its-children-down.md @@ -35,6 +35,15 @@ directory is `issues/every-program-sees-only-the-files-it-was-given.md`'s. cleanup without cooperation, run on the causing path in bounded steps. init never dies and stays tiny. +**Ruled** (owner, 2026-10-21): libc's signals are the focused set this track +plans, over native process events: a child's end (stage 4) and an interrupt, +hang-up or terminate request (stage 6), and no POSIX signal subsystem up +front. libc's adaptation stays in userland, and ordinary Rust behaviour is +kept. Further signal behaviour comes when a required program shows a +concrete need, with its semantics and cost reviewed then. An unsupported +signal operation never appears to succeed +(`issues/libc-sigaction-answers-success-and-keeps-nothing.md`). + ## Stages 1. **Done** (#926, stage 1.2 of diff --git a/issues/a-stream-cannot-move-between-processes.md b/issues/a-stream-cannot-move-between-processes.md index c833ad1877..a418f27825 100644 --- a/issues/a-stream-cannot-move-between-processes.md +++ b/issues/a-stream-cannot-move-between-processes.md @@ -22,10 +22,30 @@ starts (`sdk/toyos/src/spawn.rs`, `sdk/toyos/src/fd.rs`'s `copy`, A program that hands a stream to another process, as an inetd-style server does through its child's stdio, fails on ToyOS where it works elsewhere, and one that takes a moved stream back as a `TcpStream`, as socket activation -does, is refused, though root `CLAUDE.md` says existing Rust just works. +does, is refused, though root `CLAUDE.md` says existing Rust just works. What +replaces the refusal is a socket shared by its holders, not moved, as the +ruling below says. + +**Ruled** (owner, 2026-10-21): sharing, inheritance and the surviving holders +are preserved: a socket given to a child stays the parent's too, and the +others keep working when one holder exits or is killed. Neither resetting +every holder's connection when one is interrupted nor a temporary refusal to +give a socket to a child is approved. Nginx's shared listeners establish its +inheritance and lifetime requirements. The ruling covers every socket kind, +so listeners and datagram sockets are shared as streams are. + +Planning's reading of the evidence (2026-10-21): data a dead holder already +consumed need not be recovered. + +The sharing and recovery design is engineering's and under review, outside +the tree. It proposes, and nobody has accepted, that the descriptor work's +spawn from the table (PR 4) lands after its stages for shared streams, +listeners and datagrams, and unconnected C sockets, since every interim rule +for an inheritable socket is rejected or unapproved. Owner: the orchestrator, who stages `issues/netstack-moves-every-byte-through-two-kernel-pipes.md`. -Exit: a stream moved by its handle reads and writes in the receiving process, -and the process that moved it holds no end. +Exit: a stream, a listener and a datagram socket given to a child read, write +and accept in both processes at once, and each keeps working in the survivor +when the other holder exits or is killed. diff --git a/issues/an-unlinked-open-file-answers-stale.md b/issues/an-unlinked-open-file-answers-stale.md new file mode 100644 index 0000000000..d45b4a3814 --- /dev/null +++ b/issues/an-unlinked-open-file-answers-stale.md @@ -0,0 +1,30 @@ +--- +status: open +kind: track +opened: 2026-10-22 +--- + +# An unlinked open file answers Stale + +A file the file server holds open answers `Stale` from the moment its name is +unlinked or renamed over (`system/fileserver/src/volume.rs`, the module +header), and a test asserts it. POSIX's file lives on until its last close or +unmap, and SQLite relies on that: it opens every temporary file and unlinks it +at once, and its last connection unlinks the `-shm` file while it is still +mapped. So any SQLite statement that spills to a temporary file fails today, +cargo's bundled one included. + +**Ruled** (owner, 2026-10-21): +removing the last directory name prevents fresh opens through that name, and +existing handles and mappings keep the same file until their final release. A +new file made at the old path is a different file and never retargets them. +The file server owns the lifetime and the storage's reclaim; the kernel keeps +no filesystem identity or naming. + +**Exit**: a file unlinked while open, and one renamed over, is read and written +through its open descriptor until the last goes, and its blocks are freed +then; a crash leaves the orphans for the next mount to reclaim; each asserted +by the file server's host volume tests, replacing today's `Stale` test +(`system/fileserver/src/data.rs`). A guest test reads a mapping of an +unlinked file once file mappings exist +(`issues/process-memory-is-2-mib-pages-and-that-caps-the-process-count.md`). diff --git a/issues/libc-answers-st-nlink-0.md b/issues/libc-answers-st-nlink-0.md new file mode 100644 index 0000000000..b86951906e --- /dev/null +++ b/issues/libc-answers-st-nlink-0.md @@ -0,0 +1,18 @@ +--- +status: open +kind: defect +opened: 2026-10-22 +--- + +# libc answers st_nlink 0 + +libc's `stat`, `lstat` and `fstat` (`sdk/libc/src/posix_io.rs`, `answered`) +zero the whole `struct stat` and never set `st_nlink`, so every file has no +link. Neither the kernel's `/tmp` nor the file server has hard links, so the +true count is 1. +SQLite checks every database it opens (`verifyDbFile` in `os_unix.c`): a +count of 0 is logged as a file unlinked while open, and one past 1 as a file +with several names. + +**Exit**: a file and a directory answer `st_nlink` 1 on `/tmp`, `/system` and a +served directory, in a C corpus case. diff --git a/issues/libc-answers-uid-0-for-every-process.md b/issues/libc-answers-uid-0-for-every-process.md new file mode 100644 index 0000000000..3eaec98017 --- /dev/null +++ b/issues/libc-answers-uid-0-for-every-process.md @@ -0,0 +1,25 @@ +--- +status: open +kind: defect +opened: 2026-10-22 +--- + +# libc answers uid 0 for every process + +libc's `getuid`, `geteuid`, `getgid` and `getegid` answer 0 +(`sdk/libc/src/misc.rs`), and `stat` answers `st_uid` and `st_gid` 0, so every +C program believes it runs as root. PostgreSQL +refuses to start as root, and sshd and other daemons take root as a right +they hold. + +**Ruled** (owner, 2026-10-21): +libc reports the process's actual ToyOS user identity, through one userland +mapping from the authenticated identity to user and group ids, and never +another invented constant. No Unix root authority enters the kernel: +capabilities remain the authority. Real and effective identity and file +metadata stay coherent, and an unsupported identity change fails honestly. +The identity it maps is `issues/a-user-is-a-home-tree-and-a-login-row.md`'s. + +**Exit**: a C program run by a login reads that user's ids from `getuid`, +`geteuid`, `getgid`, `getegid` and the `st_uid` and `st_gid` of a file it +made, and `setuid` and `setgid` to another id are refused. diff --git a/issues/libc-sigaction-answers-success-and-keeps-nothing.md b/issues/libc-sigaction-answers-success-and-keeps-nothing.md new file mode 100644 index 0000000000..b6204badd7 --- /dev/null +++ b/issues/libc-sigaction-answers-success-and-keeps-nothing.md @@ -0,0 +1,19 @@ +--- +status: open +kind: defect +opened: 2026-10-22 +--- + +# libc's sigaction answers success and keeps nothing + +`signal` and `sigaction` (`sdk/libc/src/misc.rs`) answer success for every +signal but `SIGALRM` and keep nothing: no handler is installed, none runs, and +`oldact` is left as the caller passed it. A program that installs a `SIGINT`, +`SIGTERM` or `SIGCHLD` handler is told it did, and is never called. Which +signals libc delivers, and the ruling that an unsupported one never appears +to succeed, are the process track's +(`issues/a-childs-end-is-an-event-and-a-parent-takes-its-children-down.md`). + +**Exit**: `signal` and `sigaction` refuse a handler for a signal libc does not +deliver with `EINVAL`, and keep and answer what they do support, with a C +corpus case and every in-tree caller checked for the false success. diff --git a/issues/no-guest-case-reads-std-filesystem.md b/issues/no-guest-case-reads-std-filesystem.md index 6298b93d4d..c72bc04d15 100644 --- a/issues/no-guest-case-reads-std-filesystem.md +++ b/issues/no-guest-case-reads-std-filesystem.md @@ -14,7 +14,7 @@ directory by its path (`issues/remove-all-follows-a-link-swapped-in-mid-walk.md` Each of libc's is read back by `tests/corpus/tinycc/207_libc_names.c` and `206_libc_refusals.c`; no C++ program in a guest uses any of it. -Owner: `issues/toyos-builds-itself.md`, M2. +Owner: `issues/toyos-builds-itself.md`, beside its path: no exit there needs it. **Exit**: a guest test lists a directory through `std::filesystem`, and reads a file back through `std::ifstream`. diff --git a/issues/remove-all-follows-a-link-swapped-in-mid-walk.md b/issues/remove-all-follows-a-link-swapped-in-mid-walk.md index dcc4748c42..d4804b3b82 100644 --- a/issues/remove-all-follows-a-link-swapped-in-mid-walk.md +++ b/issues/remove-all-follows-a-link-swapped-in-mid-walk.md @@ -18,8 +18,8 @@ holds a descriptor for each directory and resolves every name against it `AT_REMOVEDIR`), and ToyOS has no such descriptor: `open` refuses every directory, and the kernel resolves every path from the root. -Owner: `issues/toyos-builds-itself.md`, M2, whose clang brought -`std::filesystem` in. +Owner: `issues/toyos-builds-itself.md`, beside its path, since no exit there +needs it; its M2 clang brought `std::filesystem` in. **Exit**: a directory opens as a handle that the kernel resolves names against, libc defines `openat`, `fdopendir` and `unlinkat` over it, and diff --git a/issues/sqlite-and-postgresql-run-on-toyos.md b/issues/sqlite-and-postgresql-run-on-toyos.md new file mode 100644 index 0000000000..5a6f748335 --- /dev/null +++ b/issues/sqlite-and-postgresql-run-on-toyos.md @@ -0,0 +1,65 @@ +--- +status: open +kind: track +opened: 2026-10-22 +--- + +# SQLite and PostgreSQL run on ToyOS + +SQLite, as first-release applications bundle it, and later PostgreSQL and +Nginx run on ToyOS with their own semantics. + +**Ruled** (owner, 2026-10-21; decisions, not implementation claims): + +- **SQLite uses its stock Unix VFS** through ToyOS libc, including the copies + applications bundle; a ToyOS-specific SQLite VFS is not required. The file + server and the SDK supply the native file, locking, mapping and durability + contracts, and libc adapts them in userland; no Unix filesystem policy + enters the kernel. Multiprocess WAL, locking and durable I/O are validated, + not startup alone. +- **An unlinked open file lives on** until its last release: + `issues/an-unlinked-open-file-answers-stale.md`. +- **PostgreSQL starts its processes by `EXEC_BACKEND` over native spawn**, + with the adaptation in PostgreSQL's platform port. No `fork` and no + address-space cloning are added for it. +- **A shared segment is placed by the spawner:** the parent installs the + authorised shared object at a requested address in the child before it + starts. An overlap is refused and reported as a startup failure; rights, + range, protection and accounting are enforced; address randomisation is + unchanged for other mappings. The exact interface is engineering's. +- **Identity:** `issues/libc-answers-uid-0-for-every-process.md`. PostgreSQL's + ownership checks keep their security purpose under native grants. +- **Signals** are the process track's + (`issues/a-childs-end-is-an-event-and-a-parent-takes-its-children-down.md`). + PostgreSQL's own signal table is not approved in its details. +- **Scope:** PostgreSQL and Nginx may arrive after the first release, but + constrain design now; SQLite for first-release applications is unaffected. + +**Delegated** (planning, 2026-10-21, under the owner's delegation; open to +the owner's challenge): + +- **Wake-ups between PostgreSQL's processes** are userland coordination over + the existing wait and wake primitives: no lost wake-up, no busy polling and + no kernel mechanism for PostgreSQL alone. +- **Launch** is the existing launch and service lifecycle: no `system.toml` + row is required, no quit reason is added for a database, and native quit + requests are honoured. +- **No version, connection target or patch-size cap** is a product + requirement; engineering pins versions for feasibility work and reports the + limits. + +**What SQLite needs**: record locks the file server serves +(`issues/std-file-lock-answers-ok-and-locks-nothing.md`), unlink-while-open, +`O_EXCL` on `/tmp` (`issues/create-new-on-a-kernel-path-is-not-exclusive.md`), +`st_nlink` 1 (`issues/libc-answers-st-nlink-0.md`), a sub-second clock +(`issues/userlands-wall-clock-has-whole-seconds-and-an-mtime-has-nanoseconds.md`) +and a directory `fsync`, which this track owns: SQLite syncs a directory after +the unlink that commits a transaction in its default journal mode, and +ignores only a directory it cannot open. WAL also needs coherent shared file +mappings (`issues/process-memory-is-2-mib-pages-and-that-caps-the-process-count.md`). +`issues/toyos-builds-itself.md` orders those cargo's SQLite needs too. + +**Exit**: SQLite's own test harness, built for ToyOS from upstream with the +stock unix VFS, passes its multiprocess WAL, locking and durability tests in a +guest; and PostgreSQL's regression suite passes against a cluster started by +the port. diff --git a/issues/toyos-builds-itself.md b/issues/toyos-builds-itself.md index 68e6b74351..eb7dab5e8f 100644 --- a/issues/toyos-builds-itself.md +++ b/issues/toyos-builds-itself.md @@ -42,117 +42,112 @@ AArch64 one step behind, on `issues/toyos-runs-on-arm64.md`'s track. first, without waiting for packages over HTTPS, and links them statically: "Yes, decoupled and static (Recommended)". -**To build** (2026-10-01), in an order that is open. Where a stage names -libc's state it is `userland/libc` at `15625e0cb`. -- **The `libc` crate gains a ToyOS module**, checked by `ctest` against - `sdk/libc`'s headers. `libc` 0.2.189 has none (`rg -i toyos` over its - `src` matches nothing) and is empty for an OS it does not know - (`src/lib.rs`), and `ToyOSOrg/libc` does not exist (GitHub's API answers - 404). cargo's `curl-sys`, `libssh2-sys` and `libgit2-sys` import its C types - unconditionally. -- **libc gains `select`.** curl 8.21.0, which `curl-sys` 0.4.90 builds, does - not compile without it (`lib/curlx/wait.c`: `#error "We cannot compile - without select() support."`) and waits on its sockets through it: `select.c` - calls `poll` only under `HAVE_POLL`, which `curl-sys` does not define. Stage - 3 of `issues/a-childs-end-is-an-event-and-a-parent-takes-its-children-down.md` - builds none, because a descriptor is a handle and passes `FD_SETSIZE`; what - `select` answers for such a descriptor is open. -- **libc gains what cargo's libraries call**, among them `setvbuf` and - `socketpair`, which it does not define; a file `mmap` and record locks - (`issues/libc-refuses-what-toyos-cannot-yet-answer.md`); paths resolved by - POSIX's rules, which `realpath` answers by the kernel's - (`issues/the-kernel-resolves-a-path-by-rules-of-its-own.md`); and socket - descriptors that are pollable and - can be non-blocking - (`issues/libc-close-of-a-socket-is-unproven-on-a-connected-socket.md`). Read from each - library's source and not run: SQLite's default VFS takes `fcntl` record - locks and keys them on `st_dev` and `st_ino`; libgit2 opens every repository - through `realpath` and reads packs through a file `mmap`; curl and libssh2 - wait on non-blocking sockets, curl's multi handle wakes through a +## From today to M4 + +The one list of what self-hosting still needs, in dependency order: an item +needs only items above it, and names them. An item points at the issue that +holds its facts; where the design is not yet in the tree, the item says so. +Status, as of 2026-10-22: +- **done**; +- **in progress**: built in part, on `main` or in an open pull request; +- **direction set**: the owner ruled what it must do, and its design is + engineering's, not yet written in the tree or not yet reviewed; +- **not started**. + +An owner is an issue under `issues/`, or this track. + +**M2: clang and lld installed and run.** + +| # | Item | Status | Owner | Blocked on | +|---|---|---|---|---| +| 1 | clang and `ld.lld` built for ToyOS, static, run in a guest (`hosted_clang_hello`) | done | this track (`build/hostedclang.rs`) | — | +| 2 | C reaches every file a process holds: descriptors (#975, #978) and served files through `toyos::files` (#988) | done | this track | — | +| 3 | `pkg` installs a toolchain: a streamed archive bounded by its volume's room, and links. #970 installs the clang package (83.8 MB archive, 279 MB installed) on a 1 GiB DATA in QEMU | in progress | `a-package-is-a-directory-under-apps-and-the-installer-is-a-program` | — | +| 4 | Room for a toolchain on DATA, durable. DATA serves `/apps`, `/config`, `/home` and `/state`; the full toolchain is about 1 GB | in progress | `storage-is-layers-and-a-role-is-a-filesystem` | — | +| 5 | **M2 exit**: a `pkg`-installed clang compiles, links and runs `hello.c` (#970's guest row), as `hosted_clang_hello` already does with a clang staged on ROOT | in progress | this track, `toyos-has-no-c-compiler-inside-it` | 2–4 | + +**M3: an LLVM-backed rustc.** + +| # | Item | Status | Owner | Blocked on | +|---|---|---|---|---| +| 6 | Threads mature enough for LLVM | not started | `std-and-libc-drop-the-answer-thread-join-gives` | — | +| 7 | Concurrent reads and writes of one file: `pread` and `pwrite`. Without them LLVM reads a file slice with `lseek` and `read`, which moves an offset other threads share | not started | `libc-has-no-pread-or-pwrite` | — | +| 8 | LLVM, clang and lld built by bootstrap for a ToyOS host, and the libc names they call | not started | `bootstrap-cannot-build-llvm-clang-and-lld-for-a-toyos-host`, `rustc-llvm-cannot-build-for-a-toyos-host`, `libc-lacks-names-llvm-for-a-toyos-host-calls`, `a-toyos-hosted-llvm-is-configured-as-running-on-the-build-machine` | — | +| 9 | A std binary links the C++ runtime | not started | `a-rust-std-binary-cannot-link-the-cxx-runtime` | 8 | +| 10 | A package ships its own libraries (`librustc_driver`); moot for M2's static PIEs | not started | `a-package-cannot-ship-its-own-libraries` | 3 | +| 11 | **M3 exit**: the hosted rustc, which runs LLVM on several threads, builds and runs a program in the guest, linking with the guest's `ld.lld` | not started | `nothing-builds-the-toyos-hosted-rustc`, `the-hosted-rustc-names-a-linker-toyos-does-not-have`, `the-hosted-rustcs-stage2-carries-seventeen-proc-macro-libraries-no-image-needs` | 5–10 | + +**M4: cargo, rustc and clang build ToyOS in the guest, byte-identical.** + +| # | Item | Status | Owner | Blocked on | +|---|---|---|---|---| +| 12 | Process memory: 4 KiB pages, a ledger, demand paging, one mapping inventory, then private file mappings and coherent shared ones (owner direction, 2026-10-21; the design is out of the tree, under review) | direction set | `process-memory-is-2-mib-pages-and-that-caps-the-process-count`, `one-mapping-inventory-nobody-can-read`, `anonymous-mmap-is-not-demand-paged`, `nothing-charges-kernel-memory-to-a-process` | engineering: the design, with a wait and progress design for mapped pages without an automatic hang restart (owner, 2026-10-21) | +| 13 | Record locks the file server serves, and std's and cargo's locks on them | direction set | `std-file-lock-answers-ok-and-locks-nothing`, `libc-refuses-what-toyos-cannot-yet-answer` | engineering: the lock design, part of 12's | +| 14 | An unlinked open file lives until its last close or unmap | direction set | `an-unlinked-open-file-answers-stale` | 12 for the mapping | +| 15 | An exclusive create on `/tmp` (`O_CREAT\|O_EXCL`, `create_new`) | not started | `create-new-on-a-kernel-path-is-not-exclusive` | — | +| 16 | libc tells the truth: `sigaction` refuses what it does not keep, `st_nlink` is 1, and the user and group ids are the process's own | not started | `libc-sigaction-answers-success-and-keeps-nothing`, `libc-answers-st-nlink-0`, `libc-answers-uid-0-for-every-process` | — | +| 17 | libc gains `select`, `setvbuf` and `socketpair`. A descriptor is a small table number since #978, so `FD_SETSIZE` holds | not started | this track | — | +| 18 | The `libc` crate gains a ToyOS module, checked by `ctest` against `sdk/libc`'s headers | not started | this track | 17 | +| 19 | Children: `SIGCHLD`, sessions, a quit notice and Ctrl+C, the process track's stages 4–7 | not started | `a-childs-end-is-an-event-and-a-parent-takes-its-children-down` | — | +| 20 | Sockets shared by their holders and inherited by a child, which curl, libssh2 and every inheriting server need | direction set | `a-stream-cannot-move-between-processes`, `libc-close-of-a-socket-is-unproven-on-a-connected-socket` | engineering: the sharing and recovery design, out of the tree, under review | +| 21 | cargo's eight C libraries cross-built: curl, libgit2, libssh2, OpenSSL, SQLite, nghttp2, zlib, blake3; five stopped on headers, types and functions libc lacked at `15625e0cb` | not started | this track | 17, 18 | +| 22 | cargo's TLS: OpenSSL through its Perl `Configure`, or curl's rustls backend on `ToyOSOrg/ring`. A build from vendored sources needs neither, and neither needs the `toyos-crypto` track | not started, choice open | this track, `the-internet-clients-work-unchanged` | 21; HTTPS on `the-lan-is-not-yet-production-grade` | +| 23 | cargo at run time: its SQLite and its locks, sockets and children | not started | this track, `sqlite-and-postgresql-run-on-toyos` | 13–16, 19–21 | +| 24 | The tools a build runs: Perl and Python started by spawn, brush as `sh`, uutils, make, CMake with libuv on spawn, awk, and a script started by its interpreter line | not started | `ninja-runs-every-command-through-a-bin-sh-toyos-does-not-have`, `nothing-launches-a-script-by-its-interpreter-line` | 19 for the C tools | +| 25 | git in the guest | not started | this track | 21 | +| 26 | The toolchain as `pkg` packages, none of it in the image. #970 makes the clang package; how the rest splits, and how a self-host test's guest reaches them, is open | in progress | this track | 3, 10 | +| 27 | Storage durable and fast enough for an LLVM build tree, unmeasured | not started | `storage-is-layers-and-a-role-is-a-filesystem`, `an-fsync-of-a-served-file-syncs-its-whole-volume` | 4 | +| 28 | A build reproduces: an image's GUIDs from the commit, and one checkout building one kernel | not started | `an-images-guids-are-drawn-from-the-hosts-random-source`, `one-checkout-of-one-tree-builds-two-kernels` | — | +| 29 | **M4 exit**: the guest builds a userland program and then the kernel, and the hashes equal the host build's of the same commit | not started | this track | 11–28 | + +What the items rest on. The compiles were made against libc at `15625e0cb` +(2026-10-01), before #975, #978 and #988 changed it: +- **3, 26, measured:** the old `pkg` inflated a whole archive in memory under + `MAX_INFLATED`, 256 MiB, and refused every link (`system/pkg/src/main.rs`, + `archive.rs`), which #970 replaces. Measured on the Linux-host release + `toolchain-linux-x86_64-48dd24f826263d6c`, stripped: `librustc_driver` + 184.7 MB, clang 122.8 MB, lld 78.6 MB, and 181.1 MB of + `x86_64-unknown-toyos` libraries. +- **10, read:** rustc's launcher names `librustc_driver` as needed + (`llvm-readobj --needed-libs`). +- **17, read:** curl 8.21.0, which `curl-sys` 0.4.90 builds, does not compile + without `select` (`lib/curlx/wait.c`), and waits through it, since `curl-sys` + does not define `HAVE_POLL`. curl's multi handle wakes through a `socketpair`, and its TLS key log sets its buffering through `setvbuf`. -- **cargo's eight C libraries are cross-built for ToyOS**: curl, libgit2, - libssh2, OpenSSL, SQLite, nghttp2, zlib and blake3. Compiled and linked - with the toolchain's clang against that libc's C sysroot, the sources of - nghttp2, zlib and blake3 build with nothing undefined, and the other five - stop on headers, types or functions libc lacks. `openssl-src` 300.6.1 knows - no ToyOS target and refuses one it does not know (`src/lib.rs`), and - `openssl-sys` is a `cfg(unix)` dependency of `curl-sys` and `libssh2-sys`, - which ToyOS is not. Open until M4: whether cargo's OpenSSL is built for - ToyOS through its Perl `Configure`, or cargo takes curl's rustls backend - there. -- **The tools a self-build runs are built for ToyOS**: Perl and Python, ported - to start a child by spawn and never by fork; brush as the POSIX `sh` - (`issues/ninja-runs-every-command-through-a-bin-sh-toyos-does-not-have.md`); - uutils; make; CMake, its libuv ported to spawn; and awk. Those written in C - or C++ wait on stage 4 of the child-process track. A search of Perl 5.44.0's - sources for `posix_spawn` matches nothing. Open: which Python, which make and - which awk. -- **`pkg` installs a toolchain**: an archive past 256 MiB, and links. It - inflates a whole archive in memory under `MAX_INFLATED`, 256 MiB, and refuses - every link (`system/pkg/src/main.rs`, `archive.rs`). A toolchain's size is - measured on a proxy, the Linux-host release - `toolchain-linux-x86_64-48dd24f826263d6c`: stripped with `llvm-objcopy - --strip-all`, its `librustc_driver` is 184.7 MB, its clang 122.8 MB and its - lld 78.6 MB, beside 181.1 MB of `x86_64-unknown-toyos` libraries. And the - loader finds a package's own libraries - (`issues/a-package-cannot-ship-its-own-libraries.md`): rustc's - launcher names `librustc_driver` as needed (`llvm-readobj --needed-libs`). -- **The toolchain is `pkg` packages, and the image carries none of it.** Open: - how it is split into packages, and how a self-host test's guest reaches - them. - -**Blocked on other tracks.** M2 needs room for about a gigabyte of toolchain, -and threads and `mmap` mature enough for LLVM -(`issues/std-and-libc-drop-the-answer-thread-join-gives.md`). Packages over -HTTPS -(`issues/a-package-is-a-directory-under-apps-and-the-installer-is-a-program.md`) -wait on the network stack under them -(`issues/the-lan-is-not-yet-production-grade.md`, -`issues/the-internet-clients-work-unchanged.md`), and M2 does not. -M4 also needs the rest of the child-process track -(`issues/a-childs-end-is-an-event-and-a-parent-takes-its-children-down.md`). M4 needs git in the guest, storage durable and fast -enough for an LLVM build tree -(`issues/storage-is-layers-and-a-role-is-a-filesystem.md`), and -memory beyond what 2 MiB process pages allow -(`issues/process-memory-is-2-mib-pages-and-that-caps-the-process-count.md`). - -**No linker runs inside ToyOS** (owner, 2026-10-04: "Just get rid of toyos -ld"). `/system/bin/toyos-ld` went with its crate, its `[programs]` row and its -host tests, and no image ships a linker: `ld.lld` links inside ToyOS only -where `hosted_clang_hello` stages it, and -the ToyOS-hosted rustc names no linker the guest has -(`issues/the-hosted-rustc-names-a-linker-toyos-does-not-have.md`). *Exit*: the -commit that lands M2's compile-and-run test, which links inside the guest. - -**What stops M2: the install.** `cargo run -- --hosted-clang` makes a clang -and an `ld.lld` for `x86_64-unknown-toyos`, static PIEs that name no library -(`build/hostedclang.rs`): 147.5 MB and 85.6 MB, 121.9 MB and 70.9 MB stripped. -Both run in a guest: `hosted_clang_hello` (`tests/common/hostedclang.rs`, run -on demand) stages them on ROOT, compiles `hello.c` with `clang -c`, has the -driver link it, `clang hello.c`, which starts LLD as a child by libc's -`posix_spawn` and waits for it by `wait4`, runs the program, and both products -are the host's bytes. -- Install: `pkg` installs neither (**To build**, "`pkg` installs a toolchain"). -- Run past `hello.c`: what `issues/libc-refuses-what-toyos-cannot-yet-answer.md` - lists, and `issues/libc-has-no-pread-or-pwrite.md`. - -**What M3 adds: a rustc that carries that LLVM**, after all of M2's. -- Build: `issues/rustc-llvm-cannot-build-for-a-toyos-host.md`, and - `issues/bootstrap-cannot-build-llvm-clang-and-lld-for-a-toyos-host.md`, - whose build of clang and lld replaces M2's. -- Link: `issues/a-rust-std-binary-cannot-link-the-cxx-runtime.md`. -- Test: `issues/nothing-builds-the-toyos-hosted-rustc.md`. - -M3's exit then waits on a linker in the guest -(`issues/the-hosted-rustc-names-a-linker-toyos-does-not-have.md`). - -**Also owed, by milestone.** -- M3: the host triple the ToyOS-hosted LLVM bootstrap builds records - (`issues/a-toyos-hosted-llvm-is-configured-as-running-on-the-build-machine.md`, - whose other half, the configure inside ToyOS, is M5's). -- M3: `issues/the-hosted-rustcs-stage2-carries-seventeen-proc-macro-libraries-no-image-needs.md`. -- M4: cargo's file locks, - `issues/std-file-lock-answers-ok-and-locks-nothing.md`. -- M5: `issues/n2-does-not-compile-for-toyos.md`. -- The tools of "To build", the first programs that start a script by its - path: `issues/nothing-launches-a-script-by-its-interpreter-line.md`. +- **18, read:** `libc` 0.2.189 has no ToyOS module (`rg -i toyos` over `src`) + and is empty for an OS it does not know; `ToyOSOrg/libc` does not exist. + cargo's `curl-sys`, `libssh2-sys` and `libgit2-sys` import its C types + unconditionally. +- **21, compiled at `15625e0cb`:** nghttp2, zlib and blake3 build with nothing + undefined; curl, libgit2, libssh2, OpenSSL and SQLite stop on headers, types + or functions libc lacked. +- **21 and 23, read:** SQLite's unix VFS takes `fcntl` record locks keyed on + `st_dev` and `st_ino`; libgit2 opens every repository through `realpath` and + reads packs through a file `mmap`; curl and libssh2 wait on non-blocking + sockets. `openssl-src` 300.6.1 refuses a target it does not know + (`src/lib.rs`), and `openssl-sys` is a `cfg(unix)` dependency of `curl-sys` + and `libssh2-sys`, which ToyOS is not. That cargo bundles SQLite is read from + its dependencies, not run. +- **24, read:** Perl 5.44.0's sources never call `posix_spawn`. Which Python, + which make and which awk is open. + +Beside the path, and needed by none of its exits: +- **The boot handoff** (owner, 2026-10-21) removes the kernel's ROOT mount: the + loader hands the supervisor and the system file server over as signed + payloads, and `/system` goes onto the file protocol + (`the-kernel-is-small-interrupts-post-and-threads-wait`, step 9). +- **Paths on `/tmp` resolve by the kernel's rules** + (`the-kernel-resolves-a-path-by-rules-of-its-own`); a served path resolves + by POSIX's (#988). +- **libc++'s `std::filesystem`** is proven in no guest, and its `remove_all` + walks by path where a directory handle (`openat`, `unlinkat`) would not + follow a link swapped in (`no-guest-case-reads-std-filesystem`, + `remove-all-follows-a-link-swapped-in-mid-walk`). clang, lld and rustc + call neither, so no exit waits on them; a C++ program that does is + affected. +- **M5:** `n2-does-not-compile-for-toyos`, and Python and CMake for ToyOS. +- Every AArch64 stage lands one step behind x86-64, on `toyos-runs-on-arm64`. + +**No image ships a linker** (owner, 2026-10-04: "Just get rid of toyos ld"). +Until item 5 lands, `ld.lld` links inside ToyOS only where `hosted_clang_hello` +stages it.