Skip to content

issues: self-hosting is one ordered list from today to M4 - #991

Merged
Japabu merged 1 commit into
mainfrom
wt/toyos-selfhost-list
Oct 21, 2026
Merged

Japabu merged 1 commit into
mainfrom
wt/toyos-selfhost-list

Conversation

@Japabu

@Japabu Japabu commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

An issues-only change. It links one issue record that #988 added: an-fsync-of-a-served-file-syncs-its-whole-volume.

Problem

The owner asked whether everything self-hosting needs is written down. issues/toyos-builds-itself.md held it in five lists whose order it called open, and the decisions and designs of 2026-10-21 were in none of them.

Result

  • One list from today to M4, in dependency order. Each of its 29 items has:

    • a status: done, in progress, direction set, or not started. "Direction set" means the owner ruled what the item must do and its design is still engineering's.
    • the issue that owns it, or this track;
    • what blocks it.

    The evidence behind the items follows the table, with libc's revision (15625e0cb, 2026-10-01) for the compiles and each claim marked as measured, compiled or read.

  • Folded in:

  • Each owner ruling of 2026-10-21 is recorded once, in its owning issue:

    • the signal scope, in a-childs-end-is-an-event-and-a-parent-takes-its-children-down;
    • sharing rather than moving, in a-stream-cannot-move-between-processes, whose exit described a move;
    • the database rulings, in the new sqlite-and-postgresql-run-on-toyos, split from the planning dispositions made under delegation.

    Only in-tree records are cited.

  • New homes:

    • an-unlinked-open-file-answers-stale;
    • libc-answers-uid-0-for-every-process (user and group ids);
    • libc-sigaction-answers-success-and-keeps-nothing;
    • libc-answers-st-nlink-0.

Checks

None run: the change touches issues/ only. Every slug the list names exists in this tree, or in #988 for an-fsync-….

Risks

🤖 Generated with Claude Code

@Japabu

Japabu commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator Author

Review of #991, round 1 (head 9030e4c90, base 2661b9e6e)

Read: the commit; the full text of all six files; issues/README.md; every linked slug on origin/main and on #988's head; the libc, pkg and fileserver code the items cite; #970's description; the owner rulings and the memory and socket design texts outside the tree. No build was run, since the change touches issues/ only.

Architecture

Replacing five unordered lists with one dependency-ordered table is the right shape, and the five new records give homes to real defects. I verified these against origin/main:

  • misc.rs:26-35 returns 0 for getuid, geteuid, getgid and getegid.
  • misc.rs:144-161 returns success from signal/sigaction for every signal except SIGALRM, and leaves oldact unwritten.
  • posix_io.rs:195 answered zeroes the whole struct and never sets st_nlink.
  • volume.rs's header documents Stale, and data.rs:879 asserts it.
  • MAX_INFLATED is in pkg/src/main.rs:24.
  • libc has no select, setvbuf or socketpair.
  • The descriptor table numbers descriptors lowest-free (sdk/toyos/src/fd.rs).

Every linked slug exists on main or in #988 (an-fsync-of-a-served-file-syncs-its-whole-volume). The change is +238 −113, all of it in issue text. The blockers fall into four groups:

  • the status vocabulary is applied to designs and rulings that do not meet it;
  • some of the ordering is wrong;
  • the "one list" leaves out issues that name this track as their owner;
  • owner rulings are cited from documents outside the tree, or given more weight than the owner did.

Blockers

  1. issues/toyos-builds-itself.md:50-51, 79-80, 81, 87 — "designed" is defined as "a design the owner accepted", but the items marked designed do not meet that definition.

    • Items 11 and 12 rest on the memory design (revision 10), which calls itself "a proposal: nothing in it is implemented or measured". The owner accepted a direction and answered questions. His §N note says it is "not a completed API". Its §9.5(b) T_stall restart was rejected.
    • Item 19's revision 3 is not accepted either. Neither connection-reset option nor the interim restriction was approved, and the recovery design is unfinished.
    • Item 13 is a semantic ruling, and no design for it exists.

    This presents agent proposals as the owner's. Fix: give the status a truthful name and definition (for example "direction set: the owner's ruling, design in progress"), or demote these items to not started, citing the ruling.

  2. toyos-builds-itself.md:79, 87 — "owner pending" is false on both items.

    • On memory, the latest checkpoint says the owner has answered 21/22/24 and that engineering should "produce a progress/wait design consistent with no automatic hang restart".
    • On sockets: "Complete the recovery/readiness design … Return a demonstrated conflict, not the same preference question."

    In both cases the work is engineering's, and nothing waits on a decision from the owner. Marking them owner-pending tells agents to wait. Fix: drop the mark and state the owed design as the blocker.

  3. toyos-builds-itself.md:79, 87; an-unlinked-open-file-answers-stale.md:17; libc-answers-uid-0-for-every-process.md:14; sqlite-and-postgresql-run-on-toyos.md:11, 13-14 — citations point at documents that are not in the tree.

    • "questions 13, 15–17 and 21–24", "question 18" and "the database feasibility study's question 2 / 6 / eight questions" all number entries in a local, uncommitted file and an off-tree study. issues/README.md requires a pointer that something can check, and these point at nothing.
    • The range "21–24" also takes in 23, which is the crypto question rather than a memory one.
    • requirements/product.md#additional-native-application-compatibility does not exist on origin/main: git ls-tree origin/main requirements/ is empty, and no open PR adds it.

    Fix: keep the date and the ruling's content, and drop the numbers and the requirements/ path, or land that file first.

  4. sqlite-and-postgresql-run-on-toyos.md:13-46 — the list headed "Ruled (owner …)" mixes owner rulings with delegated planning decisions and with things nobody approved.

    • The wake-up bullet (28.5), the launch bullet (28.7) and the version/connection/patch half of scope (28.8) are the planning assistant's delegated dispositions, not the owner's.
    • "PostgreSQL's port handles its own SIGUSR1, SIGUSR2, SIGURG and SIGQUIT" contradicts the owner's words: "not blanket approval of all details of its proposed signal table."
    • "There is no ToyOS VFS" is stronger than "do not require a ToyOS-specific SQLite VFS".

    Fix: split the list into owner rulings and delegated decisions, and drop or soften the last two statements.

  5. toyos-builds-itself.md:48-49 claims "Each fact lives in the issue an item links", which is false for items 11, 12, 18 and 19.

    • None of the four memory slugs records the 4 KiB ledger, demand paging, stage L or stage 7. process-memory-is-2-mib-pages… still says "never killed at a fault".
    • std-file-lock… says nothing about file-server record locks.
    • a-stream-cannot-move-between-processes has the exit "the process that moved it holds no end". That is a move, and it contradicts the sharing that item 19 asserts and the owner required.
    • The owner's signal scope for 2026-10-21 now appears in three places, none of them its owner: item 18, libc-sigaction-…:15-18 and sqlite-…:39-43. a-childs-end-is-an-event-… does not carry it.

    Fix: record each ruling once, in its owning issue, and point to it from the others. The minimum is the socket issue's exit and the process track's signal scope. Otherwise, drop the claim for the items whose design is outside the tree. These records are directly affected by what this PR folds in. That is not unrelated cleanup.

  6. Dependency order (toyos-builds-itself.md:47-48: "an item needs the items above it that it names").

    • Item 5, in M2, is blocked on item 11, in M4, which is below it. Item 6, the M2 exit, does not name 5, so either 5 is not an M2 item or 6 is missing a blocker.
    • Item 13's exit reads and writes "through its mapping" (an-unlinked-open-file-answers-stale.md:24-25), yet its blocker is "—". It needs at least item 11's file mappings.
    • Item 20 says the five libraries "stop on what 12–17 add". A cross-build stops on missing headers, types and functions (16, 17). It does not stop on unlink lifetime (13), record-lock semantics (12) or st_nlink (15). The old text said "headers, types or functions libc lacks".
  7. The statuses of items 3, 4 and 24 contradict pkg streams a package into /apps within its volume's room, and packages the toolchain #970.

    So item 3 is in progress, not "not started". Item 24 is partly in progress. Item 4's "a toolchain's size there is unmeasured" is contradicted by that evidence. Line 62's "DATA serves /apps and /home" also leaves out /config and /state (lib/manifest/src/lib.rs:115).

  8. The "one list" is incomplete, against the PR's claim that "nothing beside the path … is lost".

    • libc-has-no-pread-or-pwrite was in the old "Run past hello.c" and is gone. LLVM reads file slices through the shared offset.

    • These issues name this track as their owner and appear nowhere:

      • no-guest-case-reads-std-filesystem (M2)
      • remove-all-follows-a-link-swapped-in-mid-walk (M2)
      • toyos-has-no-c-compiler-inside-it (exits on M2)
      • an-images-guids-are-drawn-from-the-hosts-random-source (M4)
      • one-checkout-of-one-tree-builds-two-kernels (M4)

      The last two are byte-identity prerequisites for item 26.

    • sqlite-…:48-54 says SQLite's needs are "each owned elsewhere" and ordered by this list. But the directory fsync and the sub-second clock (userlands-wall-clock-has-whole-seconds-and-an-mtime-has-nanoseconds) have neither a named owner nor a row.

    • The old item 24 had an open question, "how a self-host test's guest reaches them", which was dropped.

  9. toyos-builds-itself.md:96, 104-110 — evidence.

    Fix: restore the revision anchor and say which claims were measured and which were read from source.

  10. toyos-builds-itself.md:60 — the row breaks the table's own rules. "done once files: one client of the file servers, in toyos, for std and libc alike #988 lands" is not one of the four statuses, and "the descriptor work" is neither an issue nor "this track". Because the PR lands after files: one client of the file servers, in toyos, for std and libc alike #988, this should read "done", with its owner named or the row folded into the preface.

Suggestions

  • toyos-builds-itself.md:123 — wt/toyos-boothandoff is a local-only branch with no commits beyond main. Name the owning issue rather than a branch.
  • toyos-builds-itself.md:131-132 — "No linker runs inside ToyOS yet", followed by "ld.lld links inside ToyOS where hosted_clang_hello stages it", reads as a contradiction. "No image ships a linker" says what is meant.
  • an-unlinked-open-file-answers-stale.md:26-27 — "A guest test asserts all three" prescribes a guest test for fileserver lifetime, reclaim and orphan recovery. A host volume test can establish those, as data.rs:879 does today. Keep a guest test only for the mapping.
  • libc-answers-st-nlink-0.md:11 — "Neither server" is ambiguous, since /tmp is the kernel's. Say "neither the kernel's /tmp nor the file server has hard links".
  • libc-answers-uid-0-for-every-process.md — getgid and getegid also return 0, and the ruling covers group ids. Put them in the exit.
  • "Found by the database feasibility study" (three files) points at a document nobody can read. Delete it; the commit can carry the provenance.

PR description (the orchestrator's to correct)

SEND BACK

@Japabu
Japabu force-pushed the wt/toyos-selfhost-list branch from 9030e4c to 5709ee1 Compare October 21, 2026 22:43
@Japabu

Japabu commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator Author

Round 2 (5709ee14d) answers review r1.

  1. Status words: a new status, "direction set", is the owner's ruling of what an item must do, with its design still engineering's. It applies to the memory, lock, unlink and socket items (13, 14, 15, 21). Each of them names its design as out of the tree and under review, and none claims "designed".
  2. No "owner pending": items 13 and 21 now name the engineering design they wait on. Item 13's design includes a wait and progress design for mapped pages without an automatic hang restart.
  3. Citations: no question numbers and no requirements/ path remain. Rulings are cited as "owner, 2026-10-21".
  4. Database track: split into owner rulings and planning dispositions made under delegation (28.5, 28.7 and the version/connection/cap half of 28.8). It now says "a ToyOS-specific SQLite VFS is not required", and PostgreSQL's signal table is "not approved in its details".
  5. Each ruling in its owning issue:
    • the signal scope is in the process track, and the sigaction and database records point at it;
    • the socket issue records sharing, its exit now asks for sharing and survivors rather than a move, and the design's proposed sequencing (PR 4 after the sharing stages) is recorded as a proposal;
    • the list no longer claims that every fact lives in the linked issue: the memory and socket items say their design is out of the tree.
  6. Order and blockers:
    • file mmap left M2's item, because LLVM reads a file it cannot map; the M2 exit names its blockers;
    • unlink-while-open names 13 for mappings;
    • the cross-build names only 18 and 19, headers and functions;
    • cargo at run time is its own item (24).
  7. pkg streams a package into /apps within its volume's room, and packages the toolchain #970: items 3 and 27 are in progress, item 4 is in progress with pkg streams a package into /apps within its volume's room, and packages the toolchain #970's measurement, and DATA's four directories are named.
  8. Missing issues: libc-has-no-pread-or-pwrite (5), no-guest-case-reads-std-filesystem and remove-all-follows… (7), toyos-has-no-c-compiler-inside-it (8), and an-images-guids… and one-checkout-of-one-tree-builds-two-kernels (29). The database track gives the directory fsync and the sub-second clock their owners. "How a self-host test's guest reaches them" is back in item 27.
  9. Evidence: the 15625e0cb (2026-10-01) anchor is restored, and each claim is marked measured, compiled or read.
  10. Item 2: status "done", owner "this track".

Suggestions taken:

  • the boot handoff names its owning issue rather than a branch;
  • "No image ships a linker";
  • the unlink exit uses host volume tests, with a guest test only for the mapping;
  • the st_nlink wording;
  • gid in the uid record;
  • "Found by …" deleted.

The PR description is corrected: the one slug from #988 is named, and the "nothing is lost" claim is gone.

@Japabu

Japabu commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator Author

Review of #991, round 2 (head 5709ee14d, base 2661b9e6e)

Read: the r1→r2 diff and the full text of all eight changed files at the head; the M2 owning issues (libc-has-no-pread-or-pwrite, std-and-libc-drop-the-answer-thread-join-gives, no-guest-case-reads-std-filesystem, remove-all-follows-a-link-swapped-in-mid-walk, toyos-has-no-c-compiler-inside-it); tests/guest/rust-tests/src/bin/hosted_clang_hello.rs on origin/main; #988's head 2b122fe85 (it adds no pread); the socket design revision 4; and the owner's text for items 18, 25, 28 and the 2026-10-21 checkpoint. I checked that every slug exists at the head, or in #988 for an-fsync-…. No build was run: the change touches issues/ only.

Architecture

The shape is still right. Round 2 makes the status words honest, keeps owner rulings apart from delegated planning, records each ruling once in its owning issue, and cites only in-tree records. Two problems remain: one is an ordering error, and the other is a planning statement credited to the owner.

r1 blockers

  1. Fixed. "Direction set" is defined truthfully. It applies to 13, 14, 15 and 21, each grounded in an owner ruling: memory direction, 28.1, 28.2 and the socket checkpoint.
  2. Fixed. No item is marked owner-pending, and 13 and 21 name the engineering design they wait on.
  3. Fixed. No question numbers, requirements/ paths or study references remain.
  4. Fixed. The rulings and the delegated dispositions are split as the owner's text splits them (28.5, 28.7, the 28.8 half). The VFS line and the PostgreSQL signal-table line now match the owner's words.
  5. Fixed for the claim, the signal scope and the socket exit. The new socket "Ruled" paragraph introduces N2 below.
  6. Partly fixed. Item 5 no longer depends on a later item, 15 names 13, and 22 names 18 and 19. The rewrite introduces N1 below.
  7. Fixed.
  8. Fixed. Every issue that names this track as its owner is now in the list. Both SQLite needs have owners, and the open question on how the guest reaches the packages is back in item 27.
  9. Fixed.
  10. Fixed.

Blockers

N1. issues/toyos-builds-itself.md:67, 68, 69, 70 — the M2 exit names blockers that it does not need, and item 5 misstates its owning issue.

  • Item 8 is "blocked on 2–6". Item 1 (done) refutes 5 and 6: hosted_clang_hello runs the same clang in the guest today. It compiles hello.c, links it by starting ld.lld as a child, runs it and compares bytes, all with no pread and with today's thread_join handling. The exit differs from that test only by the pkg install and served paths, which are items 2–4.
  • Item 5 says pread/pwrite are what "LLVM reads file slices with". Its owning issue says the opposite: no header declares pread, so LLVM reads with lseek and read, and the defect is the shared offset.
  • Item 7 is in M2, but neither 8 nor any later exit names it. By the list's own rule, nothing needs it. This is the same shape r1 flagged for the old item 5.
  • Consequence: the list tells agents that M2 waits on pread and the join fix. It does not, and those two items are never ordered where a threaded LLVM build actually needs them.
  • Fix:
    • Item 8 names what it needs.
    • Items 5–7 sit where an exit needs them: named from item 12 or 30, or moved beside the path with the reason.
    • Item 5's wording follows its issue.

N2. issues/a-stream-cannot-move-between-processes.md:27-34 — the "Ruled (owner, 2026-10-21)" paragraph credits planning text to the owner.

  • "Data a dead holder already consumed need not be recovered" comes from the planning review of the socket evidence ("application data already consumed by the dead process need not be recovered"). It is not in the owner's direction or the checkpoint.
  • "Nginx's shared listeners make every socket kind a requirement" overstates the owner. The owner wrote that Nginx's shared listeners "establish inheritance/lifetime requirements". The every-kind requirement follows from the checkpoint's "preserve sharing, inheritance and surviving holders", and it is the design's §9 inference.
  • This is r1 blocker 4's class: it relaxes a requirement under the owner's name.
  • Fix: move the recovery clause into a sentence labelled as planning, or drop it. Credit listeners to Nginx, and credit every socket kind to the general sharing ruling.

Suggestions

  • toyos-builds-itself.md:94 — item 22 says "libc lacks", in the present tense. The evidence at line 122 says "lacked" at 15625e0cb. Use the same anchor in the table.
  • sqlite-and-postgresql-run-on-toyos.md:60 — "The order they come in is toyos-builds-itself.md's" is not true of the sub-second clock, which has no row there. Name it in item 24, or narrow the sentence.
  • toyos-builds-itself.md:146-148 — my r1 wording turned a temporary state into a permanent rule that still carries an Exit. One fix: "No image ships a linker (owner, …). Until item 8 lands, ld.lld links inside ToyOS only where hosted_clang_hello stages it."
  • an-unlinked-open-file-answers-stale.md:27-28 — "beside today's Stale test they replace" contradicts itself. Write "replacing today's Stale test".
  • a-stream-cannot-move-between-processes.md — the title and first two paragraphs still describe a move, while the ruling and exit now describe sharing. A rename can wait, but a sentence pointing to the ruling would help.

SEND BACK

@Japabu
Japabu force-pushed the wt/toyos-selfhost-list branch from 5709ee1 to 6961c92 Compare October 21, 2026 22:54
@Japabu

Japabu commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator Author

Round 3 (6961c92db) answers review r2.

N1 (fixed).

  • The M2 exit (now item 5) is blocked on items 2 to 4 alone, since hosted_clang_hello already compiles, links and runs.
  • pread/pwrite (item 7) and thread join (item 6) move to M3, which the M3 exit names: the hosted rustc runs LLVM on several threads.
  • Item 7's wording follows its issue: without pread, LLVM reads a slice with lseek and read, which moves an offset other threads share.
  • libc++'s std::filesystem and remove_all sit beside the path, with the reason: clang, lld and rustc call neither.
  • The items are renumbered from 1 to 29, and the evidence anchors and every blocker with them.

N2 (fixed).

  • The owner's ruling now says only what the owner said: sharing, inheritance and surviving holders are preserved, and neither the reset nor the interim refusal is approved.
  • Listeners are credited to Nginx's inheritance and lifetime requirements, and every socket kind to the general sharing ruling.
  • "Consumed data need not be recovered" is labelled as planning's reading of the evidence.

Suggestions taken:

  • item 21 says what libc lacked at 15625e0cb;
  • the database track's ordering sentence is narrowed;
  • "No image ships a linker", with "until item 5 lands";
  • "replacing today's Stale test";
  • the socket issue's opening points to the sharing ruling. The title keeps its slug's wording, because it still states today's defect.

@Japabu

Japabu commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator Author

Review of #991, round 3 (head 6961c92db, base 2661b9e6e)

Read: the r2→r3 diff, the full self-host track at the head, every issue that names the track as its owner, and the owner's socket text. No build was run, since the change touches issues/ only.

r2 blockers

N1. Fixed.

  • The M2 exit (item 5) is now blocked on 2–4 alone, and it cites hosted_clang_hello as the reason.
  • Threads (item 6) and pread/pwrite (item 7) now sit in M3, where the exit (item 11) names them and gives the reason: the hosted rustc runs LLVM on several threads.
  • Item 7's wording now agrees with its issue, which says LLVM falls back to lseek and read on a shared offset.
  • std::filesystem and remove_all sit beside the path, with the reason stated.

N2. Fixed.

  • The "Ruled" paragraph keeps to the owner's checkpoint: sharing, inheritance and surviving holders are preserved, and neither the reset nor the interim refusal is approved.
  • Listeners are credited to Nginx's inheritance and lifetime requirements, and every socket kind to the general sharing ruling.
  • The consumed-data clause is labelled as planning's reading.

Renumbering

I checked every reference against the old→new map (old 1–4→1–4, 8→5, 6→6, 5→7, 9–12→8–11, 13–30→12–29; old 7 left the list). No reference is broken:

  • Blockers: 9←8, 10←3, 11←5–10, 13←12, 14←12, 18←17, 21←17–18, 22←21, 23←13–16 and 19–21, 24←19, 25←21, 26←3 and 10, 27←4, 29←11–28.
  • Evidence anchors: 3/26, 10, 17, 18, 21, 21/23 and 24 match their items.
  • Linker paragraph: it points at item 5.
  • Order: every item names only items above it.
  • Other issues: no issue outside the track cites one of its item numbers.

Blocker

N3. issues/no-guest-case-reads-std-filesystem.md:17 and issues/remove-all-follows-a-link-swapped-in-mid-walk.md:21 still say "Owner: issues/toyos-builds-itself.md, M2".

  • The track now places both beside the path, needed by no exit.
  • Round 3's move directly affects both records. They should say the track owns them beside the path. This is a one-line edit in each file.

SEND BACK

The owner asked whether everything self-hosting needs is written down.
toyos-builds-itself.md held it in five lists in an order it called open:
what to build, what other tracks block, what stops M2, what M3 adds and
what else is owed. It is now one list in dependency order: each item has
its status, the issue that owns it and what blocks it, and the evidence it
rests on is below it, with libc's revision for the compiles.

The status words say only what is true: "direction set" is the owner's
ruling of what an item must do, with its design still engineering's. The
memory and socket designs, which are outside the tree and under review, are
named as such, and so is the sequencing the socket design proposes for the
descriptor work's spawn.

Folded in:
- C reaches served files (#975, #978, #988);
- #970's toolchain package;
- the memory direction;
- the process track's stages 4 to 7;
- socket sharing;
- the boot handoff, beside the path;
- the database feasibility study's libc findings;
- the issues that name this track as their owner, among them pread, libc++'s
  std::filesystem, and the two that keep a build from reproducing.

Each owner ruling of 2026-10-21 is recorded once, in its owning issue:
- the signal scope in the process track;
- sharing rather than moving, in the socket issue, whose exit described a
  move;
- the database rulings in a new track, split from the planning
  dispositions made under delegation.

Four findings get a home:
- unlink-while-open;
- libc's uid and gid;
- sigaction's false success;
- st_nlink 0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu
Japabu force-pushed the wt/toyos-selfhost-list branch from 6961c92 to 9a6f392 Compare October 21, 2026 23:27
@Japabu
Japabu marked this pull request as ready for review October 21, 2026 23:27
@Japabu
Japabu added this pull request to the merge queue Oct 21, 2026
Merged via the queue into main with commit 5e01f8b Oct 21, 2026
8 checks passed
@Japabu
Japabu deleted the wt/toyos-selfhost-list branch October 21, 2026 23:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant