Repository navigation
files: one client of the file servers, in toyos, for std and libc alike - #988
Conversation
Review of #988, round 1 (head
|
#978 implemented fcntl's F_DUPFD_CLOEXEC on the descriptor table: the lowest free descriptor at or above the argument, marked close-on-exec, as POSIX has it. 206_libc_refusals still expected the old refusal (-1, ENOSYS), and as a metal-only corpus case nothing ran it before the T14 boot of #988, which answered 10. The record of what libc refuses loses the same stale lines, and F_DUPFD and F_GETFD's departures with them: both are POSIX's now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r1 of #988. An open of a served file installed its descriptor on the capability's turn. On a full table the table let the new open file go, whose close asks the server on that same turn, so the thread waited on itself and every later user of the directory behind it. The open now hands back what it opened and installs it once the turn is let go, and a full table answers the table's own refusal, EMFILE in C and QuotaExceeded in std, on /tmp and on a served path alike; the kernel arm answered EINVAL and the served one ENFILE. A corpus case fills the table and opens both. libc's realpath resolves the names a file server holds as POSIX does, one at a time: a link before the last name is followed, `..` after one is the parent of where it led, and a relative one is read against its own directory. At the first name the kernel holds it hands the rest to the kernel's resolver, so a served link to /tmp or /system resolves there, where it answered ENOENT for a file that exists. The turnstile's sleep is private, a futex, and a yield in a host test, rather than a public trait and type parameter that existed for the test. A descriptor that names no kernel handle is refused a child with EOPNOTSUPP (std's Unsupported), as an addopen of a served path is; a kernel NotSupported is EOPNOTSUPP in C. A served refusal of a name the volume cannot store is EINVAL. std's stdin reads whoever serves it. Issue records this made stale are brought up to date: - a served file's raw fd is retired; - the served flush issue is now about fsync, since flush is a no-op as everywhere else; - the kernel's non-exclusive create names C's O_EXCL too; - /tmp's symlink is left with the remaining exit only; - two records and grants.rs point at toyos::files; - a served file at stdio stopping every spawn is recorded; - the corpus licence counts the four new files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…0da367a21cc1: host tests, 2 resolver controls, --ci host, both builds, 13 QEMU rows, testcases staging with the whole corpus, round 1 T14 logs; masked
|
Round 2 ( B1 (fixed).
B2 and B3 (fixed, not recorded as a gap).
B4 (done).
B5 (fixed). The licence now counts 334, 78 and 77, with the B6 (fixed). B7 (stated and recorded).
Optionals:
Also in this round: Checks:
|
Round 2 (head
|
Review r2 of #988. realpath handed a path to the kernel at the first name no file server held, so every path under a folder grant deeper than one name, such as fs:/home/toy/Games without fs:/home, went to the kernel at /home and answered ENOENT. The handoff is now asked of the whole path as it stands, and a name above the capability that holds it is passed through as a directory, as the client's routing passes it. A host case with a three-name capability pins it. 212_full_table filled the table with /dev/null opens, each a kernel handle, and on the T14 the kernel's handle table ran out first: its 4096 slots are MAX_FDS's, and the process's own handles take some of them. libc answered that EINVAL, and the served open, which needs no kernel handle, then succeeded. The case now fills the table with duplicates of one /home descriptor, which take no kernel handle, so the /home open runs the install-after-turn path that hung at 2d3c5d8, and then fills the kernel's table on its own. toyos::fd::MAX_FDS is the kernel's slot count by name. libc answers the kernel's ResourceExhausted ENOMEM, as std answers it OutOfMemory, where it fell to EINVAL. It is the kernel's one word for a full handle table and for no memory, so a full table cannot yet be EMFILE; issues/a-full-kernel-handle-table-answers-enomem-not-emfile.md records it. A rename between two of the kernel's mounts is EXDEV, as one between a server and the kernel is, rather than EOPNOTSUPP. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d6d801fdddee: host tests, 3 resolver controls, --ci host, both builds, 13 QEMU rows, proctreecase and testcases staging, round 2 T14 log; masked
|
Round 3 ( N1 (fixed). N2 (fixed).
Optionals:
Checks:
|
Round 3 (head
|
|
Metal, T14 (ThinkPad T14 Gen 2), head |
std kept the process's directory capabilities, its routing of a path to the kernel or a file server, its link following and its open served files in std's own fs module, so a C program reached none of them: every path under /home, /apps, /config, /state, /log and /boot answered ENOENT. toyos::files now keeps that state, in the copy of toyos that owns the descriptor table, behind toyos_fs_*. An open served file is a descriptor naming an object of toyos's own (tag SERVED_FILE), held in a registry its object word indexes, so read, write, seek, fstat, truncate, fsync and close dispatch on the descriptor whoever serves it. A capability's connection is shared by threads through a futex turnstile, which replaces std's copied Mutex and Condvar one (sdk/std/copies, sys/turns.rs go). std's fs is a thin client of it. libc's open, read, write, lseek, fstat, ftruncate, fsync, stat, lstat, unlink, rmdir, mkdir, rename, chdir, readlink, realpath, opendir and readdir route through it, with each refusal as POSIX's errno; symlink works where a file server holds the name, which refuses one that exists, and stays ENOSYS on the kernel's. stdout and stderr write a descriptor through it, so a served file at 1 or 2 is written as one. A corpus case writes, reads, links, resolves, renames, lists and removes files under /home. A posix_spawn addopen of a served path answers EOPNOTSUPP until a served file can be given to a child. realpath of a served path follows its last name's links, POSIX-relative to the link's directory; the server resolves the names before it. A refused fd: endowment is now said on stderr, once the table's lock is let go. rust: library/Cargo.lock names toyos's alloc edge (9d8bb4c347f). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
lowest free descriptor at or above the argument, marked close-on-exec, as POSIX has it. 206_libc_refusals still expected the old refusal (-1, ENOSYS), and as a metal-only corpus case nothing ran it before the T14 boot of #988, which answered 10. The record of what libc refuses loses the same stale lines, and F_DUPFD and F_GETFD's departures with them: both are POSIX's now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r1 of #988. An open of a served file installed its descriptor on the capability's turn. On a full table the table let the new open file go, whose close asks the server on that same turn, so the thread waited on itself and every later user of the directory behind it. The open now hands back what it opened and installs it once the turn is let go, and a full table answers the table's own refusal, EMFILE in C and QuotaExceeded in std, on /tmp and on a served path alike; the kernel arm answered EINVAL and the served one ENFILE. A corpus case fills the table and opens both. libc's realpath resolves the names a file server holds as POSIX does, one at a time: a link before the last name is followed, `..` after one is the parent of where it led, and a relative one is read against its own directory. At the first name the kernel holds it hands the rest to the kernel's resolver, so a served link to /tmp or /system resolves there, where it answered ENOENT for a file that exists. The turnstile's sleep is private, a futex, and a yield in a host test, rather than a public trait and type parameter that existed for the test. A descriptor that names no kernel handle is refused a child with EOPNOTSUPP (std's Unsupported), as an addopen of a served path is; a kernel NotSupported is EOPNOTSUPP in C. A served refusal of a name the volume cannot store is EINVAL. std's stdin reads whoever serves it. Issue records this made stale are brought up to date: - a served file's raw fd is retired; - the served flush issue is now about fsync, since flush is a no-op as everywhere else; - the kernel's non-exclusive create names C's O_EXCL too; - /tmp's symlink is left with the remaining exit only; - two records and grants.rs point at toyos::files; - a served file at stdio stopping every spawn is recorded; - the corpus licence counts the four new files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r2 of #988. realpath handed a path to the kernel at the first name no file server held, so every path under a folder grant deeper than one name, such as fs:/home/toy/Games without fs:/home, went to the kernel at /home and answered ENOENT. The handoff is now asked of the whole path as it stands, and a name above the capability that holds it is passed through as a directory, as the client's routing passes it. A host case with a three-name capability pins it. 212_full_table filled the table with /dev/null opens, each a kernel handle, and on the T14 the kernel's handle table ran out first: its 4096 slots are MAX_FDS's, and the process's own handles take some of them. libc answered that EINVAL, and the served open, which needs no kernel handle, then succeeded. The case now fills the table with duplicates of one /home descriptor, which take no kernel handle, so the /home open runs the install-after-turn path that hung at 2d3c5d8, and then fills the kernel's table on its own. toyos::fd::MAX_FDS is the kernel's slot count by name. libc answers the kernel's ResourceExhausted ENOMEM, as std answers it OutOfMemory, where it fell to EINVAL. It is the kernel's one word for a full handle table and for no memory, so a full table cannot yet be EMFILE; issues/a-full-kernel-handle-table-answers-enomem-not-emfile.md records it. A rename between two of the kernel's mounts is EXDEV, as one between a server and the kernel is, rather than EOPNOTSUPP. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
4327073 to
2b122fe
Compare
…aeb547dc38528edf: --ci host, builds, 3 QEMU rows; masked
|
Rebased onto The conflicts were all in issue text. Main's audits renamed things in five records that this PR edits or retires:
The code merged cleanly. The only file both sides touched is The rust gitlink stays at Checks at the head: The T14's 146/146 at The checks script's parent expression is now |
The owner asked whether everything self-hosting needs is written down. toyos-builds-itself.md held it in five lists in an order it called open: what to build, what other tracks block, what stops M2, what M3 adds and what else is owed. It is now one list in dependency order: each item has its status, the issue that owns it and what blocks it, and the evidence it rests on is below it, with libc's revision for the compiles. The status words say only what is true: "direction set" is the owner's ruling of what an item must do, with its design still engineering's. The memory and socket designs, which are outside the tree and under review, are named as such, and so is the sequencing the socket design proposes for the descriptor work's spawn. Folded in: - C reaches served files (#975, #978, #988); - #970's toolchain package; - the memory direction; - the process track's stages 4 to 7; - socket sharing; - the boot handoff, beside the path; - the database feasibility study's libc findings; - the issues that name this track as their owner, among them pread, libc++'s std::filesystem, and the two that keep a build from reproducing. Each owner ruling of 2026-10-21 is recorded once, in its owning issue: - the signal scope in the process track; - sharing rather than moving, in the socket issue, whose exit described a move; - the database rulings in a new track, split from the planning dispositions made under delegation. Four findings get a home: - unlink-while-open; - libc's uid and gid; - sigaction's false success; - st_nlink 0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The owner asked whether everything self-hosting needs is written down. toyos-builds-itself.md held it in five lists in an order it called open: what to build, what other tracks block, what stops M2, what M3 adds and what else is owed. It is now one list in dependency order: each item has its status, the issue that owns it and what blocks it, and the evidence it rests on is below it, with libc's revision for the compiles. The status words say only what is true: "direction set" is the owner's ruling of what an item must do, with its design still engineering's. The memory and socket designs, which are outside the tree and under review, are named as such, and so is the sequencing the socket design proposes for the descriptor work's spawn. Folded in: - C reaches served files (#975, #978, #988); - #970's toolchain package; - the memory direction; - the process track's stages 4 to 7; - socket sharing; - the boot handoff, beside the path; - the database feasibility study's libc findings; - the issues that name this track as their owner, among them pread, libc++'s std::filesystem, and the two that keep a build from reproducing. Each owner ruling of 2026-10-21 is recorded once, in its owning issue: - the signal scope in the process track; - sharing rather than moving, in the socket issue, whose exit described a move; - the database rulings in a new track, split from the planning dispositions made under delegation. Four findings get a home: - unlink-while-open; - libc's uid and gid; - sigaction's false success; - st_nlink 0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The owner asked whether everything self-hosting needs is written down. toyos-builds-itself.md held it in five lists in an order it called open: what to build, what other tracks block, what stops M2, what M3 adds and what else is owed. It is now one list in dependency order: each item has its status, the issue that owns it and what blocks it, and the evidence it rests on is below it, with libc's revision for the compiles. The status words say only what is true: "direction set" is the owner's ruling of what an item must do, with its design still engineering's. The memory and socket designs, which are outside the tree and under review, are named as such, and so is the sequencing the socket design proposes for the descriptor work's spawn. Folded in: - C reaches served files (#975, #978, #988); - #970's toolchain package; - the memory direction; - the process track's stages 4 to 7; - socket sharing; - the boot handoff, beside the path; - the database feasibility study's libc findings; - the issues that name this track as their owner, among them pread, libc++'s std::filesystem, and the two that keep a build from reproducing. Each owner ruling of 2026-10-21 is recorded once, in its owning issue: - the signal scope in the process track; - sharing rather than moving, in the socket issue, whose exit described a move; - the database rulings in a new track, split from the planning dispositions made under delegation. Four findings get a home: - unlink-while-open; - libc's uid and gid; - sigaction's false success; - st_nlink 0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Draft, PR 3 of 4 in the descriptor work (
fd-design.mdrevision 8). PR 2 was #978. PR 4 adds the inheritance rule (Q9) and gives served kinds to a child; its socket rule is with the owner as Q18.Problem
std kept the process's file-server client in its own
fsmodule: the directory capabilities, routing a path to the kernel or a file server, following links, and the open served files. A C program reached none of it. Every path under/home,/apps,/config,/state,/logand/bootansweredENOENT(issues/c-programs-name-no-file-a-file-server-holds.md).Resulting behaviour
toyos::filesholds that state in the copy oftoyosthat owns the descriptor table (fd: one descriptor table numbers every raw fd, and a child's descriptors are its fd:<n> endowments #978), behindtoyos_fs_*. std and libc therefore share one set of connections and one table.toyos's own (tagSERVED_FILE). The object's word indexes a registry of open files, so no refcount is hand-rolled.read,write,seek,fstat,truncate,fsyncand close all dispatch on the descriptor, whichever server holds the file.toyos::sync::Turnstile). It replaces std's copiedMutex+Condvarone;sdk/std/copiesandsys/turns.rsare deleted.toyosgainsalloc, throughrustc-std-workspace-allocunderrustc-dep-of-std.fsis a thin client oftoyos::files, and aFileis one descriptor whoever serves it.File::flushis a no-op, as on every other platform. It used to befsync, which made everyBufWriterflush sync the whole volume. AFilebuffers nothing: eachwritebelongs to the server once it answers, andWrite::flushasks for no durability.toyos::filestoo.errcode::of_files):open,read,write,lseek,fstat,ftruncate,fsync,stat,lstat,unlink,rmdir,mkdir,remove,rename,chdir,readlink,realpath,opendir,readdir,isattyand stdio's.symlinkworks where a file server holds the name, and the server refuses a name that exists, as POSIX has it. It staysENOSYSon the kernel's/tmp, whoseSYS_SYMLINKreplaces the name.realpathresolves the names a file server holds as POSIX does, a name at a time (resolve::resolve_path, host-tested): every link is followed where it is met, a relative one against its own directory, and..after a link is the parent of where it led. Once the whole path as it stands is the kernel's, it goes to the kernel's resolver, so a served link to/tmpor/systemresolves there. A path under a folder grant deeper than one name (fs:/home/toy/Gameswithoutfs:/home) is walked, with the names above the capability passed through as the client routes past them.EMFILEon/tmpand on a served path alike. A full kernel handle table, which for descriptors that are kernel handles binds first, answersENOMEM. That is the kernel's one word for a full table and for no memory alike, so it can't beEMFILEwithout a kernel word of its own (issues/a-full-kernel-handle-table-answers-enomem-not-emfile.md). It used to fall toEINVAL.EXDEV, as one between a server and the kernel does. The open file is installed only after the capability's turn is released, so its close on a full table can't wait on that turn.readdirgivesd_type, butd_inois 0: a server's listing names no node.toyos::files, so a served file at 1 or 2 is written to correctly. The stream sink no longer runsfstaton an object that is not a kernel handle.fd:endowment (fd: one descriptor table numbers every raw fd, and a child's descriptors are its fd:<n> endowments #978) is now reported on stderr, after the table's lock is released.211_home_files.cand212_full_table.c.212fills the table with duplicates of one/homedescriptor, which take no kernel handle, and opens a/tmpand a/homefile, each answeringEMFILE. It then fills the kernel's handle table with/dev/nullopens (ENOMEM).211, under/home:chdir, lists and removes;O_EXCL, a read-only write,readlinkof a file,rmdirof a full directory and a gone name.Decisions
Owner (the four-PR plan): giving a served file to a child is PR 4. Until then:
EOPNOTSUPP(std'sUnsupported), and so is anaddopenof a served path;posix_spawnasks for descriptors 0 to 2 on every spawn, a C program that hasdup2'd orfreopen'd a served file onto stdio has everyposix_spawn,systemandpopenrefused, as is anadddup2of a served descriptor;Commandwith inherited stdio refuses in the same case.Before this PR, no served file could sit at 0 to 2, so this is new. It is recorded in
issues/a-served-file-at-stdio-stops-every-spawn.md.Mine:
preadorpwrite, since libc has no caller of them;Arc;Issue records:
issues/a-served-file-panics-when-asked-its-raw-fd.mdis retired, because a servedFileis a descriptor;issues/an-fsync-of-a-served-file-syncs-its-whole-volume.md;create-new-on-a-kernel-path-is-not-exclusivenames C'sO_EXCLon/tmpas well;/tmpsymlink record keeps only its/tmpexit;the-kernel-resolves-a-path-by-rules-of-its-owndescribesrealpath's two halves.issues/c-programs-name-no-file-a-file-server-holds.mdis retired. Its remaining exit,/systemover the protocol and the kernel's file syscalls deleted, is already step 9's file half inissues/the-kernel-is-small-interrupts-post-and-threads-wait.md, which now says libc is a client.Fork commits
main9d8bb4c347flibrary/Cargo.locknamestoyos'sallocedgeThe
rustgitlink names it.Checks (head
43270738e, ona8ea09d20)cargo test -p toyos --lib(the files error encoding, path normalising, the turnstile)cargo test -p toyos-libc-copies(errno codes againsterrno.h; 18 path-resolution tests, served trees beside a kernel one, among them a three-name capability)cargo run -- --ci hostcargo run -- --build-only,--arch aarch64hosted_clang_helloMetal (T14):
2d3c5d8a6: 9/10.206_libc_refusalsfailed only on fd: one descriptor table numbers every raw fd, and a child's descriptors are its fd:<n> endowments #978's staleF_DUPFD_CLOEXECexpectation, fixed here as a commit of its own. Until this PR lands, main fails206_libc_refusalson metal.7cef3dd71: 144/145, the whole C corpus among them.212_full_tablefailed: its/dev/nullopens filled the kernel's handle table before the descriptor table, so the case tested nothing it claimed. It now fills the table with duplicates.app_view, since the open path changed) and testcases (std_fs file_mtime disk_backtrace stream_handlesand the 140 corpus cases with an expectation).Evidence (masked): round 3 https://github.com/ToyOSOrg/evidence/tree/ed178f2641250fe12887bfa29d78109b05d9c2ad/r3; round 2 https://github.com/ToyOSOrg/evidence/tree/7b22cdbd83dfee95bb9b9fe00a6b66b89e56a7d2/r2; round 1 https://github.com/ToyOSOrg/evidence/tree/68a661d0ca78b24c43585ce6b27ede01a2a6cb42/r1
Not done here
spawn.rsandlaunch.rs. Whichever lands second resolves the conflict.🤖 Generated with Claude Code