Repository navigation
fileserver: a volume's cache keeps an eighth of physical memory clean - #1099
Merged
Merged
Conversation
Collaborator
Author
Review of #1099 at f436c15 (base 370d7d9, against origin/main e4bdb0a)ArchitecturePassing the clean limit into The policy is weaker than the PR says. The precedent's own rationale is "an eighth leaves seven eighths for the kernel, the other daemons, and the clients' own heaps" ( FindingsBLOCKER 1.
BLOCKER 2.
BLOCKER 3. "No other row moved beyond the base's spread" (the PR body, and the handoff's A/B judgment) is false, and a regression is uninvestigated.
SUGGESTION. SUGGESTION. SUGGESTION. A clean block is the one allocation the server can safely skip. Allocating clean slots fallibly, and not caching on refusal, would remove the abort in Blocker 2 for reads. That might belong in the record's exit instead of this PR. Evidence checked
SEND BACK |
Japabu
force-pushed
the
disk-path-cache
branch
from
October 28, 2026 22:16
f436c15 to
156e709
Compare
Collaborator
Author
Review r2 of #1099 at 156e709 (rebased onto f407918)Round-1 blockers
Taken from the suggestions:
FindingsBLOCKER 1.
BLOCKER 2. The AMD
SUGGESTION. For the orchestrator (PR body). The body cites SEND BACK |
The cache kept at most 16,384 clean blocks, 64 MiB, and drops the oldest read first without moving a block it hits. osbench's fs.disk.seq_read.4096 rereads a 64 MiB file: its blocks and the tree nodes an open reads come to one more than the cache kept, so each fetched run evicted the blocks the reader reached next and every pass came off the disk whole. #1097's AMD readback has 158 MB/s p50 against 4,960-5,054 under Linux. Why one round of five read 426 MB/s is not shown; dirty blocks are outside the clean count, so a round run before the write-back would fit. A host test written as the benchmark reads (write 64 MiB in 4 KiB calls, sync, then open, read whole and close in rounds) read 16,385 blocks off the disk in each round after the first at the old limit. It now runs on a cache holding twice the file and reads none. Each cache is made with its limit, and the server gives it an eighth of physical memory: the share netstack and the compositor take for the same reason, that nothing signals memory pressure, so the bound is all that keeps a cache from holding the machine. The eviction order is unchanged, so a file larger than that still misses on every pass, which #1097's a-sequential-read-of-a-file-as-large-as-datas-clean-cache-reads-the-disk.md records. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each server took an eighth of physical memory, so the DATA, log and boot servers together could keep three eighths clean, and writes fill a cache as well as reads: a flushed block stays as a clean one. The eighth netstack and the compositor take rests on leaving seven eighths to everyone else, which three such shares do not. Now the log and boot roles keep 16 MiB each, their FAT and directories, and DATA the rest of one eighth. The interim, that nothing gives a clean block back to another job, is the-file-servers-clean-blocks-are-given-back-to-no-other-job.md, owned with the memory track: its exit is clean blocks dropped under the memory service's pressure and CLEAN_SHARE deleted. A block read in is cached only if the allocator gives it room. Box::new cannot fail, so a miss under another job's exhaustion aborted the server and lost what it held dirty; a refused clean block is now just not kept, and the read is answered from the disk's bytes. The total-memory decode lives in the SDK (toyos::system::memory_total) for the file server, netstack and the compositor, which each had a copy. #1097's record of the eviction cliff named CLEAN_LIMIT, which is gone; it now names the server's limit, and stays open. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… limit is computed Reserving the block's place in the eviction order before allocating it makes the clean insert refusable but for the block map's node, a small allocation every few inserts; the module doc and the record now name it and the other two allocations that still abort, a dirty block and a fetch's staging buffer. The order queue was the largest single allocation the server makes, and so the likeliest to be refused. The log and boot roles pass SMALL_CLEAN_BYTES and DATA data_clean_limit(), so no limit is asked for ROOT, which has no cache. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mem.anon.map read 1.90x on the AMD bench after this PR's first round. The landing control and a limit-revert boot show the slow mode in a boot of the base, f407918, without this change, and the fast mode in both boots of this head and in the revert boot. So it follows the boot, not the cache. It is recorded as a kernel finding with its evidence (pr-1099 ec19d8d3, r4/), owned by the address spaces. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Japabu
force-pushed
the
disk-path-cache
branch
from
October 31, 2026 16:38
b3f9af2 to
963531f
Compare
Collaborator
Author
Review r3 of #1099 at 963531f (rebased onto main 0c7e62c)ArchitectureThe design is unchanged from r2 and still fits. Each server owns its cache's bound. The three servers total an eighth, the interim is recorded with an owner and an exit, and Round-2 blockers
The new kernel finding: keep it on this PR
FindingsBLOCKER 1.
BLOCKER 2. BLOCKER 3. For the orchestrator (PR body and handoff).
Evidence checked
SEND BACK |
…pped A flush cleaned a block and pushed it onto the eviction order with no reservation. That queue is the largest allocation the server makes, and in a server that mostly writes, this push is the only thing that grows it. A refusal there aborted the server mid-flush, losing the dirty blocks of the runs not yet written. Now the place is reserved first; if it is refused, the block, which the disk already holds, leaves the cache. The clean-blocks record cites `SMALL_CLEAN_BYTES` and `data_clean_limit` rather than the removed `clean_limit`, and lists the flush's dirty list and run buffer among the allocations that still cannot be refused. The metalprobes record says `/log` keeps 16 MiB of clean blocks, not an eighth. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Japabu
added a commit
that referenced
this pull request
Oct 31, 2026
This change is now stacked on #1099, whose cache allocates a block it reads in fallibly (`boxed`). Shared, that block goes into an `Rc` as its own box, which an `Rc` of the block itself could not do on stable Rust. So `SharedBlock` keeps its box, and its comment names that site instead of claiming an owner none had. The clean-blocks record's list of allocations with no refusal path follows the flush as it now is: its runs, the writes it keeps in flight, and each run's shares, plus a block's share count. A flushed block whose place in the eviction order is refused is dropped in `write_run`, as #1099's r3 did in the loop it replaces. #1113's cache tests pass the clean limit #1099's constructor takes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
Author
Japabu
marked this pull request as ready for review
October 31, 2026 17:50
Japabu
added a commit
that referenced
this pull request
Oct 31, 2026
…ithout a request (memory stage 7a) Memory design §7's paged object, without mappings. A pager process supplies one object per file; its holders read it through SYS_PAGED_READ, which copies pinned frames and never asks the file server for a page it holds. The file server's block cache loses its data half: a file's bytes are in its object, and the cache keeps the volumes' structures alone. Kernel (kernel/pure/paged.rs, kernel/src/object/paged.rs): three kinds. A process's one Pager, made only by a process started as one, neither duplicates nor transfers, and its last handle ends every object it serves. A PagedOwner per object supplies by copy into fresh frames, writes under write pins, withdraws, fails, resizes and writes back. A Paged holder reads, waiting in its own request record, linked on its pager, for a supply, a failure, a shrink, the pager's end or a kill; past the pager's report time it is counted and logged overdue and waits on. A pager's process is refused a holder's read. The index is one open-addressed table of the pager's capacity, so nothing it does allocates and its cost is in proportion to the capacity. The kernel holds the one size, and a write grows it as its bytes land. Authority: Rights::PAGER on a SysCap named by SpawnArgs.pager starts a child as a pager. The supervisor holds it and confers it on a row the signed system.toml marks `pager`, the file servers' alone; no syscap name maps to it, and ROOT's server, whose files are immutable, is none. File server (system/fileserver/src/pages.rs): each DATA, LOG and BOOT file has an object, by its volume identity, which holds the volume's node for its life and outlives its last close until room is wanted. An open to read is answered with a holder. A request is read off the volume and supplied; a write reserves on the volume, then lands dirty in the object, a page in part merged with what it held below the old end; a sync writes the dirty pages back before the volume's own sync. A truncation empties the object before the volume frees the blocks. Data passes the block cache by (BlockDev::read_data, write_data), which a counted disk write keeps durable at the next flush; fat32's write splits into reserve and write_back. Every role's cache keeps 16 MiB of structures, in place of #1099's eighth of physical memory for DATA: the file pages that eighth was for are the pager's, within its fixed capacity, and #1099's re-read test moves to the paging layer. SDK: toyos::paged wraps the calls; a served descriptor keeps its holder, and a read at its local offset or a pread goes through it without the capability's turn, the server answering where the holder cannot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 31, 2026
Japabu
added a commit
that referenced
this pull request
Nov 1, 2026
…s many writes there An acknowledged write kept its arena blocks until a flush covered it, so it could be issued again after a loss. A session's arena is 511 blocks, and a full one made Served ask the disk for a Flush before the next request. On the AMD bench (probe at 21265a0) an 8 MiB fsync paid four such Flushes, about 0.95 ms each, besides its own two; on the T14, about 3.5 ms each. And the file server's cache wrote its runs one at a time. The contract is unchanged: a write acknowledged and not covered is issued again after a loss, and Durable means on the medium. Only where its bytes wait changes. The client now gives a read or a write its arena blocks as it goes on the wire (`next_request`'s `place`) and takes them back when it is answered, and keeps an acknowledged write by its ticket; the session keeps the caller's blocks for that ticket until the client retires it, copying them into the arena each time the write goes out. A full arena now only means requests on the wire hold it, and a request waits for one of their answers. Placing every transfer as it goes out, not at submit, is what keeps that wait from deadlocking: a write queued behind a reissue holds nothing the reissue needs. The blocks are shared, not copied: SharedBlock is an Rc of a boxed block, and the cache's slots are SharedBlocks, so a flush hands the disk the cache's own blocks (one copy, into the arena, where there were two). A write to a block the disk still keeps takes a block of its own. The flush keeps up to Disk::depth runs in flight, which a served disk answers as the full-size requests its session carries at once: min(DEPTH 64, 511 / 32) = 15, under the NVMe driver's 63 per queue. The swap track's design rested on acknowledged writes holding the arena; its two bullets now say what the session keeps instead. Review r1: the client carries each write's payload and bounds what it keeps Two blockers from review: a session's retention had no bound once the arena's was gone, and the model did not check what the session kept for a write. The client now carries the caller's payload with each write (`Ask::Write`'s `payload`, generic) and hands it to `place` each time the write goes out. The session's `held` map, `take_retired` and the `expect` for a write whose blocks were gone are deleted: a covered or given-up write drops its payload with it. No copy comes back; the session's payload is the cache's shared blocks. `Placing` is an enum of a read's ticket and a write's payload. What a client keeps is bounded where it is kept. Once acknowledged writes reach the limit it was made with, the caller's next write waits while the client sends a flush of its own. If that flush fails past MAX_ATTEMPTS, writes meeting the limit are answered Device unsent until a caller's flush succeeds, as the arena path refused them. The session's limit is 32 MiB, the file server cache's own dirty bound. The 511-block bound it replaces cost 4 Flushes per 8 MiB round on metal. The model runs a limit of one block, its write's payload being its ticket, and holds a new law: no caller's write first goes out while the client keeps its limit. Removing the wait breaks it. Dropping a payload at Done rather than at the covering flush breaks the durable law. Also: the module header states the payload contract once; cache.rs's header is rewrapped; `Disk::depth` says it is at least one and the `max(1)` goes; the swap record no longer derives a flush threshold from the arena, and says a swap-in read can wait behind a write. Review r1, stacked on #1099: This change is now stacked on #1099, whose cache allocates a block it reads in fallibly (`boxed`). Shared, that block goes into an `Rc` as its own box, which an `Rc` of the block itself could not do on stable Rust. So `SharedBlock` keeps its box, and its comment names that site instead of claiming an owner none had. The clean-blocks record's list of allocations with no refusal path follows the flush as it now is: its runs, the writes it keeps in flight, and each run's shares, plus a block's share count. A flushed block whose place in the eviction order is refused is dropped in `write_run`, as #1099's r3 did in the loop it replaces. #1113's cache tests pass the clean limit #1099's constructor takes. Review r2: `KEPT_BLOCKS`'s doc keeps only its reason: no less than the file server cache's dirty bound, so a cache flushing by that bound is never made to flush by the session's. The counts and timings behind the choice are #1113's: the 511-block arena bound forced 4 Flushes per 8 MiB of persist.seq_write_fsync, 0.95 ms each on the AMD bench and 3.58 ms on the T14. The cache now asserts `DIRTY_LIMIT <= KEPT_BLOCKS` at compile time, so the two crates' constants cannot drift apart silently. A write at the limit with a caller's flush on the wire waits for that flush rather than asking one of the client's own beside it. Only a flush on the wire counts: one queued behind the waiting write could never go out. On main after #1099 and the event doorbells (3e0fd67, f83957d), as one change: those turned the session's doorbell pipe into two events in the same functions this change edits (`pump`, `wait`, the submits and the session's fields). The session keeps main's `bells` and their `ring` and `answered` waits, and this change's payload placement and reads table around them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Japabu
added a commit
that referenced
this pull request
Nov 1, 2026
…zed as #1099 sized the cache A pager addresses its objects by id through its one handle (B3): owner calls take the pager handle and the object id the create answered, the PagedOwner kind is gone, and an object is let go by PagedOp::Release. The server holds one handle however many files it keeps; a create the pager refuses for want of room drops the oldest file nobody holds and asks again. Holders keep a handle each, counted to their client. An object's id is its table slot, so no counter can wrap (B2). Every offset, length and size a pager names is checked against the last page a u64 can end at before the object sees it (B1). The core is kept for the pager's life (B5): kernel::paged::Core counts it, and a frame that leaves while the core would hold less is held aside for the next supply rather than given back and promised again. A pager's capacity is the caller's, bounded by what its job is charged for the index (B6). The index and the object table are kept in pieces of at most 1 MiB (kernel::chunked): a capacity of an eighth of memory asked the kernel heap for one block past its largest allocation, which panics. The file servers split an eighth of physical memory as #1099 did, 16 MiB each for LOG and BOOT and the rest for DATA. The SDK takes a received handle as a holder only once a read of nothing through it answers as a holder's does (B7), and refuses anything else as Malformed. One page constant (B8). linkcase's file-server row is a pager (B4), and the build refuses a file-server row that is not. A holder's wait is reported overdue from its own start and stops being counted when it ends (S1). The withdrawal test bounds its probes, so its control fails rather than hangs (S2). A truncation, and a truncating open, change the volume first and the object after, so a refusal drops no dirty page (S3). A request the server cannot supply is failed (S6). pagerprobe, a pager row testcases' test-runner launches, ends its pager under a helper parked on a page it never supplies: the helper answers Gone, and the pager is refused a holder's read of its own object. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Japabu
added a commit
that referenced
this pull request
Nov 1, 2026
…ithout a request (memory stage 7a) Memory design §7's paged object, without mappings. A pager process supplies one object per file; its holders read it through SYS_PAGED_READ, which copies pinned frames and never asks the file server for a page it holds. The file server's block cache loses its data half: a file's bytes are in its object, and the cache keeps the volumes' structures alone. Kernel (kernel/pure/paged.rs, kernel/src/object/paged.rs): three kinds. A process's one Pager, made only by a process started as one, neither duplicates nor transfers, and its last handle ends every object it serves. A PagedOwner per object supplies by copy into fresh frames, writes under write pins, withdraws, fails, resizes and writes back. A Paged holder reads, waiting in its own request record, linked on its pager, for a supply, a failure, a shrink, the pager's end or a kill; past the pager's report time it is counted and logged overdue and waits on. A pager's process is refused a holder's read. The index is one open-addressed table of the pager's capacity, so nothing it does allocates and its cost is in proportion to the capacity. The kernel holds the one size, and a write grows it as its bytes land. Authority: Rights::PAGER on a SysCap named by SpawnArgs.pager starts a child as a pager. The supervisor holds it and confers it on a row the signed system.toml marks `pager`, the file servers' alone; no syscap name maps to it, and ROOT's server, whose files are immutable, is none. File server (system/fileserver/src/pages.rs): each DATA, LOG and BOOT file has an object, by its volume identity, which holds the volume's node for its life and outlives its last close until room is wanted. An open to read is answered with a holder. A request is read off the volume and supplied; a write reserves on the volume, then lands dirty in the object, a page in part merged with what it held below the old end; a sync writes the dirty pages back before the volume's own sync. A truncation empties the object before the volume frees the blocks. Data passes the block cache by (BlockDev::read_data, write_data), which a counted disk write keeps durable at the next flush; fat32's write splits into reserve and write_back. Every role's cache keeps 16 MiB of structures, in place of #1099's eighth of physical memory for DATA: the file pages that eighth was for are the pager's, within its fixed capacity, and #1099's re-read test moves to the paging layer. SDK: toyos::paged wraps the calls; a served descriptor keeps its holder, and a read at its local offset or a pread goes through it without the capability's turn, the server answering where the holder cannot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Japabu
added a commit
that referenced
this pull request
Nov 1, 2026
…zed as #1099 sized the cache A pager addresses its objects by id through its one handle (B3): owner calls take the pager handle and the object id the create answered, the PagedOwner kind is gone, and an object is let go by PagedOp::Release. The server holds one handle however many files it keeps; a create the pager refuses for want of room drops the oldest file nobody holds and asks again. Holders keep a handle each, counted to their client. An object's id is its table slot, so no counter can wrap (B2). Every offset, length and size a pager names is checked against the last page a u64 can end at before the object sees it (B1). The core is kept for the pager's life (B5): kernel::paged::Core counts it, and a frame that leaves while the core would hold less is held aside for the next supply rather than given back and promised again. A pager's capacity is the caller's, bounded by what its job is charged for the index (B6). The index and the object table are kept in pieces of at most 1 MiB (kernel::chunked): a capacity of an eighth of memory asked the kernel heap for one block past its largest allocation, which panics. The file servers split an eighth of physical memory as #1099 did, 16 MiB each for LOG and BOOT and the rest for DATA. The SDK takes a received handle as a holder only once a read of nothing through it answers as a holder's does (B7), and refuses anything else as Malformed. One page constant (B8). linkcase's file-server row is a pager (B4), and the build refuses a file-server row that is not. A holder's wait is reported overdue from its own start and stops being counted when it ends (S1). The withdrawal test bounds its probes, so its control fails rather than hangs (S2). A truncation, and a truncating open, change the volume first and the object after, so a refusal drops no dirty page (S3). A request the server cannot supply is failed (S6). pagerprobe, a pager row testcases' test-runner launches, ends its pager under a helper parked on a page it never supplies: the helper answers Gone, and the pager is refused a holder's read of its own object. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Japabu
added a commit
that referenced
this pull request
Nov 1, 2026
…ithout a request (memory stage 7a) Memory design §7's paged object, without mappings. A pager process supplies one object per file; its holders read it through SYS_PAGED_READ, which copies pinned frames and never asks the file server for a page it holds. The file server's block cache loses its data half: a file's bytes are in its object, and the cache keeps the volumes' structures alone. Kernel (kernel/pure/paged.rs, kernel/src/object/paged.rs): three kinds. A process's one Pager, made only by a process started as one, neither duplicates nor transfers, and its last handle ends every object it serves. A PagedOwner per object supplies by copy into fresh frames, writes under write pins, withdraws, fails, resizes and writes back. A Paged holder reads, waiting in its own request record, linked on its pager, for a supply, a failure, a shrink, the pager's end or a kill; past the pager's report time it is counted and logged overdue and waits on. A pager's process is refused a holder's read. The index is one open-addressed table of the pager's capacity, so nothing it does allocates and its cost is in proportion to the capacity. The kernel holds the one size, and a write grows it as its bytes land. Authority: Rights::PAGER on a SysCap named by SpawnArgs.pager starts a child as a pager. The supervisor holds it and confers it on a row the signed system.toml marks `pager`, the file servers' alone; no syscap name maps to it, and ROOT's server, whose files are immutable, is none. File server (system/fileserver/src/pages.rs): each DATA, LOG and BOOT file has an object, by its volume identity, which holds the volume's node for its life and outlives its last close until room is wanted. An open to read is answered with a holder. A request is read off the volume and supplied; a write reserves on the volume, then lands dirty in the object, a page in part merged with what it held below the old end; a sync writes the dirty pages back before the volume's own sync. A truncation empties the object before the volume frees the blocks. Data passes the block cache by (BlockDev::read_data, write_data), which a counted disk write keeps durable at the next flush; fat32's write splits into reserve and write_back. Every role's cache keeps 16 MiB of structures, in place of #1099's eighth of physical memory for DATA: the file pages that eighth was for are the pager's, within its fixed capacity, and #1099's re-read test moves to the paging layer. SDK: toyos::paged wraps the calls; a served descriptor keeps its holder, and a read at its local offset or a pread goes through it without the capability's turn, the server answering where the holder cannot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Japabu
added a commit
that referenced
this pull request
Nov 1, 2026
…zed as #1099 sized the cache A pager addresses its objects by id through its one handle (B3): owner calls take the pager handle and the object id the create answered, the PagedOwner kind is gone, and an object is let go by PagedOp::Release. The server holds one handle however many files it keeps; a create the pager refuses for want of room drops the oldest file nobody holds and asks again. Holders keep a handle each, counted to their client. An object's id is its table slot, so no counter can wrap (B2). Every offset, length and size a pager names is checked against the last page a u64 can end at before the object sees it (B1). The core is kept for the pager's life (B5): kernel::paged::Core counts it, and a frame that leaves while the core would hold less is held aside for the next supply rather than given back and promised again. A pager's capacity is the caller's, bounded by what its job is charged for the index (B6). The index and the object table are kept in pieces of at most 1 MiB (kernel::chunked): a capacity of an eighth of memory asked the kernel heap for one block past its largest allocation, which panics. The file servers split an eighth of physical memory as #1099 did, 16 MiB each for LOG and BOOT and the rest for DATA. The SDK takes a received handle as a holder only once a read of nothing through it answers as a holder's does (B7), and refuses anything else as Malformed. One page constant (B8). linkcase's file-server row is a pager (B4), and the build refuses a file-server row that is not. A holder's wait is reported overdue from its own start and stops being counted when it ends (S1). The withdrawal test bounds its probes, so its control fails rather than hangs (S2). A truncation, and a truncating open, change the volume first and the object after, so a refusal drops no dirty page (S3). A request the server cannot supply is failed (S6). pagerprobe, a pager row testcases' test-runner launches, ends its pager under a helper parked on a page it never supplies: the helper answers Gone, and the pager is refused a holder's read of its own object. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The file server's cache kept at most 16,384 clean blocks (64 MiB). It drops the block read longest ago and doesn't move a block it hits.
fs.disk.seq_read.4096rereads a 64 MiB file, and that file's blocks plus the tree nodes an open reads came to one more than the cache kept. Each fetched run then evicted the blocks the reader needed next, so every pass came off the disk whole. Before this change the AMD bench read 151–163 MB/s, against Linux's 4,960–5,054 MB/s.A host test that reads the way the benchmark does showed the cause before any change: write 64 MiB in 4 KiB calls, sync, then open, read whole and close in rounds. At the old limit, every round after the first read 16,385 blocks off the disk.
Change
Cacheis made with its clean limit, whichDataVolume::probe/formatandFatVolume::mountpass in.SMALL_CLEAN_BYTES), enough for their FAT and directories; little of what logkeeper writes is read again. DATA keeps the rest of the eighth (data_clean_limit).issues/the-file-servers-clean-blocks-are-given-back-to-no-other-job.mdholds it, owned with the memory track. Its exit: clean blocks dropped under the memory service's reported pressure, andCLEAN_SHAREdeleted.no-physical-memory-fairness.mdpoints to it.Box::newcould not fail, so a cache miss while another job had exhausted memory aborted the server, losing whatever was still dirty. Now the cache reserves the block's place in the eviction order (order.try_reserve(1)), the largest single allocation the server makes, and then the block (Vec::try_reserve_exact). If either is refused, the read is answered from the disk's bytes and nothing is cached. A block a flush cleans reserves its place too; if refused, the block, already on the disk, leaves the cache (review r3).vm_reservegrants no mapping, and memory is committed at reservation, not at a fault.toyos::system::memory_total()replaces three copies of the sysinfo decode: the file server, netstack and the compositor.CLEAN_LIMIT, which is gone, and now names the server's limit. It stays open: the eviction order is unchanged, so a file larger than DATA's limit still misses on every pass.metalprobes-usb-read-is-answered-from-fileservers-cache.mdnamedCLEAN_LIMITand is updated.Checks
Evidence: https://github.com/ToyOSOrg/evidence/tree/5d0acb50932854107ebbcb8ea031db961763ab0d
repro/): exit 101,[16385, 16385, 16385].data::tests::a_file_the_cache_holds_is_read_again_from_the_cache: the same rounds on a cache made to hold twice the file read nothing from the disk. An open, a read and a close neither bypass the cache nor drop its blocks. The reproduction is not a mutation of what this test guards: it shows the cause on a different input, the old limit.boxedis: refusal needs an allocator that refuses.cargo test -p fileserverexit 0, 87 passed;cargo clippy -p fileserver --all-targetsexit 0, with the same 20 warnings as before the change.flusheswhere this addsreads. The resolution keeps both, and bcachefs: a sync with nothing uncommitted writes no superblock #1098's new test passesCLEAN_LIMITtoformat. It is the one already booted in 34409c9. At 1782ad0:cargo run -- --ci hostexit 0, 105 steps green (evidence https://github.com/ToyOSOrg/evidence/tree/1def8a8495e5b5a018e1869f51fd159827e0122a,r4/rebase/). The head 963531f adds only the issue file below. Itscargo test -p toyos-build --lib sourcegatepassed, exit 0.cargo run -- --build-onlyexit 0, which builds the compositor and netstack for the target. At b3f9af2,--ci hostexit 0, 105 green (r3/, https://github.com/ToyOSOrg/evidence/tree/5d0acb50932854107ebbcb8ea031db961763ab0d). Rebasing onto bcachefs: a sync with nothing uncommitted writes no superblock #1098 leaves this PR's code unchanged.fs.disk.seq_read.4096rose from 151–163 to 371–472 MB/s, with rounds now tight. It is still about 11x below Linux. By arithmetic, an inference: 9–11 µs per 4 KiB call is about one file-server round trip (read_4k_atis 17–21 µs on the same boots). That cost is recorded ina-file-server-request-costs-three-to-twelve-times-a-kernel-file-call.mdand is the fs-rtt agent's.r4/). The boots: twelve AMD boots of six trees; the revert boot (cc47fc9, the first round with the limit back at 64 MiB); the landing control, two boots of b3f9af2 against two of f407918.mem.anon.map(1.90x) is bimodal per boot. Nine boots read 198–215 ns/MiB and three read 376–388. One of the three is a boot of the base f407918, without this change. Both heads read 202 and 204 and the revert boot 200. Only the reserve itself moves (touch,unmapandtotaldo not), and not with the clock. Recorded as a kernel finding with the evidence:issues/a-32-mib-anonymous-reserve-on-the-amd-bench-costs-twice-as-much-in-some-boots.md, owned by the address spaces.proc.spawn_wait.large(1.23x) does not follow the change either. The heads read 210 and 217 µs against the base's 178 and 192. But the revert boot, whose file server is functionally osbench: persist and the disk's cached rows run on ToyOS's DATA, the bench's NVMe #1097 + bcachefs: a sync with nothing uncommitted writes no superblock #1098, reads 216. The blockring: the arena holds only what is on the wire, and a flush keeps many writes there #1113 boot, which has neither this change nor the revert, reads 219. The program spawned lives in the memory filesystem, not on DATA. The row spreads 155–219 across the twelve boots.crunch.*,mem.memcpy.*andtime.now. Two pairs of boots of one tree lean as far: 31 against 16, and 23 against 16 (r4/count-landing.txt). So this is boot-to-boot variance, not attributable to the change.pipe.throughput.*, 2–9% lower MB/s. They lie inside the four same-tree A boots' range (r4/pipe.txt), exceptpipe.throughput.64at 122 against 125–136.fs.disk.readdir.1000is 3% slower (701–702 against 678–679 µs).fs.disk.mkdir_rmdirandpersist.seq_write_fsyncare better.Coordination
system/fileserver/src/cache.rs: this adds the limit field, and blockring: the arena holds only what is on the wire, and a flush keeps many writes there #1113 makes the slots shared. Whichever lands second rebases.🤖 Generated with Claude Code