Repository navigation
Conversation
Review of #1057, round 1 (head
|
…oor it has now, and only a memory refusal waits Review r1 of #1057. - A pipe's ring is counted to the job that made the pipe, so both rings of a connection are its connecting job's: a server's reply spends nothing of its own, and no client can spend a server's floor by connecting to it. - A charge records what it holds of each heap pool, and each growth takes from the pool its job charges then: the supervisor's handle table and range charge, made before its floor is set, grow from the floor after. - A job's account counts its memory refusals. The supervisor waits for memory only when a start raised its job's count; any other refusal, a limit or the supervisor's own memory, closes the ports and says so. - usbd has no floor: it drives disks too, so a USB keyboard is not protected until HID is served apart from storage, which the fairness issue records with the other debt. - A service closed for good drops its job and so its floor. - memory_floor holds that a server's reply leaves its account unchanged, and that every service the exhaustion ended starts again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Response to review r1 (head
|
Review of #1057, round 2 (head
|
…uperframe is answered by one Review r2 of #1057. - A login authority is handed its session's controls as a namespace and connects its own control connection, and a session owner connects its own link (`toyos_session::CONTROL`, `LINK`): under the creator rule both were the supervisor's, up to 4 MiB a session against its floor. The supervisor accepts the link without waiting, through a non-blocking accept. - A spawn reads its program's image for the job the child is endowed to run in, which keeps it as long as the child runs (`Image::read_for`), and a launch is endowed its job before `prepare` reads the image. On `images/system.toml` the supervisor held 1 MiB more per live launch, its children's images; it now holds 43 MiB, flat across launches. - A refusal records the whole superframes in a row it lacked, and a job is ready, and the memory-returned post made, only once they are spare, so a restart refused a ring or a heap superframe no longer reads ready while only scattered frames are free. - The supervisor says what each floor job holds of its floor when its boot is done. - `frames::Held` is `PoolSplit`, beside `object::Held`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r2: response (head
|
Review of #1057, round 3 (head
|
Review r3 of #1057. A floor job refused a superframe while its own floor's came back pinned was never told once the pin went: readiness and the memory-returned post counted the spare whole superframes only, which exclude every floor's, the waiter's own among them, while `admit` lets a floor job take its own. The supervisor then left a floor service such as ps2server down until an ordinary job gave a whole superframe back. A refusal now records the request itself, its frames, whole superframes and the row they need, and `ready` asks `admit` (and the row) whether it would be granted. A waiter that is not ready arms what it lacks beside what its own floor gives it, and is armed again when it gives back to its floor; `refill` posts once the room covers the least lacked beside the floors' deficit, the spare and floor-held whole superframes cover the least lacked beside the floors', and a row lies whole where one is needed. Each is necessary for some waiter's grant, so no waiter is missed, and for a single waiter without a floor it is exact. Laws: a floor job's wait answered by its own floor's superframe, and not while the floor is short; a run answered by a row and not by superframes apart; and a 40 000-step model with two floors, pins and runs asserting after every step that no account still waits whose request `admit` would grant. Also: `endow` has its doc back, the spawn path says whose job an image read for the child is counted to, and the compositor logs why its session's controls refused it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r3: response (head
|
Review of #1057, round 4 (head
|
…usal re-arms its wait Review r4 of #1057. The test waited for the greedy job to take at least what starveling gave back. It takes it at once, but 4 KiB less than the 6 MiB given, so the step waited for the next memory to come back, which was fileserver dying of a refused writeback 1.6 s later; with no such death the test stalled (g4's fourth run). Measured: after starveling's give-back the greedy job took 6140 KiB in each of three runs, its last refusal a single frame. It has taken what came back once its job, after a take, is not ready. frames: - A refusal re-arms a job already waiting, so a smaller request refused while it waits is what memory coming back answers. - The kernel taking a frame of a kept superframe refills, so the rest of that superframe answers a waiter at once. - `floors_after` saturates, so a refused request of every frame is weighed without overflow. - What a waiter lacks in whole superframes is named. std: a boot image the kernel refuses to map for memory reports the kernel's error, not `InvalidInput`. The fairness issue says what removing the supervisor's floor now shows, and records that a restart refused for the supervisor's own memory closes the service. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r4: response (head
|
Review of #1057, round 5 (head
|
Review r5 of #1057. `Frames::alloc_frame` refills once it takes a kept superframe, which can meet a waiter's want, but the kernel's `alloc_frame` took the allocator's lock directly, so the post waited for the next unrelated allocator call. It goes through `try_with`, `with` answering `None` before the allocator exists, which takes the flag under the lock and posts after it. memory_floor's take step ends only on a take made since starveling was asked, so a line from before cannot end it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r5: response (head
|
…rt refused for memory waits for it Since the machine-level charges, a process that took every frame had every other process refused, the supervisor and the input path among them, and a service refused its restart for memory stayed down. Wanted: per-job accounting, no fixed per-app limits, and controls that stay usable. The scheduler's job is the account. `kernel::frames::Ledger` counts each job's committed pages, page tables, runs and heap charges; every allocation a process causes takes the job it is for: the caller's, the child's for a spawn (resolved before anything is built, its program image read for the child's job), the space's, the creator's. A pipe's rings are its creator's, so a connection's are its connecting job's, and a login authority and a session owner connect their own control connection and link. A charge takes each growth from the pool its job charges then. An ended job asserts its ledger empty. `SYS_ACCOUNT_INFO` reads one. A floor is a protected total allowance, cgroup v2 `memory.min`'s contract (owner, 2026-10-24): the first `floor` frames of a job's use, held against every other job as its unused frames exactly and the whole superframes they make, refilled as whole ones come back. A floor job's own heap pool keeps its entitlement as far as its floor. `SYS_JOB_FLOOR` sets one through a job object, on a `SysCap` carrying `FLOOR`, which only the supervisor's has; the supervisor sets its own and each `memory-floor-mib` row's (the supervisor and ps2server; not usbd, a mixed service; not the compositor). A job handle is readable once its last refused request would be granted, by `admit`'s own rule. A waiter that is not ready arms what it lacks beside its own floor, and is armed again when it gives back to its floor or is refused again; every give-back path, the kernel's split of a kept superframe included, posts one IrqWatch, which frees nothing, once some waiter may be granted. The supervisor keeps a service's job across starts; a restart refused for memory its job recorded keeps the ports and is tried again on that readiness; any other refusal closes them and says so. File calls answer at once while a storage service waits for memory. A child's log ring is counted to the child's job (`SHM_FOR_JOB`). memory_floor_holds_the_controls and the memory_floor metal row hold it with one ordinary job holding every frame: a stop's request answered, a key through ps2server, the greedy job killed by a system holder, and a starved service serving again once memory came back. No exhaustion recovery is claimed for visible controls or USB input; what remains is in `issues/no-physical-memory-fairness.md`. Reviewed in #1057 over six rounds; the history is on the PR and its evidence branch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
825114d to
38f3c28
Compare
Owner's decisions recorded; rebased onto
|
Review r6: the rebase of #1057 (
|
…e supervisor's Review of #1057's rebase: the safety comment names the ring's owner as it is, and the fairness issue no longer says every launched program's image is its own job's, since a served image is its server's object. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rebase review: response (head
|
Two sightings of session_lifecycle stalling under parallel host load, the second with the typed command arriving as `detach /pholder`. The harness owns it; the exit is to find where the keys drop and fix it there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Host timeout in merge-queue run 40088051331: the runner was slow; #1057 adds no heavy testEvidence: https://github.com/ToyOSOrg/evidence/tree/d06042d2b5b35290d3c31f951220cc28026fce8a/r9
The branch merges cleanly with |
Problem
Since #1045 every frame and kernel object a process causes is refused at its call when memory runs out. The charges went to the machine, though. When one process took everything, every other process was refused too, the supervisor and the input path included, and a service refused its restart for memory stayed down for good.
The owner asked for:
What changes
Every charge is counted to the job that caused it. The scheduler's job is the account (
kernel::frames::Ledgeronkernel::sched::job::Job). Promises, page tables, runs (pipe rings, device memory) and kernel-heap charges each name their job:charge::current);loader::Commit). The spawner's handle to the child and the child's notice are the spawner's.pipe.rs), so a connection's rings are the connecting job's. A server's reply therefore costs it nothing, and no client can spend a server's floor by connecting to it. A pipe a server makes and hands out, such as fileserver's stream pipe or netstack's bells, is the server's, which chose and bounds it.Ordinary work is the job that uses it:
toyos_session::CONTROL,LINK). The supervisor accepts the link without blocking (ACCEPT_NONBLOCK).fs::IMAGE), and otherwise reads a copy for the child's job (Image::read_for). A launch is endowed its job beforepreparereads the image.SYS_ACCOUNT_INFOreads an account on a job handle carryingREAD: floor, committed, kernel, peak, what the last refusal wanted, and how many requests were refused. An ended job asserts its ledger is empty.Floors.
SYS_JOB_FLOORholds memory back for a job for as long as the job object it was set through lives. It needsSysCapFLOOR, which only the supervisor's carries; no manifest name grants it. The allocator holds every floor job's unused floor against every other job: its frames exactly, and the whole superframes they make for its runs and its own heap pool (kernel/pure/frames.rs's header has the rule).Each charge takes from the pool its job charges at the time (
frames::PoolSplit). So the supervisor's handle table and range charge, made before its floor is set, grow from the floor afterwards. A floor job's own pool stays entitled as far as its floor, so a superframe the heap still holds is never counted free for it.system.tomldeclares floors (memory-floor-mib, and[supervisor] memory-floor-mib). The supervisor sets its own floor, and each floor row's at the creation of that row's job. An installed package's row never has one.images/system.tomlandimages/console.tomlgive the supervisor 96 MiB and ps2server 24 MiB.The retry. A job handle becomes readable once its last refused request would be granted: the allocator asks
admitand the row a run needs, the same rule the request met, so a floor job counts what its own floor holds for it. A waiter that is not ready arms what it lacks under the allocator's lock, and is armed again when it gives back to its floor or is refused again. Every give-back path posts oneIrqWatchonce some waiter may be granted, after the lock is let go. That condition is necessary for some waiter's grant and exact for a single waiter without a floor. The post frees nothing.The supervisor keeps each service's job across its starts. When a restart was refused memory that the job's account recorded, the service keeps its ports and starts again on that readiness, and the supervisor says so both ways. Any other refusal, such as a limit or the supervisor's own memory, closes the ports and is reported as what it was, so a stale want cannot spin the loop. A service closed for good drops its job and floor.
While a storage service waits for memory, the supervisor's file calls answer at once instead of blocking its loop for
FILES_BOUND.Children's log rings are their own.
SHM_FOR_JOBcounts a region to a job handle the caller holds withWRITE. The supervisor's rings for its children now go to each child's job.Consequential decisions
floorof a job's use, cgroup v2memory.min's contract. It is not an emergency allowance on top of the job's use, so each floor's size includes its job's running footprint.Checks and evidence
Evidence:
The branch is
38f3c282b, one commit rebased ontoeba9e8eac, and4bfe7e489, which fixes one comment and one issue sentence. At it, from a clean tree:cargo run -- --build-only,cargo run -- --ci host(103 steps green), the proving test ×3, andsession_lifecycle,console_image_bootsandkernel_objects_refused_alone, each exit 0. The other runs below are at4635148fd(round 5) and825114d83(round 6), except the measurements and the regression rows, which are round 3's atc06cc695e. Rounds 4 and 5 changed onlyframes.rs's readiness, the proving test's take step, the std spawn path's comments and error kind, and one compositor log line, so those still apply.cargo run -- --ci host: 103 steps, all green, exit 0.Host laws in
kernel/pure/frames.rs: 15 new, covering:c06cc695e's rule);admitwould grant;Nineteen mutations, each red and each restored.
The proving test,
memory_floor_holds_the_controls, runs on QEMUProfile::Metal(the T14's shape: q35, i8042, no USB keyboard) in its own boot oftests/images/floorcase.toml. Green three times at this head, and six at round 4's. It needs the guest because it holds the kernel's accounts and floors, the supervisor's restart and the input path together. It judges order and completion only, with no timing. With one ordinary job holding every frame:fileserver dataamong them, starts again.Controls, each one change:
Floor jobs' accounts,
committed + kernel, which the supervisor now prints once its boot is done:images/console.tomlimages/system.toml, at its loopimages/system.toml, desktop session and 4 launchesRound 2 measured 66 MiB after the same launches, growing about 1 MiB a launch for the children's images. Now the four launches add 15 KiB in all.
Regression:
4bfe7e489, after the rebase: the 22 QEMU rows green on a second run. The first run, with every build cold, stalledsession_lifecyclewhen keys typed through QMP were lost (detach /pholder);sessioncasehas no floors, and the row is green alone ×3 (r8 evidence). The rows includekernel_objects_refused_alone,session_lifecycleandconsole_image_boots(the session protocol changed), the spawn, drives, device-memory, usbd, stream, compositor and state rows, and 8 AArch64virtrows;c06cc695e: the 88 shared Rust members in one q35 boot: 81 green on the shipping kernel, and the 7 actuator members green on the test kernel. No kernel panic.T14 at
c06cc695e: all three boots passed:floorcase(memory_floor, without the key, which no bench can type),benchcase(osbench) andtestcases-refused(kernel_objects_refused). Inmemory_floorthe greedy job held 15 599 MiB. On metal the supervisor held 36.9 MiB (peak 40.9) and ps2server 12.0 MiB, matching QEMU.T14 at
71e3b8d78:floorcasepassed.T14 at
e8829f986:floorcasepassed.T14 at
38f3c282b, after the rebase:floorcasepassed. The greedy job held 15 604 MiB, and on metal the supervisor held 36.9 MiB (peak 38.9) and ps2server 11.3 MiB.The test needs a few contrivances, each in its own image or programs:
starvelinggives back most of its stack before it ends, so its end frees less than a start needs;floorcase.toml, so the job can read the supervisor's refusal while every frame is held.The reason: a dead process's memory is freed after its end is published, so a restart otherwise races whoever asks first.
Unresolved
issues/no-physical-memory-fairness.md, each with an owner and an exit:drivesstarts;account_infowithout resident, retained or live-process counts.🤖 Generated with Claude Code