Skip to content

osbench: the same scheduler and system benchmarks on ToyOS and on Linux - #983

Draft
Japabu wants to merge 1 commit into
mainfrom
wt/toyos-bench
Draft

Japabu wants to merge 1 commit into
mainfrom
wt/toyos-bench

Conversation

@Japabu

@Japabu Japabu commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

The owner asked for a scheduler benchmark and other numbers against Linux. This adds osbench, one std program built unchanged for ToyOS and, statically, for Linux, so each bench runs the same workloads under both of its systems. It builds on #987 (landed), which fixed two ToyOS std defects the benchmark ran into, and on #979's jobs.

Measured. Both benches have run both sides at 951c7d1: Linux binary e4b84c18, ToyOS main, and a #979 probe that never lands. The comparison is published at https://claude.ai/artifact/C155GeweFbe6CAsqRWNQ7K. A separate round-2 PR stacks on this branch.

What it measures

Every benchmark has a fixed warm-up and repetition count, stated in its doc comment, and prints one line, osbench v=1 os=<os> bench=<name> unit=<u> n=<samples> min= p50= p90= p99= p999= max= mean= [key=value...]. A percentile the sample count cannot resolve prints -.

  • system: syscall.null (getpid), thread.spawn_join, process.spawn_wait (this program as osbench nop), pipe.throughput.{64,65536}, mem.anon.{map,touch,unmap,total} over 32 MiB, mem.map_unmap_4k, and mem.bw.{read,write,copy,memcpy} over 64 MiB, all on one thread.
  • sched: wake.periodic.{idle,loaded} (cyclictest-style, 500 µs, 5,000 samples, p50 to p99.9 and max), wake.onto_busy (a wake whose thread last ran on a CPU a spinner holds, with idle CPUs beside it), wake.storm.{call,drain}.w{1,4,16,64}, and pingpong.{pipe,futex}.{same_cpu,cross_core}.
  • fair: three jobs on every CPU. build is one process with one thread, threaded is one process with a thread per CPU, and helpers keeps one 100 ms helper process per CPU alive. The result is each job's share of the work over five 2 s windows. Linux runs it three ways: flat (one session and cgroup, so the unit is the thread), session (setsid per job, so the unit is the autogroup where autogroup applies) and scope (systemd-run --user --scope -p CPUWeight=100 per job). A cond fair.<grouping>.units line says which unit actually applied, for example 3xcgroup or 1xcgroup. If systemd-run is missing or refused, the scope run prints refused=... rather than a number. ToyOS runs it two ways, now that Scheduler stage 3: job fairness, with per-CPU two-level EEVDF and creation authority #979's jobs have landed: onejob (all three in osbench's own job, the counterpart of flat) and perjob (each a job of its own from SysCap::new_job, the counterpart of scope). A per-job run that cannot make a job is refused, never run as one job.

Linux also prints the conditions it reads, and changes none of them: kernel, distro, SMT, cpufreq driver, governor, EPP, max frequency, turbo/boost, AC and battery, THP, autogroup, timer slack, every vulnerabilities/* mitigation, load average and a 1 s busy fraction. No host name, path or identifier is printed. A cgroup path names the user, so the scope check compares paths and never prints them. Both systems print cpu.chunk: the same instructions on both, timed as a 2 ms single-threaded burst. It shows the clock a CPU leaving idle runs at, not the clock it settles at. An earlier est_mhz field assumed 4 cycles per iteration and read 9.5 GHz on the T14, so the compiler had shortened the chain; the field is gone.

File I/O is not measured, because ToyOS stores on the boot USB stick and Linux on NVMe.

Decisions

  • Where it lives. tests/guest/osbench. A program only the test estate runs sits in tests/guest/ (build::PROGRAM_ROOTS), and as a workspace member the image's [programs.osbench] row builds it. The ToyOS side is a metal row, as the repository's other measurements are. There is no benchmarks directory to follow.
  • Shared code; per-OS code only where the facility differs. src/sys/{linux,toyos}.rs hold the futex, the clock every process shares (needed to align the fairness windows across processes; std's Instant cannot cross a process), anonymous mappings, the CPU a thread is on and CPU topology. Linux uses sched_getcpu and sysfs. ToyOS has no such call, so it uses CPUID's APIC ID. Placement (src/place.rs) pins with affinity on Linux. ToyOS has no affinity, so it starts candidate threads until one lands on the wanted CPU, and the results report whether the threads stayed there (placed_held, ran_on_busy, skipped_spinner_away).
  • The futex storm advances its word. A ToyOS futex wait returns only once the word has changed (scheduler::futex_wait); a Linux one returns on any wake. The storm bumps a generation, so both systems end the wait the same way.
  • Same code generation. Linux is built with nightly-2026-07-22, the upstream compiler nearest ToyOS's 1.99 fork, which the build already installs as its stage-0 cargo. It uses the same toyos profile and forced frame pointers like the ToyOS targets. .cargo/config.toml links the musl target with the toolchain's rust-lld, so no C toolchain or Linux toolchain is involved.
  • A metal row that holds no number. osbench (on tests/images/benchcase.toml) holds only that each of the three groups exited 0 and printed group=<g> done. A group exits non-zero only if one of its benchmarks could not measure. The image starts only what a job list needs to reach its log on the boot stick: logkeeper, diskserver, fileserver, usbd and the runner, with an nvme row that only QEMU's controller matches. The runner's capability carries only jobs and dup: jobs for the per-job run, and dup to hand it down. Supported machines: the T14 and the AMD bench, as stated at the row.
  • The std fixes landed as std: available_parallelism is the kernel's CPU count, and an ended thread's stack comes back #987. Without them, threaded would be one thread on ToyOS.

Checks

First measured at 951c7d1, on main db0cf636d; ToyOS was rerun on both benches at 07e3e16 (PASS), with the onejob/perjob runs that replace fair.process. The PR head is 37d913b, on main 2661b9e; it differs from 808fd33, where the host checks ran, only in doc comments. Since 07e3e16 it adds review r1's fixes:

  • a clippy shape checking -p osbench for x86_64-unknown-linux-musl, the Linux half no image builds (it found an unread pipe read, now handled);
  • wake.onto_busy recording where each partner parked;
  • CPUID moved into src/arch/, with issues/osbench-builds-for-no-aarch64-toyos.md;
  • started fairness workloads stopped before a refusal;
  • a time limit on the load child saying it is spinning.

Logs and their exact commands for the measured head: https://github.com/ToyOSOrg/evidence/tree/c5502d37c215a8c67b4283fb113a61b913dad231/osbench (its README says which revision each file is at).

check result
cargo run -- --ci host 0, 87 steps green, at the measured head, at 69eafdb (the tree of 07e3e16) and at 808fd33
cargo run -- --build-only --boot-config tests/images/benchcase.toml 0
cargo +nightly-2026-07-22 build -p osbench --target x86_64-unknown-linux-musl --profile toyos 0, static-pie x86-64 ELF
cargo test --test toyos-build -- --metal --metal-readback <dir> osbench 2 (staged, machine untouched)
QEMU smoke boot of the benchcase image, TCG, 4 vCPUs, before and after the std fix before: cpus=1 and the thread::spawn OOM; after: all three groups exit 0
osbench all as a native arm64 musl build in a Linux container all groups ran; fair.scope refused (no systemd-run in the container)

The QEMU and container runs show only that the code paths run end to end. Their numbers mean nothing and are not in any comparison. They did catch the std defects, a bug in reading the process's CPU mask from a pinned thread, and the futex semantic difference, all fixed here. The Linux pinning, sysfs topology, setsid and cgroup paths ran on a real Linux kernel in the container. systemd-run and the x86 Linux binary run only on the benches.

Running it

ToyOS (the orchestrator runs metal):

cargo test --test toyos-build -- --metal --metal-readback <dir> osbench
# then run <dir>/request.txt's toyos-metal line, with --host <user>@<machine> for the AMD bench
cargo test --test toyos-build -- --metal --metal-readback <dir> osbench   # judges the readback

The numbers are the osbench v=1 os=toyos lines of the readback's log.

Linux (built here, run over ssh, nothing left behind):

rustup target add --toolchain nightly-2026-07-22 x86_64-unknown-linux-musl
cargo +nightly-2026-07-22 build -p osbench --target x86_64-unknown-linux-musl --profile toyos
scp target/x86_64-unknown-linux-musl/toyos/osbench <bench>:/tmp/osbench-<rev>
ssh <bench> '/tmp/osbench-<rev> all; s=$?; rm -f /tmp/osbench-<rev>; exit $s' > linux-<bench>.txt

all takes about a minute. Nothing it starts outlives it: the load child stops itself after 30 s, and the fairness jobs end with their last window.

Report template

# osbench: ToyOS against Linux on <bench>   (osbench <rev>, one session)

Conditions                ToyOS                               Linux
image / kernel            ToyOS <rev>                         cond kernel, cond distro
CPU, CPUs, threads/core   cond cpu_model, cpus, threads_per_core (both)
SMT                       on (every logical CPU brought up)   cond smt_active
frequency policy          T14: HWP, EPP 0x80, lowest..highest cond cpufreq_driver, governor, epp, boost
                          AMD: firmware's P-state, no OS request
power                     operator: AC / battery              cond ac_online, battery_*
idle before the run       only the job list's servers         cond loadavg, busy_permille_1s
cpu.chunk p50 (burst)     <>                                  <>
other                                                         cond thp, autogroup, timer_slack_ns, vuln_*

metric                                   unit     ToyOS p50 (p99, max)   Linux p50 (p99, max)
syscall.null                             ps
thread.spawn_join                        ns
process.spawn_wait                       ns
pipe.throughput.64 / .65536              MB/s     median (min..max)
mem.anon.total (map / touch / unmap)     ns/MiB
mem.map_unmap_4k                         ns
mem.bw.read / write / copy / memcpy      MB/s
wake.periodic.idle (p90, p99.9 too)      ns
wake.periodic.loaded                     ns
wake.periodic.*.slack1ns                 ns       n/a
wake.onto_busy (ran_on_busy)             ns
wake.storm.call / drain, w1..w64         ns
pingpong.pipe.same_cpu / cross_core      ns       (placed_held)
pingpong.futex.same_cpu / cross_core     ns       (placed_held)

Fairness, permille of the work, median of 5 windows (min..max), cpus_milli_p50
grouping            build   threaded   helpers   units
ToyOS onejob                                     1xjob
ToyOS perjob                                     3xjob
Linux flat                                       cond fair.flat.units
Linux session                                    cond fair.session.units
Linux scope                                      cond fair.scope.units
Derived, not measured, on n CPUs: per job build 1/n, others (n-1)/2n each;
per thread build 1/(2n+1), others n/(2n+1). n=8: 125/437/437 and
59/471/471. ToyOS onejob shares within one job by thread, as Linux flat does.

Not like for like
- Fairness differs by design: ToyOS shares per job, two-level EEVDF (#979);
  Linux per thread
  within a cgroup or autogroup, per job only when each job has its own. Over
  ssh, the session's cgroup normally holds the cpu controller, so `session`
  is expected to equal `flat`; `units` says what applied.
- Placement: Linux pins; ToyOS stages and reports placed_held and
  skipped_spinner_away.
- Anonymous memory: ToyOS commits, zeroes and maps the whole range in 2 MiB
  pages inside mmap, and a 4 KiB mmap takes a 2 MiB page. Linux faults 4 KiB
  pages on touch. Only mem.anon.total compares.
- Bandwidth: Linux asks for THP to match ToyOS's 2 MiB pages; huge_pages says
  what it got. memcpy is each system's own (musl on Linux).
- Library: Linux is static musl, so its malloc and memcpy are musl's, not glibc's.
- Timer slack: an ordinary Linux sleep is padded up to 50 us; slack1ns shows
  Linux without it. ToyOS has none.
- Fairness counts work: equal to CPU share on one kind of CPU with every CPU
  busy, which both benches are.
- Not measured: file I/O (USB stick against NVMe), real-time scheduling (no
  privilege on either side).
- Background: ToyOS's usbd and file servers write this run's own log to the
  USB stick as it goes; Linux runs its services and the ssh session.
- QEMU numbers never count.

Follow-up: the clock it measures at

issues/osbench-times-cpus-leaving-idle-and-not-their-settled-clock.md, filed here. Each benchmark runs straight after the previous one, and cpu.chunk is a 2 ms burst at the start of each group, so both read CPUs that are leaving idle. On the T14 the same chunk took 10.5 µs on Linux and 3.3 µs on ToyOS, which idles in hlt. Under all-CPU load both run one clock (2.389 M against 2.371 M chunks/s). Dividing by cpu.chunk is therefore no normalisation: the equal memory bandwidths come out a quarter apart. The fix is a steady-state warm-up before each benchmark, with the chunk time printed beside it. It changes the protocol, so both sides would have to be rerun; that is round 2's to decide, not this PR's.

Risks

  • fair.scope depends on systemd-run --user reaching the user manager over ssh. If it cannot, the run is reported as refused, not faked; on both benches it reached it (3xcgroup).
  • On ToyOS, pingpong.*.same_cpu does not stay on one CPU: it read placed_held=0/41 on the T14. Without affinity it is a cross-CPU number, and placed_held says so.
  • wake.onto_busy on ToyOS now skips, and counts in skipped_parked_elsewhere, every round whose partner did not park on the busy CPU, and reports where the waker ended instead of refusing. The runs at 07e3e16 predate this, so their ToyOS ran_on_busy=0 does not by itself say that ToyOS moved the wake. osbench round 2: files, IPC, networking, processes, memory, time, the tail, a workload and boot #989's boots run this code.
  • The ToyOS job list's bound is 60 s, counted from boot. At 07e3e16 the T14's last job ended at 44.9 s, and the AMD bench's Scheduler stage 3: job fairness, with per-CPU two-level EEVDF and creation authority #979 probe at 40.1 s, now that fairness runs two groupings of about 12 s each.
  • The units check reads the cgroup hierarchy the process can see. Inside a container's cgroup namespace it misreads (the arm64 container reported 3xautogroup for session, although the container's own cgroup governs). Over ssh on a bench it sees the whole hierarchy.

🤖 Generated with Claude Code

@Japabu

Japabu commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator Author

Review of #983 at 07e3e16, round 1

Architecture

The kernel and the SDK are unchanged. osbench is one std program under tests/guest/. Each benchmark is shared code over a narrow sys layer per OS, and ToyOS fairness now uses the real Job object. onejob/perjob correspond to Linux flat/scope. A run that cannot make a job is refused, never measured as one job. That is the right shape.

The capability grant is justified:

  • jobs is needed for SysCap::new_job.
  • dup is needed because test-runner can only pass a duplicate down (tests/guest/test-runner/src/main.rs:241-250), and duplicate needs DUP.
  • osbench receives both rights, since a duplicate cannot narrow. No other row gains anything.

The separate boot is justified by its quiet image. The row holds only that each group ran to its end, which matches its job: it delivers evidence and asserts no bound.

Size: +2051 lines. About 1900 are osbench, plus a 62-line image, a 24-line row and a 26-line issue. There are no tests. I am not asking for host tests: the crate is guest-built, so sys/ has no host arm, and the pure parts (nearest-rank percentiles, window indexing) are simple and correct on inspection.

I checked these and found them sound:

  • Windows::work never indexes past count, because the stop check comes before the index.
  • Nearest rank is correct for n=5, 200, 1,000 and 5,000.
  • cpu_cgroup follows cgroup v2 subtree_control correctly, and its autogroup fallback applies only at the root task group, which is when Linux applies autogroup.
  • The storm's generation word and its final drain are ordered correctly.
  • The futex ping-pong cannot lose a wake.
  • IRQ_CPU = 0 matches apic::MSG_DEST.
  • The CPUID topology leaves are read correctly.

Evidence:

  • The tree of 69eafdb is identical to 07e3e16's (git diff --stat is empty), so --ci host at 69eafdb (exit 0, 87 steps) holds for the head.
  • QEMU serial-perjob.log shows perjob changing the shares compared with onejob.
  • The Scheduler stage 3: job fairness, with per-CPU two-level EEVDF and creation authority #979 probe runs on metal and the head's T14 rerun (metal-r1b-t14, PASS) give perjob per-job shares matching the derived values (T14 127/442/430 against 125/437/437). So the endowed job does take effect.
  • The AMD rerun was not done when I read the scratchpad.

Findings

BLOCKER 1. The Linux half is compiled by no gate.

  • Where: tests/guest/osbench/src/sys/linux.rs, the cfg(target_os = "linux") arms in fair.rs, place.rs and main.rs, and build/clippy.rs.
  • Problem: osbench is a PROGRAM_ROOTS member, so hostws::guest_built puts it in $GUESTS, and every workspace clippy shape excludes it. The ToyOS arm builds with each image. Nothing ever builds or lints the Linux arm: about 300 lines of sys/linux.rs plus the Session/Scope groupings and pinning.
  • Consequence: half of the deliverable can stop compiling unseen. Root requires changed configuration arms to be covered by build/clippy.rs.
  • Fix: add a clippy shape, -p osbench --target x86_64-unknown-linux-musl (a check, so nothing links), and install that target with the shapes' other targets. Any other check that compiles the target_os = "linux" arms is equally acceptable.

BLOCKER 2. wake.onto_busy does not establish its precondition on ToyOS.

  • Where: tests/guest/osbench/src/wake.rs:147 and :176-200, with place.rs (the ToyOS spawn_on).
  • Problem: the bench claims each partner "last ran on the busy one and parked there". On Linux that holds, because the partner is pinned until release() (:245). On ToyOS, spawn_on checks only where the candidate first ran. The candidate then blocks in judged.recv() and is woken by the placer, and the scheduler may place that wake on any CPU before the body calls park_until. So where the partner parked is unknown.
  • Consequence: ToyOS's ran_on_busy=0 (T14 and AMD) cannot be read as "ToyOS moved the wake to an idle CPU": the partner may never have been on the busy CPU. This is the like-for-like question this PR exists to answer.
  • Fix: in the body, store sys::current_cpu() in the Seat just before parking. Either skip rounds whose partner parked elsewhere (as skipped_spinner_away does) or report a parked_on_busy count. This costs one atomic.

BLOCKER 3. CPUID intrinsics and a target_arch gate sit outside an architecture module, and the AArch64 gap is unrecorded.

  • Where: tests/guest/osbench/src/sys/toyos.rs:2-5 and :72-130.
  • Problem: root keeps architecture intrinsics and target_arch in architecture modules or selectors. This file is an OS selector arm that holds __cpuid_count, the topology decoding and a compile_error! on target_arch. The guest crates follow the convention: tests/guest/metalprobe/src/arch/mod.rs and tests/guest/rust-tests/src/arch/.
  • Consequence: the fact that osbench cannot build for aarch64-unknown-toyos is recorded nowhere but a compile_error!. AArch64 is an equal target. The cause is that ToyOS offers no call naming a thread's current CPU, and MPIDR_EL1 is not readable at EL0.
  • Fix: move current_cpu/core_of/threads_per_core/cpu_model behind src/arch/{mod,x86_64}.rs, with the compile_error! in the selector. Record the AArch64 gap and its cause in issues/ (the new issue or toyos-runs-on-arm64.md), so it is not silent debt.

BLOCKER 4. A garbled doc comment.

  • Where: tests/guest/osbench/src/sys/toyos.rs:93.
  • Problem: "leaf 0BH's EDX, or leaf 01H's leaf 0BH's EDX, or leaf 01H's EBX[31:24]" repeats a fragment.
  • Fix: "leaf 0BH's EDX, or leaf 01H's EBX[31:24] where leaf 0BH enumerates nothing".

BLOCKER 5. A stale module contract.

SUGGESTION 6. tests/guest/osbench/src/main.rs:27-29, "so both systems run the same code generation". It is true of osbench's own code: the same compiler version, the toyos profile, baseline x86-64 and frame pointers. It is not true of std's platform layer or the libc beneath it (musl's prebuilt std, malloc and memcpy against ToyOS's). The PR's template says so; the comment overclaims. The toolchain name also duplicates build/sysroot.rs:1152 STAGE0_CARGO, so it goes stale when the fork moves.

SUGGESTION 7. tests/guest/osbench/src/fair.rs:269-308. Each early return refuse(...) after the first spawn leaves started children unkilled and unreaped. They keep spinning until the windows end (about 12 s) and overlap the next grouping's measurement (flat→session→scope on Linux). Kill and wait for running before refusing.

SUGGESTION 8. tests/guest/osbench/src/fair.rs:204 and :316-322. On ToyOS, units/kinds are computed and then discarded with let _ = (units, kinds), and group_key() returns the constant "job". Making the set computation and group= Linux-only would remove the dead path. The Plain if cfg!(...) guard arm would read more directly as two #[cfg] arms.

SUGGESTION 9. tests/images/benchcase.toml:9-10. "That is Linux's systemd-run --user, which any user may run" argues a ToyOS grant by analogy with Linux. The first sentence (the perjob run needs jobs, the hand-down needs dup) is the whole justification.

SUGGESTION 10. tests/guest/osbench/src/wake.rs:101. Spinners::start blocks on read_line with no timeout. On Linux over ssh, a child that never says spinning hangs the run. On ToyOS only the list bound ends it.

SUGGESTION 11. In fair.rs, "job" names both a workload (JOBS, fair-job) and a ToyOS Job object (Grouping::Job, "each job a job of its own"). Calling the three "workloads" would remove the ambiguity.

PR description (for the orchestrator)

SEND BACK

@Japabu

Japabu commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator Author

Review r1 answered at 808fd33. --ci host is green (87 steps), including the new osbench shape.

Blockers

  1. build/clippy.rs adds a shape, -p osbench --target x86_64-unknown-linux-musl. It is a check, so nothing links. The host job installs that target with the bare ones; the list is now clippy::TARGETS, and the step is "clippy and its targets". On its first run the shape found an unread read in pipe_throughput, which is now handled.
  2. Each wake.onto_busy partner stores the CPU it parks on, just before parking. A round whose partner parked off the busy CPU is skipped and counted as skipped_parked_elsewhere. Where the waker ended is now reported as waker_end_cpu rather than refused. That is the refusal that failed osbench round 2: files, IPC, networking, processes, memory, time, the tail, a workload and boot #989's T14 run 3.
  3. CPUID lives in src/arch/{mod,x86_64}.rs. The compile_error! is in the selector and names issues/osbench-builds-for-no-aarch64-toyos.md, which records the gap and its cause: ToyOS has no call naming a thread's CPU, and MPIDR_EL1 is not readable at EL0.
  4. The doc comment reads "leaf 0BH's EDX, or leaf 01H's EBX[31:24] where leaf 0BH enumerates nothing".
  5. sys/mod.rs now says Linux adds affinity and the cgroup and autogroup readers.

Suggestions

  • 6 taken: the claim is narrowed to osbench's own code generation, and the comment points at STAGE0_CARGO.
  • 7 taken: started workloads are killed and reaped before any refusal.
  • 8 taken: name uses two cfg arms, the unit report is per OS, and ToyOS prints no group=.
  • 9 taken: the analogy is dropped.
  • 10 taken: the load child's spinning line is waited for for at most 5 s.
  • 11 declined: renaming the jobs to workloads would ripple through round 2, which builds on these names. Where the docs touch the ToyOS job object, they now say "workload".

The PR body's Risks are corrected: the 44.9 s list end, the scope and placement results as measured, and the nvme row.

The owner asked for scheduler and system numbers against Linux. osbench is
one std program, built unchanged for x86_64-unknown-toyos and, statically,
for x86_64-unknown-linux-musl, so a bench runs the same workloads under
both systems:

- system: null syscall, thread spawn+join, process spawn+exit+wait, pipe
  throughput at 64 B and 64 KiB writes, anonymous memory map/touch/unmap,
  4 KiB map+unmap, and one thread's memory bandwidth.
- sched: a cyclictest-style periodic wake idle and under load, a wake onto
  a busy CPU beside idle ones, a futex wake storm's call and drain, and
  pipe and futex ping-pong on one CPU and across cores.
- fair: a single-threaded build, a threaded job and a helper-spawning job
  on every CPU, each job's share of the work over five windows, once with
  all three in this program's own group and once with a group per job.
  ToyOS's groups are jobs: `onejob`, and `perjob` with a job each from
  `SysCap::new_job`. Linux runs `flat`, a session per job, and a
  `systemd-run --user` scope per job, and says which fair unit applied.

Each benchmark has a fixed warm-up and repetition count and prints one
`osbench v=1 os=... key=value` line with its distribution; Linux also
prints the conditions it read (kernel, governor, EPP, SMT, power, THP,
mitigations, load). The code is shared except where the facility differs:
the futex, the clock every process shares, anonymous mappings, the CPU a
thread is on, and placement, which Linux does by affinity and ToyOS, having
none, by staging spawned threads and reporting whether they stayed.

It lives in tests/guest/ because that is where a program only the test
estate runs sits, and it builds as a workspace member so the image's
program row builds it. tests/images/benchcase.toml boots it with only the
servers a job list needs, its runner holding `jobs` and `dup`, and the `osbench` metal row runs the three groups
on the T14 and the AMD bench. The row is evidence and asserts no number:
it holds only that each group ran to its end. File I/O is left out: ToyOS
stores on a USB stick and Linux on NVMe.

No image builds the Linux half, so a clippy shape checks `-p osbench` for
x86_64-unknown-linux-musl, and the host job installs that target with the
bare ones. ToyOS has no call naming a thread's CPU, so osbench reads it with
CPUID in src/arch/, and refuses AArch64 by name:
issues/osbench-builds-for-no-aarch64-toyos.md records the gap.

.cargo/config.toml links the musl target with the toolchain's rust-lld,
so the Linux binary needs no C toolchain, and keeps the frame pointers the
ToyOS targets keep.

issues/osbench-times-cpus-leaving-idle-and-not-their-settled-clock.md
records that each benchmark reads CPUs leaving idle, not their settled clock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator Author

Review of #983 at 37d913b, round 2

Assessment

The r1 head was 07e3e1652; I read git diff 07e3e1652 37d913bb5 in full. 808fd33..37d913b touches only fair.rs doc comments, so the host run at 808fd33 (ci-host-6.log, exit 0, "Host: 87 step(s), all green") holds for the head. That run includes cargo clippy -p osbench --target x86_64-unknown-linux-musl. The architecture is unchanged from r1, and the r1 points that were settled stay settled.

Blockers from r1:

  1. Fixed. build/clippy.rs adds the musl shape, and clippy::TARGETS installs its target. The ci.rs doc and step name follow, and no references to BARE_TARGETS or "the bare targets" remain. The shape has already caught a real lint (pipe_throughput's unread first read), and that fix is correct.
  2. Partly fixed, and the fix opens a new gap. The partner half is right: parked_on is stored just before park_until, the round waits for it, and the round is skipped if the partner parked off the busy CPU. But the same change turned the waker check from a refusal into an end-of-run report. That is finding 1 below.
  3. Fixed. CPUID is in src/arch/x86_64.rs and the compile_error! is in the src/arch/mod.rs selector. mod arch is ToyOS-only. issues/osbench-builds-for-no-aarch64-toyos.md follows the issue format and states the cause and the exit.
  4. Fixed.
  5. Fixed.

The suggestions that were taken are implemented correctly:

  • stop() reaps children on every refusal path, including the wait_with_output loop.
  • The ToyOS arms of units and name are direct.
  • The spinning line has a 5 s limit. On timeout the reader thread ends at EOF once the child is killed.

Findings

BLOCKER 1. wake.onto_busy no longer establishes the waker's precondition on ToyOS, but its contract still states it.

  • Where: tests/guest/osbench/src/wake.rs:150-157, the round loop :249-273, and :275/:287.
  • Problem: the doc says "the waker spins on its own CPU". On Linux that holds by construction: spawn_on pins the waker. On ToyOS the waker is unpinned and is read only once, after the last round (waker_end_cpu). A waker that moves onto the busy CPU or the IRQ CPU partway through, and perhaps back again, changes what every later round measures: the waker, the spinner and the woken partner share one CPU, and ran_on_busy changes meaning. Nothing on the line shows which rounds were affected.
  • Evidence that it happens: the osbench round 2: files, IPC, networking, processes, memory, time, the tail, a workload and boot #989 runs show refused=the_waker_left_CPU_4_for_1 on the T14 (metal-t14-3, e5573b7fe), and the rebased QEMU run measured all 200 rounds with waker_end_cpu=0, which is the IRQ CPU.
  • Fix: in each round, read sys::current_cpu() beside the at != busy check. Skip and count the round, for example as skipped_waker_away, exactly as skipped_spinner_away does. Then either drop waker_end_cpu or keep it as a field, and update the doc and the PR's Risks bullet. This costs one CPUID per round, outside the timed interval. Under Linux pinning the count is zero.

PR description (for the orchestrator)

  • Decisions, "Same code generation", still claims the two builds are generated alike. The code comment was narrowed (suggestion 6) and the PR body was not: std's platform layer and the libc under it are each system's own.
  • The report template's "Placement: ... reports placed_held and skipped_spinner_away" omits skipped_parked_elsewhere, plus the field that finding 1 adds.
  • At the head, no metal run includes the new onto_busy code. The Risks bullet says so, and the osbench round 2: files, IPC, networking, processes, memory, time, the tail, a workload and boot #989 r2b boots are where it will first run.

SEND BACK

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant