Skip to content

osbench: the rows that compare ToyOS with Linux on one machine - #1053

Merged
Japabu merged 6 commits into
mainfrom
wt/linux-compare
Oct 24, 2026
Merged

Japabu merged 6 commits into
mainfrom
wt/linux-compare

Conversation

@Japabu

@Japabu Japabu commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

issues/no-program-measures-toyos-against-linux-on-one-machine.md asks for one program, built for Linux and for ToyOS, that measures each bench under its Ubuntu and under ToyOS. The owner added the LAN (wired, and Wi-Fi under Linux) and a short CPU crunch that does not overheat the laptops.

Decision: extend osbench, not a second program

tests/guest/osbench already is that program (one source, x86_64-unknown-toyos and static x86_64-unknown-linux-musl, one osbench v=2 line per result, unsupported for a missing facility). This PR adds the missing rows to it. Two nested crates hold what a host must build: osbench/rules (pure judgements, host-tested) and osbench/lan (both LAN ends; the peer binary runs on any host). osbench itself builds for no host, so its pure logic cannot carry host tests.

Resulting rows

row Linux ToyOS
mem.memcpy.<64 B..64 MiB by 4x> (replaces mem.bw.memcpy) measured measured
pipe.throughput.{64,256,1K,4K,16K,64K}; round trips are the existing pingpong.pipe.* measured measured
mem.munmap_touched_remote.4k: sibling on another core read the page; 10 windows, each side's APIC ID (CPUID.0BH:EDX, 01H where 0BH enumerates nothing) before/after, osbench_rules::pair::admit refuses a window with shared or changed IDs pinned staged placement
time.timer_1ms.late (periodic wake at 1 ms) measured measured
nvme.read.4k.random, nvme.read.1m.seq: O_RDONLY|O_DIRECT, nothing writes needs root (sudo osbench nvme), else unmeasured unsupported: no grant reaches a disk's blocks outside a file server's role or the idle slot
energy.idle_60s.package: powercap package-0, per-second mW, wrap via rules::energy::span needs root unsupported: no counter carries package energy
identity: DMI model, microcode, PCI, USB, NVMe, cpuidle; never serials, UUIDs, MACs or addresses sysfs PCI from the kernel inventory (inventory syscap added to benchcase's runner); USB/NVMe/cpuidle unsupported by name
lan.{wired,wifi}.tcp.{connect,rtt.64,upload,download} whichever link reaches the peer; the other says why wired via netstack on the I219; Wi-Fi unsupported (issues/there-is-no-wifi.md)
crunch.{single,all}.{deflate,sha256,matmul,parse}, 4 s each, idle cool-down after each phase stops at TjMax - 15 C one-thread phase only; with no temperature to stop at, every crunch.all.* is unmeasured

Crypto rows are not re-measured: the crypto oracle probe's table is the evidence (ToyOSOrg/evidence at bed0b00efac82546b995f0841da54aea11c45694, r2/table.md).

LAN peer

The bench broadcasts osbench-lan/1 hello to 255.255.255.255:47321 and takes the answer's source as its peer, then opens every TCP connection itself. So no address is configured or printed anywhere, and ToyOS is measured even though the host cannot open a connection to it (issues/the-host-cannot-reach-the-t14-while-it-runs-toyos.md). The peer (osbench-peer) echoes, sinks with a count, or sources for a span. It bounds sizes, spans and concurrent sessions, and refuses anything else. I chose the development Mac as peer: one bench serving the other would put the AMD bench's Wi-Fi in the T14's wired path. The peer must be wired for a wired row to measure the bench's link.

Heat limit

osbench_rules::heat::limit: TjMax is coretemp's temp*_crit (the CPU's own IA32_TEMPERATURE_TARGET). k10temp reports none, so the 4800U takes AMD's published 105 C. Each run prints crunch.limit_from. The 15 C margin is policy: stop before the CPU's own throttling. An unknown CPU stops at 85 C, labelled assumed.

Other changes

  • benchcase.toml starts netstack with pci:02:00:00/8086:15fc (on the AMD bench netstack leaves). The runner receives netstack and inventory. On the T14 every other ToyOS group now runs beside netstack's DHCP/mDNS, as Linux's do beside its stack.
  • net on ToyOS reports its loopback rows unsupported up front. With netstack running, loopback would otherwise be refused (netstack refuses 127/8 by design) and red the group.
  • The metal row's list bound goes from 300 s to 540 s (adds_ms 480 000), derived in tests/toyos.rs.
  • The issue is renamed no-published-run-compares-toyos-with-linux-on-one-machine, since the program now exists. Citations are moved. It lists every ToyOS gap.

Checks and evidence (https://github.com/ToyOSOrg/evidence/tree/8c8291f65eff77753c74c538e314b8db394ce3e7/r1)

  • cargo run -- --ci host at f644cef: exit 0, 103 steps. This includes osbench-rules (13 tests), osbench-lan (5, including a loopback discovery and every request end to end) and the osbench clippy shapes for both Linux targets.

  • Linux musl build and peers: exit 0.

  • Negative controls (checked patches, restored in the same run):

    • m1, same-CPU refusal removed: a_pair_on_one_cpu_is_refused red.
    • m2, only the before-readings compared: the sibling-moved-onto-initiator test and both moved-side tests red.
  • QEMU TCG smoke (4 vCPU, no NIC) of identity, energy, nvme, lan, mem, time, system and crunch: every job exit 0. Every unsupported row named its reason. The munmap pair kept 10/10 windows. TCG numbers are meaningless.

  • Metal staged for T14 and AMD (exit 2, machine untouched).

  • Linux runs (orchestrator, three unprivileged all per bench at f644cef): tables with spreads at https://github.com/ToyOSOrg/evidence/tree/3f183ae653e95bb941564cb725a232a6f54bf6a9/r2.

    • lan refused, because no wired peer exists yet; nvme/energy are unmeasured without root.
    • The crunch never stopped hot: the stops were 85 C on the T14 and 90 C on the AMD bench, and the hottest reading was 87.5 C.
    • crunch.all.parse, about 47x slower per unit than single-thread, is the allocator, not a measurement bug. A parse is 588,862 alloc/free pairs, and the same runs' mem.malloc_churn shows static musl's malloc collapsing under threads. Now said in the crunch header (75878e9).
  • ToyOS boots at f644cef (orchestrator, no peer):

  • ada5bf2 (before the rebase): a datagram receive ends at its read timeout.

    • The cause: ToyOS std's UdpSocket::set_read_timeout was inert (issues/std-udp-socket-ignores-its-timeouts.md). netstack parked a datagram receive until a datagram came, and the client waited without bound.
    • That commit carried the deadline in netstack's request. netstack: a listener and a datagram socket are their holders; no request names a socket by a number #1052, landed on main since, replaced that design: a datagram socket is its holder's own, and std's receive waits on it with the read timeout and the non-blocking flag. The rebase therefore drops this branch's netstack, SDK and libc changes for main's.
    • What the commit keeps (now fba2c2d): the guest test the issue's exit asked for, udp_receive_times_out on netcase, and the issue's closure. On main a receive timeout is WouldBlock, std's Unix word, so the test asserts that.
  • T14 ToyOS boot of ada5bf2 (no peer):

Rebased onto main (2a5b446), after #1052, #1054, #1058 and #1060

before after what it is now
f644cef a2718c0 osbench's comparison rows, unchanged
75878e9 2e5575c unchanged
ada5bf2 fba2c2d the UDP guest test and the timeout issue's closure only (above)
12d0eba 796d9eb no netstack change; the nonblocking issue dropped
c5dfa51 c8ed0fe unchanged in effect, on main's connection.rs
1d15a4f 627073e unchanged

Review r1 (SEND BACK, four blockers), answered in 12d0eba

  1. Unguarded all-core crunch on ToyOS. The orchestrator decided, following the owner's "don't overheat": with no temperature reading, only the one-thread phase runs, and crunch.all.* is unmeasured. The blind cool-down is gone. The gap entry cites the performance-state issue's sampler and the T14's refused PL1/PL2 writes.
  2. settle's doc is back on settle. The rebase took main's netstack, so this no longer applies.
  3. A non-zero timeout under 1 ms is now 1 ms, not unbounded, for UDP and TCP alike, since they share duration_to_ms.
    • udp_receive_times_out adds a 500 us receive. QEMU: exit 0.
    • With the truncation restored (control m4), it stalls at its hang ceiling.
  4. The tables mark every not-like-for-like row in place: crunch.all, persist, huge-page memcpy and bw, 2 MiB first touch, realloc, 1 MiB churn, and yield.

Suggestions taken: download no longer counts the first read's bytes; ASSUMED_STOP_MC is derived from its table; Duration as imported; huge pages in the not-like-for-like marks; the nonblocking issue filed. Since dropped: main honours UDP set_nonblocking.

Suggestion declined: a test of netstack's new wake in main.rs. That stands on inspection: three lines that take the soonest deadline.

Checks at 12d0eba:

  • cargo run -- --ci host: exit 0, 103 steps.
  • udp_receive_times_out (QEMU): exit 0.
  • A QEMU run of the crunch group: the one-thread phase measured, all-core unmeasured, exit 0.
  • No metal rerun. Only the ToyOS crunch's all-core phase changed, and the QEMU run shows it.

Review r2 (SEND BACK, three blockers), answered in c5dfa51

  • A. A zero socket timeout is refused with io::Error::ZERO_TIMEOUT, as std's other platform layers do. The check sits in timeout_ms, the conversion every ToyOS socket timeout setter and connect_timeout takes. udp_receive_times_out asserts InvalidInput from UdpSocket's two setters and from TcpStream::connect_timeout. QEMU: exit 0.
  • B. The crunch.all.* marks are each machine's own, so the AMD table no longer cites the T14's PL1/PL2 issue. The T14's single-thread gap is now written as ToyOS's own unexplained 3,788 MHz ceiling, owned by issues/toyos-uses-what-modern-hardware-offers-for-speed.md. The idle-C-state link is labelled an inference: https://github.com/ToyOSOrg/evidence/tree/0f94abc7cd2e2e1fd663649dc06bfdcc51147b3e/r6
  • C. The nonblocking issue was corrected here. The rebase dropped it, because main's netstack: a listener and a datagram socket are their holders; no request names a socket by a number #1052 keeps UDP's non-blocking flag and a non-blocking receive answers WouldBlock at once.

Both suggestions taken: cool() returns when no starting temperature was read, and the duplicate comment is gone.

cargo run -- --ci host at c5dfa51, from a clean target: exit 0, 103 steps. An earlier run hit the Mac's full disk and was discarded.

Review r3 (one blocker, D), answered in 1d15a4f

  • D. A sensor found but reading nothing no longer lets the all-core phase run unguarded.
    • The group's first successful reading gates the all-core phase. Without one, only the one-thread phase runs and every crunch.all.* is unmeasured.
    • cool() takes that start reading as a plain value.
    • The phase watcher stops a phase at a failed reading as it does at the limit, and each result says which: stopped=no|hot|unread. This replaces the stopped_hot field.
  • Checked by cargo run -- --clippy (exit 0, 27 invocations clean) and the Linux musl build. Per the brief, no run: reading the code establishes it.

Waiting on the owner

  • A wired peer for lan: the development Mac is on Wi-Fi and the AMD bench is Wi-Fi-only. Until a wired peer exists, the T14 metal judge is red on osbench_lan alone, and Linux's all exits 1 for the same reason.
  • sudo osbench nvme and sudo osbench energy under Linux. Without root these rows are unmeasured, and ToyOS reports both unsupported.

Should "no peer" be unmeasured rather than a refusal?

My recommendation is no. Keep the refusal.

  • The cases look alike to the program. A bench with no peer and a ToyOS that never put the broadcast on the wire produce the same thing: sends that succeed and no answer. An unmeasured row would turn a broken DHCP, route or broadcast path green.
  • The hang shows the cost. The kill this PR fixed was visible only because the row failed.
  • The judge tells the truth. The metal row's claim is that every group measured, and a red lan says the boot lacked its precondition.
  • If the owner wants a boot that passes without a peer, the clean form is a boot without the lan job. The bench's environment should decide which boot runs, rather than a pass covering a missing measurement.

Not run: a LAN measurement on either system; the Linux root rows. The Linux NVMe, powercap and getifaddrs link paths have not executed past their permission or peer checks.

Risks / open

  • First image on main giving netstack the T14's I219 (issues/the-host-cannot-reach-the-t14-while-it-runs-toyos.md: no T14 row reads netstack on the I219; issues/the-t14-hung-after-rebooting-with-its-i219-faulted.md). If netstack fails on metal, lan refuses and the row reds.
  • The T14 ToyOS boot's lan refuses unless osbench-peer runs on the LAN. Discovery depends on broadcast reaching the peer: AP client isolation or the macOS firewall can block it.
  • An owner decision, not taken here: a read-only block grant for a benchmark row, or kernel counters for package energy and temperature (MSRs only the kernel reads), would make those ToyOS rows comparisons. Each changes an authority or the ABI.
  • ToyOS's all-core crunch waits on a temperature reading: issues/the-kernel-owns-cpu-performance-state.md (its sampler) owns it, and the comparison issue's gap entry says so.

🤖 Generated with Claude Code

@Japabu

Japabu commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator Author

Review r1: PR #1053 at ada5bf2

Architecture

Extending osbench instead of writing a second program is right. It already was the one-source ToyOS/Linux program. The two nested crates have real reasons to exist: rules holds the pure judgements, which a host can test, and lan holds the peer, which must build for any host. Platform code stays where it belongs: cpu_id is in arch/, and sysfs and hwmon are in sys/linux.rs. The patch adds about 1,990 lines and removes 131. Tests are 13 host tests in rules, 5 in lan (including a loopback discovery and every request end to end) and one QEMU test.

The UDP fix has the right owner. netstack writes the datagram into the socket's pipe and then answers, so a client that gave up on its own would race that write. Putting the deadline in UdpRecvFromRequest and letting netstack answer TimedOut removes the race instead of working around it. The new wake in main.rs takes the soonest deadline (Instant::since saturates). settle delivers before it times out, so a datagram arriving at the deadline is still delivered. libc passes 0, the same unbounded wait as before.

Checked and fine:

  • x2APIC pair rule. pair::admit is a pure function over four readings. The five tests cover same-CPU, each side moving, and the sibling moving onto the initiator. The m1 and m2 patches each turned the tests they target red (r1/controls, logs). Reading the sibling's "before" at a window's first round is consistent with the initiator's. The 01H fallback is ordinary capability selection.
  • NVMe. OpenOptions::read(true) + O_DIRECT means O_RDONLY|O_DIRECT. There is no write path, and identity reads only model and firmware_rev, never serial.
  • Identifiers.
    • The broadcast payload is a constant. The peer prints only {request:?}. osbench prints lan.link as kind, driver and speed, never the interface name or an address. io-error strings carry no addresses.
    • Identity reads no product_serial, product_uuid, board_serial or NVMe serial.
    • I searched the evidence at 3d1a972 for MACs, dotted quads, IPv6 and SSID or serial shapes. The only hits are QEMU slirp's 10.0.2.x in the m3 log. The DMI fields are model-level: MTM, BIOS version.
    • The kernel logs, which hold netstack's DHCP lease line, are deliberately unpublished (r3/README.md).
  • lan with no peer stays a refusal: I agree. Without a peer, the program cannot tell a missing peer from a ToyOS broadcast that never left. This very PR's hang was visible only because the row was red. The osbench metal row is evidence that the orchestrator runs, not a nightly gate. So "run a boot without the lan job" is the right way to get a green boot without a peer.

Findings

BLOCKER 1: the ToyOS crunch runs its all-core phase with no stop, which is a change to the owner's condition that the owner hasn't approved. tests/guest/osbench/src/crunch.rs:10-15, rules/src/lib.rs:100-102.

  • The program contradicts its own policy. MARGIN_MC exists so that "a laptop never reaches the CPU's own throttling", but on ToyOS the header falls back to exactly that throttling as the guard.
  • What the evidence shows:
    • AMD: Linux reached 85.25 °C against its 90 °C stop. ToyOS's all-core clocks there were the same (all.sha256 3,342 MHz against 3,366), and nothing watched them.
    • T14: Linux ran all-core at 1.7–1.9 GHz and peaked at 77 °C. ToyOS held 3,788 MHz. The likely reason is that ToyOS refuses the firmware's DYTC writes of PL1/PL2, so the package runs under the limits boot left (issues/the-t14s-thermal-modes-do-not-move-its-package-power-limits.md).
  • Is running unguarded acceptable? The silicon is protected: Intel's TCC (SDM Vol. 3B §15.8, enabled by firmware in IA32_MISC_ENABLE, which ToyOS does not write), AMD's HTC, and THERMTRIP. The phases are bounded at about 16 s each. The tight unit spread on the T14 (all.sha256 p50 990.9 µs, p99 1,016 µs) suggests TCC never engaged. That is an inference, not a reading.
  • So the hardware protects the machine, but the guard the owner asked for is absent on one system. Accepting the CPU's own protection instead is the owner's call, not the implementer's.
  • Fix, either of:
    • (a) My recommendation: with no reading, report crunch.all.* as unmeasured ("no package temperature to stop at") and keep the single-thread phase. This needs no new mechanism.
    • (b) The owner explicitly accepts the CPU's own protection on ToyOS. Then the header and MARGIN_MC's doc must say that, and the PR lists it under "Waiting on the owner". Today it appears only as a table cell and a handoff caveat.
  • In both cases: the gap entry in issues/no-published-run-compares-toyos-with-linux-on-one-machine.md:44 should cite issues/the-kernel-owns-cpu-performance-state.md. That issue's sampler (IA32_PACKAGE_THERM_STATUS, MSR_TEMPERATURE_TARGET, MSR_PKG_ENERGY_STATUS) is the recorded owner and exit for both the reading and the energy row. Without it the gap closes when this issue does, and nobody owns it.

BLOCKER 2: settle's doc comment now sits on next_receive_deadline. system/netstack/src/serve.rs:712-720. The new function was inserted between settle's doc and settle. As a result:

  • next_receive_deadline reads "Everything the node has to say since the last call… The soonest a waiting datagram receive is owed its TimedOut."
  • settle has no doc at all.

Move the three lines back onto settle.

BLOCKER 3: a sub-millisecond read timeout becomes an unbounded wait. sdk/std/sys/net/connection.rs:59-63.

  • duration_to_ms(Some(500µs)) gives as_millis() == 0, and both std (wait) and netstack read 0 as "no bound". So after this fix, set_read_timeout(Some(Duration::from_micros(500))) followed by recv_from still hangs forever.
  • That is the defect class this PR closes. The deleted issue's exit ("recv_from returns … once the read timeout passes") does not hold for it.
  • The same mapping already affected TCP read, write and connect_timeout.
  • Fix: Some(d) => d.as_millis().clamp(1, u32::MAX as u128) as u32. std already refuses a zero Duration. Type-level and inspection suffice; no new run is needed for this line.

BLOCKER 4: the r4 reading of the crunch rows is not like for like, and it doesn't say why. Evidence r4/README.md, r4/compare-t14.md.

  • Single thread. It reports "ToyOS is 1.10-1.17x Linux's time for deflate, sha256 and matmul" with no mention that ToyOS's T14 tops out at 3,788 MHz where Linux reaches 4,189 MHz (every mhz and cpu.clock line). That ratio, 1.106, accounts for most of the gap, so the row reads as a code difference when it is a clock-ceiling difference.
  • All-core. "ToyOS's held clock is a heat it cannot see" is speculation. The recorded cause is the PL1/PL2 issue above, and that is what should be cited.
  • "ToyOS ahead" column. The table's crunch.all.* and persist.* rows carry "ToyOS ahead" verdicts with no in-table marker. Only the READMEs say they are not like for like.
  • Fix: state the clock ceilings, cite the issue, and mark those rows in the table (or move them to the "not like for like" section). This is a prose and generator change; nothing needs re-running.

Suggestions

  • lan/src/lib.rs:196: download counts the first read's bytes, up to 64 KiB, but starts its timer after that read. The bias is under 0.1 % at 5 s, but it is a wrong count. Subtract those bytes, or start the timer before the first read.
  • sdk/std/sys/net/connection.rs:619: UdpSocket::set_nonblocking answers Ok(()) and does nothing, so a non-blocking recv_from still blocks. This is the same defect class and I found no record of it. File an issue, or map nonblocking onto the new field. It doesn't need to land here.
  • UDP test coverage: udp_receive_times_out with m3 holds netstack's TimedOut answer. It does not hold the new wake in main.rs:444-447: other traffic could wake netstack within the 30 s ceiling. Inspection is enough for three lines, but the PR's "wakes for it" rests on inspection, not on the test.
  • rules/src/lib.rs:105-107: ASSUMED_STOP_MC says it is "the lowest TjMax in PUBLISHED less the margin" but hardcodes 85,000. Either derive it in a const, or drop the derivation from the doc.
  • Huge pages in the tables: mem.memcpy.* and mem.bw.* compare Linux with 128 MiB of huge pages against ToyOS with none. The row fields say so, but the table's not-like-for-like list doesn't.
  • serve.rs:609: std::time::Duration can be Duration, which is already imported.

Verdict

SEND BACK. Blockers 2 and 3 are one-line code fixes. Blocker 4 is an evidence-prose fix. Blocker 1 needs either the default change in (a) or the owner's explicit acceptance in (b).

@Japabu

Japabu commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator Author

Review r2: PR #1053 at 12d0eba

Scope: round 1's four blockers, the cool-down question, and whether the round added a defect. The round is +99/−59 over 9 files: one code commit and the r5 evidence.

Architecture

The round keeps the shape round 1 accepted. With no temperature, the program now refuses to do the risky thing instead of falling back to the CPU's own throttling. That is the simpler and more honest design: no blind timer, and the gap's exit is the kernel sampler that issues/the-kernel-owns-cpu-performance-state.md already owns.

Round 1's blockers

1. Unguarded all-core crunch: resolved on ToyOS, and the cool-down holds on Linux.

  • ToyOS. sys/toyos.rs:373 always returns Err, so crunch.rs takes the let Some(heat) = heat else arm. It runs single alone and emits unmeasured for each crunch.all.*. unmeasured does not set REFUSED, so the group stays green. The r5 QEMU serial shows exactly this: four single rows, four unmeasured=no_package_temperature… rows, secs=17, exit 0.
  • Linux cool-down. Only BLIND_COOL is gone. Where a sensor reads, cool() still runs after single and after all. It is temperature-driven: it polls once a second until heat::cooled(start, now) (within 5 °C of the group's start), with a 90 s ceiling (COOL_BOUND). So Linux keeps a wait for a temperature mark, with a ceiling, between the two phases. A 1 s poll is reasonable because hwmon gives no crossing notification. The implementer's "the fixed 30 s cool-down … is gone" is accurate, since only the no-sensor path had it. The owner's condition is not weakened.
  • Gap entry. It cites both the sampler issue and the PL1/PL2 issue.

2. settle's doc: resolved. It is back on settle, and next_receive_deadline has its own one-line doc.

3. Sub-millisecond timeout: resolved as asked, but see new blocker A.

  • duration_to_ms clamps a Some to at least 1 ms. TCP and UDP share it.
  • udp_receive_times_out adds a 500 µs receive and passes (r5 logs/guest-udp-timeout-2.log, PASS, exit 0).
  • Control m4 restores the truncation and stalls exactly at the 500 µs receive: receives 0 and 1 print, then the guard expires (control-m4-…log). That control would expose this defect.
  • The test and the control ran on the uncommitted tree that 12d0eba committed unchanged. ci host ran at 12d0eba (ci-host-4.rev): 103 steps, exit 0.

4. Not-like-for-like marks: mostly resolved, but see new blocker B.

  • compare.py's UNLIKE marks these rows in place instead of giving a verdict: crunch.all, persist, memcpy/bw, the 2 MiB faults, realloc, the 1 MiB churn and yield.
  • The README states the T14 ceilings: 3,788 against 4,189 MHz.
  • It cites the PL1/PL2 issue instead of the "heat it cannot see" speculation.

New findings

BLOCKER A: sdk/std/sys/net/connection.rs:59-66, 160-177, 538-545: a zero timeout is silently 1 ms. std's contract says it is an error.

  • Round 1 said "std already refuses a zero Duration". That was my mistake. The refusal lives in each platform layer, not in generic std: sys/net/connection/socket/unix.rs:185, hermit.rs, sgx.rs and xous all return io::Error::ZERO_TIMEOUT. std's own net/udp/tests.rs::test_timeout_zero_duration asserts InvalidInput.
  • ToyOS's layer has no such check. So set_read_timeout(Some(Duration::ZERO)), set_write_timeout and connect_timeout(…, ZERO) all return Ok, and with this clamp they now wait 1 ms. Before this round they waited without bound.
  • The new doc comment ("a non-zero timeout under a millisecond waits one") implies that zero is handled elsewhere. It isn't.
  • Fix: return Err(io::Error::ZERO_TIMEOUT) for Some(Duration::ZERO) in the four setters and in connect_timeout, as every other layer does. Inspection is enough; the type-level fact needs no guest run.

BLOCKER B: evidence r5 tables and README: two false causal claims.

  • r5/compare-amd.md:95-98. Every AMD crunch.all.* row says "(ToyOS refuses the T14 firmware's PL1/PL2 writes)". compare.py:31 applies that one string to both benches. The AMD bench is not a T14, and r5's own README says its clock ceilings match: 4,291 MHz on both systems. On the AMD bench, the only recorded difference is that ToyOS has no temperature stop. Make the mark specific to each machine.
  • r5/README.md, "It is a clock difference, not a code difference".
    • The T14's 3,788 MHz single-core ceiling is ToyOS's own open weakness: issues/toyos-uses-what-modern-hardware-offers-for-speed.md, "The T14's clock under load … what holds it there is unread". It reads 0.4 GHz short of Intel's 4.20 GHz Max Turbo, where Linux reaches 4,189.
    • The single-core turbo bin depends on how many cores are active. ToyOS's idle cores never go below C1/C1E (same issue, and issues/idle-cpus-enter-the-c-states-cst-names.md). So the ceiling may well come from ToyOS code. This is an inference, but the README states the opposite as fact.
    • Fix: say the ToyOS ceiling is unexplained and owned by that issue, and drop "not a code difference". Prose only, no rerun.

BLOCKER C: issues/std-udp-socket-set-nonblocking-does-nothing.md:15-16: the mio/tokio claim is false.

  • The issue says "A program that polls a UDP socket the ordinary way, mio's and so tokio's among them, blocks where it means to move on".
  • ToyOS's mio fork leaves UdpSocket out on ToyOS entirely. At the pinned 5c122cf, src/net/mod.rs:31-34 has #[cfg(not(any(target_os = "toyos", …)))] mod udp;. So neither mio nor tokio reaches this path.
  • Drop the clause, or say what is true: mio's ToyOS port has no UDP socket, and a std caller that polls with set_nonblocking blocks.
  • The rest of the record is accurate and well-formed: the frontmatter is valid, line 621 is the no-op, and TCP does honour the flag (lines 195, 214, 374). The exit is concrete.

Suggestions

  • crunch.rs cool(). If the first reading is None (start_mc), or the sensor stops answering, the loop cannot finish early and idles the full 90 s. In effect it becomes a fixed sleep. The old code had the same blind case, at 30 s. It is rare on Linux hwmon, but start_mc could fall back to limit.stop_mc − COOLED_WITHIN_MC, or the cool-down could report "unread" and return. Similarly, a sensor that reads at selection but returns None during a phase leaves that phase without a stop. This is pre-existing and optional.
  • lan/src/lib.rs:197. The inline comment repeats the doc two lines above it. Delete one.

Verdict

SEND BACK. Blocker 1's guard and the Linux cool-down hold, and blockers 2 and 3 are fixed with a working control. Blockers A, B and C are each a few lines of code or prose. None needs a new guest or metal run.

@Japabu

Japabu commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator Author

Review r3: PR #1053 at c5dfa51

Scope: r2's blockers A, B and C, the two suggestions the implementer took, and an identifier check of the r6 evidence. The round is one commit, +36/−18 over 5 files, plus the r6 evidence. Everything r2 accepted stays settled.

Architecture

The round keeps the design. Zero-timeout handling now sits in the one conversion every socket timeout passes through, so there is one rule in one place, as the other std platform layers have it. The issue and evidence corrections are prose only.

r2's blockers

A. Zero timeout: resolved.

  • sdk/std/sys/net/connection.rs:62-68: timeout_ms returns Err(io::Error::ZERO_TIMEOUT) for Some(Duration::ZERO).
  • The function is now fallible, so a caller cannot skip it by accident. All five call sites propagate its error with ?:
    • TCP connect_timeout (:164), which refuses before stream::connect runs;
    • TCP set_read_timeout and set_write_timeout (:173, :178);
    • UDP set_read_timeout and set_write_timeout (:540, :545).
  • No other Duration→ms conversion remains in the file. The CONNECT_TIMEOUT constant is a non-zero literal.
  • Test coverage. udp_recv_timeout.rs asserts InvalidInput on all three requested paths: UDP set_read_timeout, UDP set_write_timeout and TcpStream::connect_timeout.
    • The TCP setters are not exercised, but they call the same function, and inspection establishes that.
    • connect_timeout targets 192.0.2.1, which is TEST-NET-1. If the arm regressed, the call would clamp to 1 ms and return TimedOut. That is the wrong kind, so the assertion would fail with a meaningful message.
  • Evidence. r6 logs/guest-udp-timeout-3.log: PASS udp_receive_times_out, 1 passed. It ran on the uncommitted tree that c5dfa51 committed. The diff confirms the committed tree matches. ci-host-6.rev = c5dfa51, 103 steps green, exit=0.
  • Having no mutation run is acceptable: without the arm, expect_err fails, and inspection establishes that.

B. Evidence corrections: resolved.

  • A mark per machine. compare.py keys CRUNCH_ALL by machine:
    • AMD rows now say only "ToyOS ran with no temperature stop".
    • T14 rows say that the PL1/PL2 issue "would explain" the clock difference. That is conditional, not a cause.
    • The parenthetical "ToyOS refuses the firmware's PL1/PL2 writes" is a fact that issue records (KernelRegister refusal of MCHBAR+59A0h).
  • The T14 ceiling. The README splits the claim into Measured, Not measured and Inferred:
    • It withdraws "not a code difference" explicitly.
    • It names the 3,788 MHz ceiling as ToyOS's own open weakness, owned by issues/toyos-uses-what-modern-hardware-offers-for-speed.md, "The T14's clock under load". That entry does say what holds the clock is unread.
  • The idle-C-state link is labelled "Inferred, not measured". Its two premises match the issue's text:
    • cores never go below C1/C1E;
    • the one logged limit bit is multi-core turbo.
  • The ratio arithmetic is correct: 4,189 / 3,788 = 1.106.
  • I found no remaining sentence in r6 that claims an unestablished cause.

C. Nonblocking issue: resolved, and the claim is true.

  • The mio pin is 5c122cfa4f2f2c715133565552c98511ee60a2b8 (Cargo.lock). At that pin, src/net/mod.rs has #[cfg(not(any(target_os = "toyos", …)))] on both mod udp; and pub use self::udp::UdpSocket;.
  • src/sys/toyos/ has no UDP module, so the corrected text is accurate.
  • No tokio is in the workspace. tokio's UDP is built on mio::net::UdpSocket, so "nor tokio" follows.

The accepted suggestions

  • lan/src/lib.rs: the duplicate comment is removed. Nothing else changed.
  • crunch.rs cool(): see the blocker below.

Evidence identifiers

I scanned all seven r6 files, including the 700 KB ci-host log, and found:

  • no IPv4 addresses (zero matches in every file);
  • no MACs;
  • no GUIDs or UUIDs;
  • no vendor, product, board or BIOS strings, beyond the public CPU model "i5-1135G7";
  • no peer endpoint.

The lan rows say only that no peer answered. The home directory is redacted as /Users/<user>/. Clean.

Findings

BLOCKER D. tests/guest/osbench/src/crunch.rs:165-171: with no starting reading, the all-core crunch starts at once on a sensor known to have failed. The new doc claims a guard it cannot give.

  • suite() builds Heat from package_sensor(). On Linux that function (sys/linux.rs:632-651) only finds the hwmon label. It never checks that temp*_input reads or parses.
  • So start_mc = heat.read() can be None while heat is Some. In that case:
    • single runs with a watcher that reads nothing;
    • cool("single", None, …) now returns at once;
    • all runs on every CPU. Its watcher (:191-199) only sets hot on a successful read, so a dead sensor never stops it.
  • The new doc says "the next phase's stop still guards it". That is true only if the sensor starts answering again, and the one fact the function has is that it did not.
  • Consequence. This is the same unguarded all-core crunch on a laptop that round 1's B1 forbade. The rule there was "no temperature, no all-core", and suite() applies it only when no sensor is found, not when a found sensor reads nothing. The shape is pre-existing: r2's suggestion noted it, and the old code idled 90 s blind before running all unguarded anyway. But this round removes the idle and adds a doc that asserts the guard holds. The owner's condition is that crunch never outruns its temperature guard, so this has to be closed rather than described.
  • Fix (simpler than now). Make the starting reading the gate:
    • let Some(start) = heat.read() else { /* single only, crunch.all.* unmeasured, as for no sensor */ }.
    • cool then takes start: i64, loses its None arm, and its doc goes back to one line.
    • Optionally, have the watcher fail closed: an unread sample sets hot. That also covers a sensor that dies mid-phase.
  • Inspection is enough. No guest or metal run is needed.

Verdict

SEND BACK. A, B and C are resolved, and the evidence leaks nothing. D is a few lines in crunch.rs and needs no new run.

@Japabu
Japabu marked this pull request as ready for review October 24, 2026 08:31
@Japabu
Japabu marked this pull request as draft October 24, 2026 08:31
Japabu and others added 6 commits October 24, 2026 10:32
The issue asked for one program, built for Linux and for ToyOS, measuring
memcpy, pipes, a remote munmap, a 1 ms timer, NVMe reads, an idle minute's
energy and the machine's identity; the owner added the wired and Wi-Fi LAN
and a short CPU crunch that does not overheat the laptops. osbench is that
program already, so it gains the missing rows rather than a second program
being written:

- mem: `mem.memcpy.<size>` from 64 B to 64 MiB by fours, which replaces
  `mem.bw.memcpy`; and `mem.munmap_touched_remote.4k`, a page written here,
  read by a sibling spinning on another core and unmapped here. Each side
  reads its APIC ID (CPUID.0BH:EDX, 01H:EBX where 0BH enumerates nothing)
  before and after each of ten windows, and `osbench_rules::pair::admit`
  refuses a window whose sides share an ID or either of which moved.
- system: pipe throughput at 64 B, 256 B, 1, 4, 16 and 64 KiB writes.
- time: `time.timer_1ms.late`, the periodic wake at a 1 ms period.
- nvme: 4 KiB random and 1 MiB sequential reads of the first NVMe
  namespace, opened read-only with O_DIRECT. Linux needs root and says so
  unprivileged; ToyOS mints no grant to a disk's blocks outside a file
  server's role or the idle slot, so it reports unsupported.
- energy: the powercap package-0 counter read each second over an idle
  minute, mW, its wrap through `osbench_rules::energy::span`. ToyOS counts
  no package energy: unsupported.
- identity: DMI model, microcode, PCI, USB, NVMe and cpuidle from sysfs on
  Linux, with no serial, UUID or network identifier; PCI from the kernel's
  inventory on ToyOS, and the rest unsupported by name.
- lan: the bench broadcasts for an `osbench-peer` and times TCP to it,
  connect, a 64 B round trip, and 5 s uploads and downloads, each result
  named for the link that carried it (`lan.wired.*`, `lan.wifi.*`). The
  bench opens every connection, so ToyOS is measured although the host
  cannot reach it (issues/the-host-cannot-reach-the-t14-while-it-runs-
  toyos.md), and neither end is configured with an address or prints one.
  ToyOS's Wi-Fi rows are unsupported (issues/there-is-no-wifi.md).
- crunch: deflate of a fixed 1 MiB corpus, toyos-crypto's SHA-256, a
  160x160 f64 product and syn parsing a generated Rust program, 4 s each
  on one thread and then on every CPU, with an idle cool-down after each
  phase. Where the package temperature reads (Linux's coretemp or
  k10temp), a phase stops at TjMax less 15 C, TjMax being the sensor's
  critical temperature or, for k10temp, which reports none, AMD's
  published 105 C for the 4800U; each run names the source. ToyOS reads no
  temperature: it says so and the phases run their length, the CPU's own
  thermal control standing behind them.

The pure rules are `tests/guest/osbench/rules` and both LAN ends are
`tests/guest/osbench/lan`, nested crates because osbench builds for no
host; the peer runs on any machine on the link. On ToyOS `net`'s loopback
rows are now unsupported up front: netstack refuses 127/8, and benchcase
now starts netstack on the T14's I219 for `lan`.

The crypto rows come from the crypto oracle probe's published table
rather than being measured again. The issue is renamed for what remains,
the published runs, and lists what ToyOS cannot yet measure.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first Linux runs (three per bench, f644cef) read crunch.all.parse at
2.8 s a unit on the T14's 8 threads and 3.1 s on the AMD bench's 16,
against 59 and 73 ms on one. A unit is one thread's parse of the whole
generated program, so the figure is no unit spread across threads. A
counting allocator around the same syn and corpus finds 588,862 allocation
and free pairs per parse. The same runs' mem.malloc_churn falls from 33.6 M
to 3.2 M pairs a second in total on the T14 at 8 threads, and from 28.5 M to
1.8 M on the AMD bench at 16. At that contended rate, about 2.5 to 9 us a pair
per thread, a parse's allocations alone take 1.5 to 5 s. That is musl's
malloc serialising, which the static Linux build links; ToyOS std's
dlmalloc has one lock too. The other three workloads allocate a few blocks
a unit and scale. The row stays, since it is a real program's work on each
system, and its header now says what it measures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The T14's ToyOS osbench boot at f644cef killed `osbench lan` at the list
bound, 321 s into a group whose discovery is bounded at 60 s: with no peer
on the link its `recv_from`, under a 1 s read timeout, never returned
(issues/std-udp-socket-ignores-its-timeouts.md). That was netstack parking
a datagram receive until a datagram came. #1052 has since made a datagram
socket its holder's own and std's receive wait on it with the read
timeout, which is that issue's fix; its exit asked for a guest test with no
answering peer, and this is it.

`udp_receive_times_out`, on netcase's real netstack: two receives under a
300 ms timeout with nothing sent each end `WouldBlock`, std's Unix word for
a receive timeout, after at least 300 ms and inside a 30 s hang ceiling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- crunch: a system that reads no package temperature runs the one-thread
  phase alone and reports every all-core row `unmeasured`, so nothing is
  heated past a limit nothing reads. The owner asked that the laptops not
  be overheated, and on ToyOS the guard was the CPU's own throttling. The
  blind 30 s cool-down goes with the phase it followed. The gap entry
  cites issues/the-kernel-owns-cpu-performance-state.md, whose sampler is
  the recorded way to read both temperature and energy, and
  issues/the-t14s-thermal-modes-do-not-move-its-package-power-limits.md
  for the T14's all-core clock under ToyOS. The metal row's bound text
  drops the phases ToyOS no longer runs; the bound stands.
- std: a non-zero timeout under a millisecond is one millisecond. The
  protocol reads 0 as no bound, so a 500 us read timeout truncated to 0
  waited forever, on a datagram receive and on TCP's read, write and
  connect timeouts alike. `udp_recv_timeout` adds a 500 us receive; with
  the truncation restored it stalls at its hang ceiling.
- osbench-lan: `download` no longer counts the first read's bytes, which
  arrive before its clock starts.
- osbench-rules: `ASSUMED_STOP_MC` is derived from the table its doc names.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- std: `timeout_ms`, the one conversion every ToyOS socket timeout and
  `connect_timeout` takes, refuses `Some(Duration::ZERO)` with
  `io::Error::ZERO_TIMEOUT`, as std's other platform layers do; the last
  commit's clamp had made a zero timeout a silent 1 ms one, and before it a
  wait without bound. `udp_recv_timeout` asserts `InvalidInput` from
  UdpSocket's two setters and from TcpStream::connect_timeout.
- crunch: a cool-down with no starting temperature says so and returns
  rather than idling its whole bound.
- osbench-lan: the inline comment that repeated `download`'s doc goes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nd stops when it goes quiet

A sensor found at selection that then read nothing made `cool` return at
once, and the all-core phase ran with a watcher that skipped every failed
reading: unguarded, where the owner asked that the laptops not be
overheated. The comment that said the next phase's stop still guarded it
was false.

- The group's first successful reading gates the all-core phase. With
  none, only the one-thread phase runs and every `crunch.all.*` is
  `unmeasured`, as on a system that reads no temperature.
- `cool` takes that start reading as a value; it has no unread case.
- The phase watcher stops the phase at a failed reading as at the limit,
  and the result says which (`stopped=no|hot|unread`), so a sensor that
  dies mid-phase leaves nothing running unguarded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu
Japabu marked this pull request as ready for review October 24, 2026 09:07
@Japabu
Japabu added this pull request to the merge queue Oct 24, 2026
Merged via the queue into main with commit eba9e8e Oct 24, 2026
8 checks passed
@Japabu
Japabu deleted the wt/linux-compare branch October 24, 2026 09:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant