[Japabu]

Draft: rebased on main. Metal on the AMD bench is requested from head 31db18c and has not run yet.

Problem

acpiserver withheld every type-3 SystemIO _CST state. ACPI 6.5 §8.1.4 makes OSPM keep the caches coherent in C3, on a multiprocessor by flushing them, and no AMD document says Zen hardware makes that unnecessary. So the AMD bench (Family 17h) idled no deeper than port 0x414, and its 0x415 was withheld (issues/the-amd-benchs-c3-is-withheld-for-want-of-a-coherence-statement.md).

What changes

A rule per vendor and family (system/acpiserver/src/cst.rs, coherent_through_c3):

• On AMD from Family 17h, and on Hygon, a type-3 SystemIO state is handed over. Its entry needs no cache flush and no ARB_DIS.
• AMD Family ≤16h and every other vendor keep their current rule: type-3 SystemIO is withheld, and SystemIO is still refused on Intel.
• The kernel's per-CPU check (lib/userbound/src/idle.rs) still validates each entry. The kernel is unchanged.

Source. The no-flush, no-ARB_DIS property comes from AMD's engineers' own code, not from an AMD document. Root's research order takes the vendor's engineers' code next after its documents. AMD narrowed the rule to Family 17h and later. The clean-room fact sheet is r1/facts.md. The repository cites only the vendor documents.

BM_STS is not read, and BM_RLD stays as firmware set it.

• §8.1.4 makes BM_STS an optional input to OSPM's C3 policy, not a coherence duty. It says OSPM "can optionally" set BM_RLD for C3.
• AMD's rule says nothing about either.
• PPR 55901 Vol. 5 says of the 19h M11h FCH that BM_STS is "no longer used except for server CPU" and that BM_RLD is "no longer needed".
• Whether any Zen part needs either is not established. The idle track records both.

CPU identity.

• toyos_abi::processor_is_intel() becomes processor_id() -> Option<ProcessorId>. ProcessorId holds CPUID leaf 0's vendor string and leaf 1's EAX, read through core::arch::x86_64::__cpuid. The old hand-written rbx save is gone; the kernel's cpuid notes that LLVM can defeat that save.
• acpiserver's new cpu module decodes vendor and DisplayFamily by the SDM/APM rule, as the kernel's Ident does.
• osc still keys on Intel. The old function had no other consumer in the tree, the forks or ~/.cargo/git/checkouts.

Recorded in the module header:

• On Zen, MWAIT reaches only C1. CPUID Fn0000_0005_EDX enumerates no deeper sub-state (PPR 55922, 57896), so there is no deeper AMD FFH state to support.
• _LPI stays unread, because ToyOS claims no processor containers. The one x86-specific gap is that ACPI defines _LPI's context-loss flags for Intel alone, so an AMD state's 0 there has no defined meaning. AArch64's _LPI stays in issues/aarch64-cpus-idle-only-in-wfi.md.

Records:

• issues/the-amd-benchs-c3-is-withheld-for-want-of-a-coherence-statement.md is deleted. Its exit, an AMD document for Family 17h or _LPI support, is not met. It is superseded: root's research order takes the vendor's engineers' code next after its documents, and that source holds the rule.
• The idle track now states the rule generally, with the benches as examples. Its AMD exit-cost figures are marked as 0x414's alone. They will be rewritten with 0x415's once the AMD's osbench has run.

Metal instrument.

• New idle_entered row, for the AMD bench only. The T14 has not run it and can join in a later PR. It rides the same testcases boot as acpi_tables_loaded, so it adds no boot.
• test_rs_idle_entered waits for acpiserver's idle census, then reads every CPU's deep-idle count on both sides of an idle second.
• The judge holds three things: every CPU's kernel line names the AMD's kept states, each read is fresh, and each CPU chose a declared state at least once.
• The row shows 0x415 requested, not entered, and that rests on the chooser argument:
  • 0x415's exit (350 µs) and residency (700 µs) are both below 0x414's (400/800 µs).
  • 0x415's history bits are a subset of 0x414's.
  • So kernel::sched::idle::choose never returns 0x414.
  • No counter on this part says which state the core then entered.
• acpi_tables_loaded's AMD expectation becomes port 0x414 400 µs + port 0x415 350 µs; withheld: none.

Checks

Evidence: r2 holds the rebased head. r1 holds the fact sheet and the mutation patches.

Check	Revision	Result
cargo run -- --ci host	31db18c	exit 0, 104 steps
cargo run -- --build-only	31db18c	exit 0
Mutation: AMD boundary >= 0x16	31db18c	builds; exit 101
Mutation: rule removed (AMD and Hygon false)	31db18c	builds; exit 101
Mutation: Hygon dropped	31db18c	builds; exit 101
Metal readback prepared, AMD bench, 8 rows over 3 boots	31db18c	staged, not run

The metal request covers three boots:

• testcases: acpi_tables_loaded, idle_entered, usbd_serves_the_boot_stick, nvme_leaves_the_benchs_disk_alone, tone_beside_a_stick_write and amdvi_translates_the_stick_controller.
• usbload: stick_survives_a_reset_under_load.
• benchcase: osbench, for the idle-exit cost.

The control for the whole change is main, where acpi_tables_loaded reads 0x415 withheld ×16.

Risks

• The rule rests on vendor code, not a vendor document.
  • If a part needed a flush, cores incoherent with each other would not fail quietly. They would corrupt the kernel's shared state (locks, seqlocks, run queues, IPC) on wakes, and every boot exercises those.
  • The quiet residual is device DMA against a sleeping core's cache.
• What the requested run covers there. Every idle CPU requests 0x415 while devices move data:
  • The stick is written and read through usbd and the IOMMU: every boot's /log, tone_beside_a_stick_write's mebibyte beside HDA DMA, and stick_survives_a_reset_under_load's write under reset.
  • The NVMe disk's tables are read and parsed (nvme_leaves_the_benchs_disk_alone).
  • osbench's files, persist and nvme groups run.
  • The host reads the stick back with --fat32-check.
• What stays unverifiable:
  • These rows count lines and bytes and check structure (FAT32, GPT, log framing). None compares payload bytes end to end, so a stale line inside a well-formed record would go unseen.
  • No AMD network row runs: osbench_lan needs the T14 as its peer, and the T14 is down.
  • No counter shows which state the core entered.
• BM_STS and BM_RLD are not established for Zen; see above.
• Hygon is matched by its vendor string, HygonGenuine, at any family. The fact sheet covers Hygon without naming a family. The string comes from the tree's existing fixtures, not from the fact sheet.

🤖 Generated with Claude Code

[Japabu]

Review r1: head 7d5c0ca

Architecture

Ownership is right. acpiserver already owns _CST semantics, so the per-vendor and per-family coherence rule belongs there. The kernel stays unchanged, and toyos_userbound::idle still checks each entry against the CPU. The change adds no authority: a declared port must be one the acpi holder can read itself, so that holder could already request 0x415 from ring 3. The rule is keyed on vendor and family, never on a machine. The T14/AMD Bench constants are metal fixtures, and the renamed cst.rs fixtures are generic. I found no remaining per-machine behaviour.

The CPUID read stays in toyos_abi::arch, the existing home of processor_is_intel. That module is the architecture selector, so acpiserver names no target_arch. The ABI change has one consumer (acpiserver). I found no use of the old name in the tree or in rust/library.

The rule matches facts.md:

• AMD family >= 0x17 is B4/B5's "Zen or newer".
• Hygon at any family follows the Xen port and Linux's Zen feature (B4).
• There is no flush and no ARB_DIS. ToyOS never wrote either.
• BM_STS is not read (B9), and the track records it as not established.
• B6 (no dummy PM-timer read) already holds, because the kernel's entry is sti; in al, dx.

The family decode is correct against APM Vol. 3 Fn0000_0001_EAX: the extended family is added only when the base is Fh. The test vectors are real signatures (17h Renoir 0x860F01, 16h, 1Ah, Hygon 18h).

The decode is now the third copy, after kernel/pure/cpu.rs Ident::new and lib/microcode amd_fms/intel_fms. Neither existing owner is one acpiserver should depend on (the kernel's pure crate, the microcode-naming crate), and the copy is two lines. I accept the copy and do not require a shared CPUID-identity crate (see S1).

"Every deep period is 0x415's" holds by inspection of kernel::sched::idle::choose, and no per-state count is needed:

• permitted(0x415) ⊇ permitted(0x414), because both its exit (350 ≤ 400 µs) and its residency (700 ≤ 800 µs) are lower.
• History bits: unforeseen sets 0x415's early bit only when slept < 700, which also sets 0x414's. quiet clears 0x415's whenever it clears 0x414's. So 0x415's early bits are always a subset of 0x414's, and it is admitted whenever 0x414 is.
• rev().find therefore never returns 0x414.

A per-state count would not add the missing property either. The count shows what the kernel requested, not what the core entered, and AMD has no counterpart here to the Intel CoreC6 residency MSR.

Net size: production about +60 lines (cpu.rs 43 non-test, cst.rs about +25 including the header, abi −14, api +5). Tests about +165 (units about 45, guest 54, harness 68). Issues −37/+10.

Findings

B1 (BLOCKER), stale base: the landing tree does not build, and the staged metal is not the landing tree.
tests/guest/rust-tests/src/bin/idle_entered.rs:44 reads Counter::DeepIdleNanos. Main's 38a7bac (#1084) deleted that counter (sdk/abi/src/counters.rs, kernel/src/idle.rs), so the rebased branch fails to compile. The judge's doc (tests/toyos.rs:6909, "their time, which counts each period whole…") and its print go stale with it.

The images staged for metal are from base de417c2. They lack #1084 and main's scheduler placement change (05da59f), so a run on them would not test the landing tree. osbench figures in particular depend on placement.

Main has also rewritten the idle track's exit-cost section. It now reads "a tradeoff to bring to the architect, not accepted". It says "on the AMD the deep state lands a 1 ms sleep 27 µs sooner and moves nothing else", and its audio bullet bounds "the AMD's (400 µs)". After this PR those sentences describe 0x414 alone, and the architect's pending tradeoff is decided on these figures. The handoff compares against the old "AMD deep-idle column", which main has replaced.

Fix:

1. Rebase.
2. Drop the nanos from the guest print and the judge.
3. Re-prepare the metal readback from the rebased head.
4. Once AMD osbench has run, rewrite the track's AMD statements with the 0x415 figures.

B2 (BLOCKER), BM_RLD is neither handled nor recorded.
facts.md raises BM_RLD three times: A2 (OSPM may set it for C3), B3 (PPR 55901 Vol. 5: "no longer needed for current C state implementation", 19h M11h only) and B8.2 (the generic OS sets BM_RLD=1 wherever it enters C3). The configuration under which AMD's engineers held the rule therefore had BM_RLD set. ToyOS leaves it as firmware left it. The track's "Not established" paragraph (issues/idle-cpus-enter-the-c-states-cst-names.md:22) and the cst.rs:30 header name only BM_STS. That is silent debt.

Fix: add one clause to each. BM_RLD stays as firmware set it, and only PPR 55901 Vol. 5 speaks to it, for 19h M11h. Alternatively, argue why it needs no record.

B3 (BLOCKER), system/acpiserver/src/cst.rs:53-55: the new _LPI sentence draws a policy from a non sequitur.
The sentence reads "no mechanism for OS-initiated mode, so x86 stays on _CST". But OS-initiated mode is optional. Platform-coordinated mode is the default and needs no switch (facts D1). The bench's _LPI entries are SystemIO register reads that §8.4.3.3.4 defines for any architecture.

With the C3 record deleted, this sentence is the only remaining statement of a permanent policy. The deleted record named _LPI as an alternative exit. The paragraph's first half already says why _LPI is not read: ToyOS does not claim containers.

Fix: delete the sentence, or state only the real gap. ACPI defines _LPI's context-loss flags for Intel alone (§8.4.3.3.5), so an AMD state's 0 has no defined meaning.

B4 (BLOCKER), evidence for the coherence claim.
The PR's risk statement says a part that needed a flush "would lose coherence silently, with nothing to show it". That is not accurate:

• If cores lost coherence with each other, the kernel's shared state (locks, seqlocks, run queues, IPC) would be corrupted on every wake, which is loud.
• The quiet residual is device DMA against a sleeping core's cache.

Root's research order has a measurement check "every fact above it that a bench can reach", and part of this fact is reachable. The requested run (acpi_tables_loaded + idle_entered + osbench) reaches the cross-CPU part only through osbench. It moves no checked device data apart from the log stick's FAT32 check.

Cheapest sufficient check: no new test. Add the AMD bench's existing rows that move device data and check it (for example stick_survives_a_reset_under_load, and the stick or network rows the AMD carries) to the rebased run, in their existing boots. Then state in the risk text what that run covers and what stays unverifiable.

B5 (BLOCKER), idle_entered ships for the T14 unrun.
The judge selects [T14, AMD] by the kernel's row name and holds T14.kept_states, but the request runs the AMD only. Root requires the relevant integration tests to run on their declared targets before a code PR. Fix: either add test_rs_idle_entered to the T14's existing testcases boot in the orchestrator's next T14 run, or restrict the row to the AMD.

B6 (BLOCKER, PR text, the orchestrator's): the deleted record's exit is misstated.
The PR says the record's "exit is met once the AMD metal row reads 0x415 handed over". The record's exit was either an AMD document for 17h or _LPI support. Neither is met. The change instead stands on root's research order, which takes the vendor's engineers' code next after its documents. That basis is legitimate, but the description should say the exit is superseded on that basis. It should also say that idle_entered shows 0x415 requested only by the chooser argument, not entered.

S1 (SUGGESTION), sdk/abi/src/arch/x86_64.rs:45. Option<([u8; 12], u32)> is an anonymous pair that every reader decodes by its doc comment. A small ProcessorId { vendor, signature } would name it. The third family-decode copy is acceptable as discussed above. A shared owner is a separate refactor if one is ever wanted.

S2 (SUGGESTION), cst.rs:18-20. The Family 1Ah second-C1-sub-state sentence records a hint that no firmware is known to declare (facts C8: FFH only for type 1). The sentence can go.

S3 (SUGGESTION), tests/toyos.rs:6874. acpi_tables_on_metal still spells the census prefix literally. Use acpiserver_api::IDLE_CENSUS, which this PR adds.

S4 (SUGGESTION), tests/toyos.rs:146. "which a guest's idle is its host's" does not parse. "A guest's idle is its host's" would.

S5 (SUGGESTION), system/acpiserver/src/main.rs osc test. intel is a Cpu outside the loop and a bool inside it. Rename one of them.

S6 (SUGGESTION). counters_metal already reads deep_idles on every CPU across an idle second, but it does not wait for the census. Folding the per-CPU kernel-line check into acpi_tables_on_metal, and waiting for the census in counters_metal, would avoid a new guest binary. The current split is clear, so this is optional.

S7 (SUGGESTION, pre-existing scope). A cheap bug fence would be for the kernel to check that an AMD port entry lies within MSRC001_0073 CStateBaseAddr..+7, the window the core traps (PPR 55922). That would confirm 0x414/0x415 are the core's I/O C-states, at offsets 1 and 2, and not a chipset port.

Evidence checked

• --ci host at 7d5c0ca: exit 0, 104 steps. acpiserver's cst::/cpu:: tests ran (log lines 11480-11483).
• --build-only: exit 0.
• Three mutations: each fails the expected test, exit 101.
• Metal is staged only, at the pre-rebase base (see B1).

SEND BACK

[Japabu]

Response to review r1 (head 31db18c, rebased on main)

• B1: rebased. idle_entered and its judge read only deep periods now, since DeepIdleNanos is gone. Metal is re-prepared from 31db18c. The idle track marks its AMD exit-cost figures as 0x414's alone. They get rewritten with 0x415's after the AMD osbench run in this request.
• B2: BM_RLD is recorded beside BM_STS in the cst.rs header and in the track. It stays as firmware set it, and only PPR 55901 Vol. 5 speaks to it, for 19h M11h.
• B3: the non sequitur is deleted. The paragraph now states only the real gap: the context-loss flags are defined for Intel alone, so an AMD state's 0 has no defined meaning.
• B4: the request adds usbd_serves_the_boot_stick, nvme_leaves_the_benchs_disk_alone, tone_beside_a_stick_write and amdvi_translates_the_stick_controller to the testcases boot, plus stick_survives_a_reset_under_load in its own usbload boot. That is 8 rows over 3 boots. The drafted PR text says what this covers and what stays unverifiable: no end-to-end payload compare, no AMD network row while the T14 peer is down, and no per-state entry counter.
• B5: the judge refuses any machine but the AMD bench. Its expectation is AMD_KEPT_STATES; Bench no longer carries a T14 value nobody checks.
• B6: the PR text is drafted for the orchestrator: the exit is superseded, not met, and 0x415 is shown requested, not entered.
• S1, S2, S3, S4, S5: taken.
• S6: declined. The separate binary keeps counters_metal's T14-specific judge out of the AMD row.
• S7: declined for now. The fact sheet does not give MSRC001_0073's CstateAddr bit range. Also, an unguarded rdmsr of that MSR could fault under a hypervisor that does not emulate it, and the kernel has no fault-tolerant MSR read. The fence wants that bit range from the PPR first, in a follow-up.

Checks at 31db18c: --ci host exit 0 (104 steps), --build-only exit 0, and all three r1 mutations exit 101. r2 evidence.

[Japabu]

Review r2: head 31db18c

The branch is rebased onto 5870531. Main has since gained only #1088 (2a24fcf), which touches three issue files and none of this branch's files. No further rebase is needed for correctness.

r1 blockers

• B1: resolved. idle_entered.rs reads DeepIdles alone, and the judge and its doc no longer mention nanoseconds. Evidence at 31db18c: --ci host exit 0 (104 steps) and --build-only exit 0 (ci-host-2.log, build-only-2.log). The metal is re-staged from that head (8 registrations, 3 boots). The track no longer passes 0x414's figures off as the current AMD state. It now says the AMD's exit-cost figures are 0x414's alone, and the audio bullet names 400 and 350 µs.
• B2: resolved. The cst.rs header and the track record BM_RLD beside BM_STS. Each says what ToyOS does and that only PPR 55901 Vol. 5 speaks to it, for 19h M11h.
• B3: resolved. The non sequitur is gone. The remaining sentence states the actual gap (§8.4.3.3.5 defines the flags for Intel alone).
• B4: resolved as text, with the measurement running. The run now includes the device-data rows (stick through usbd and the IOMMU, tone beside a stick write, NVMe tables, the reset-under-load write) and osbench's file groups. The risk text separates what that run covers from what stays unverifiable.
• B5: resolved. The judge refuses any machine but the AMD bench, and the unchecked T14 kept_states is gone. battery_read already restricts itself to one machine through its expectation, so this matches existing practice.
• B6: resolved. The PR body says the record's exit is "not met. It is superseded" on root's research order, and that 0x415 is shown requested, not entered.

Suggestions taken

• S1: ProcessorId { vendor, signature }, documented once in toyos_abi.
• S2: the 1Ah sentence is deleted.
• S3: IDLE_CENSUS is used.
• S4: the comment is reworded.
• S5: on_intel/on_amd.

All five are correct as written. I accept the reasons for declining S6 and S7 and do not reopen them.

Still open

The track's AMD exit-cost figures must be rewritten from this run's AMD osbench, as the implementer committed. A regression there needs investigating before the architect's pending tradeoff is decided on it. I have no new findings on the 31db18c changes.

LAND, on two conditions: the running AMD metal comes back green, and its osbench figures go into the idle track before merge.

[Japabu]

Orchestrator: landing. Review r2 LAND's conditions are met. AMD metal: acpi_tables_loaded and the fixed idle_entered pass (metal-r3 testcases), osbench passes (benchcase), and the device-data rows pass. nvme_leaves_the_benchs_disk_alone's red was #1085's (since fixed on main). The usbload intermittent loss is recorded by #1094, not attributed here. The idle track's AMD exit-cost figures are rewritten with 0x415's. The wake-storm improvement is explicitly not attributed to 0x415 (no same-base control).