From e42dd7ef71593ffe80f7e7f1fbb78ca477952969 Mon Sep 17 00:00:00 2001 From: japabu Date: Tue, 20 Oct 2026 23:00:35 +0200 Subject: [PATCH] issues: user programs use AVX under XSAVE is done, and the records stop saying otherwise The track landed in #859 (XCR0 and CR4.OSXSAVE from control_regs' declaration; each CPU saves by XSAVES, XSAVEC or XSAVEOPT, the owner's ruling there) and #871 (the save at the switch). Its exit, item by item: - Threads with different vector contents each find their own after a switch: vector_state on the T14 (XSAVES, Width(Avx512)) and the AMD bench (XSAVEC, Width(Avx)) exited 0 at #871's merged head cdd090baa (ToyOS#871 comment 6127208453). Since then fpu.rs, switch.rs, entry.rs and vector_state are unchanged; xsave.rs and control_regs.rs changed in #873 and #915 without changing either bench's XCR0 or form. Under TCG, vector_state_survives_a_switch passed again at 4b8684f6f: "5 threads each held Width(Avx) through 3 rounds". - The EPYC KVM proving machines: merge-queue run 38725606232, job 116394005489 (guest / suite under KVM on an AMD EPYC 9V45, at 45b66f8d6, whose diff to 4b8684f6f is empty over fpu.rs, switch.rs, entry.rs, xsave.rs, control_regs.rs, sched/, build/arch.rs and the test): "vector_state: 5 threads each held Width(Avx512) through 3 rounds", PASS. It is one EPYC form: the 9V45 is outside erratum 1386's families, and its save form is not in the job log; a family-19h EPYC guest saves with XSAVEC since #915 and has no reading here. - Mutations: the save mask short of YMM (XSAVEOPT eax=3) under TCG went red again at 4b8684f6f, "ymm0 changed while it was switched out" in four threads, applied and restored in the same run. Short of Hi16_ZMM (eax=0x67) on the T14 went red at zmm16 in #859 (365bd213b), when the save still sat in the entry bracket; the mask is the same operand of the same instruction at the switch now, not re-run there. - The T14's ChaCha20-Poly1305 figure: #932's parity boots, with the ISA each build recorded (avx, avx2, avx512f/vl on the T14). The citing records drop the slug, and ap-control-registers-inherit-init no longer says OSXSAVE is asserted clear (it is in CR4_REQUIRED) or counts the CR4 bits its gate names. Co-Authored-By: Claude Opus 5.5 --- issues/ap-control-registers-inherit-init.md | 15 ++++++------- ...in-the-tree-on-par-with-ring-and-secure.md | 2 -- ...s-what-modern-hardware-offers-for-speed.md | 3 --- .../user-programs-have-no-protection-keys.md | 5 ++--- issues/user-programs-use-avx-under-xsave.md | 22 ------------------- 5 files changed, 9 insertions(+), 38 deletions(-) delete mode 100644 issues/user-programs-use-avx-under-xsave.md diff --git a/issues/ap-control-registers-inherit-init.md b/issues/ap-control-registers-inherit-init.md index aa3a6354190..977cbf138ab 100644 --- a/issues/ap-control-registers-inherit-init.md +++ b/issues/ap-control-registers-inherit-init.md @@ -11,7 +11,7 @@ opened: 2026-08-09 and `PG` into, so cores 1..N ran with `CD`/`NW` set — caching off — and with `WP`, `NE` and `MP` clear. Both registers are now written whole from one declaration, applied on every CPU and asserted there, and the `control_regs` -gate holds every bit of `CR0` and eight of `CR4` against that declaration +gate holds every bit of `CR0` and `CR4` it names against that declaration independently of the kernel's own self-check. What that leaves is a number nobody has: **how much an uncached AP cost.** It is @@ -61,13 +61,12 @@ on a host that prices `CD` at zero it would answer the same on both. ## What is *not* open, recorded because three places used to say it was **`CR4.OSXSAVE` diverging between the BSP and the APs** — the hypothesis -that firmware leaves it set so cpu0 permits AVX and an AP `#UD`s on it, -killing a thread that migrates. The -declaration is written whole rather than OR'd, so `write_cr4` clears the bit on -the BSP as well; it is in neither `CR4_REQUIRED` nor `CR4_OPTIONAL`, the gate -asserts it clear on every CPU by name, and a second arm refuses any bit the gate -never named. The divergence is unrepresentable on the T14 as much as here, so -that hypothesis needs no machine. +that one CPU permits AVX and another `#UD`s on it, killing a thread that +migrates. The declaration is written whole rather than OR'd; `OSXSAVE` is in +`CR4_REQUIRED`, every CPU asserts it set and asserts the `XCR0` the BSP +declared, the gate checks it set on every CPU by name, and a second arm refuses +any bit the gate never named. The divergence is unrepresentable on the T14 as +much as here, so that hypothesis needs no machine. **Whether `CR0.NE` clear was *the* cause of `fault_gates`' one `0xb881` survivor** — an unmasked x87 exception signalled on FERR# instead of raising diff --git a/issues/toyos-crypto-is-in-the-tree-on-par-with-ring-and-secure.md b/issues/toyos-crypto-is-in-the-tree-on-par-with-ring-and-secure.md index d8f274c3247..f8d8fbbc98a 100644 --- a/issues/toyos-crypto-is-in-the-tree-on-par-with-ring-and-secure.md +++ b/issues/toyos-crypto-is-in-the-tree-on-par-with-ring-and-secure.md @@ -157,8 +157,6 @@ Not consumers: - **Exit:** - On both benches, every row a consumer uses passes on the ISA each build records. That ISA is now AVX-capable state on both benches. - `issues/user-programs-use-avx-under-xsave.md`'s remaining exit, - userland threads' state, is not a gate here. - Parity work reads ring's, BoringSSL's and OpenSSL's perlasm or C only through a clean room, as root's research rule says. - Faster code is accepted only with stage 4's evidence for it. diff --git a/issues/toyos-uses-what-modern-hardware-offers-for-speed.md b/issues/toyos-uses-what-modern-hardware-offers-for-speed.md index eb8fda40976..753202fabe7 100644 --- a/issues/toyos-uses-what-modern-hardware-offers-for-speed.md +++ b/issues/toyos-uses-what-modern-hardware-offers-for-speed.md @@ -79,8 +79,6 @@ The CPU cap is PR #865's (`wt/toyos-cpus`), a design round of its own. **Exit**: every issue below is closed. They are listed in the order they are taken: the instruments, then what is measured, then the design rounds not yet measured. -`issues/user-programs-use-avx-under-xsave.md` is PR #859's, built ahead of -that order. - `issues/no-test-can-hold-a-thread-on-a-named-cpu.md` - `issues/no-program-measures-toyos-against-linux-on-one-machine.md` @@ -89,7 +87,6 @@ that order. - `issues/a-wake-storm-costs-the-t14-twice-the-amd-benchs-time-per-waiter-and-nothing-says-why.md` - `issues/idle-is-hlt-so-every-wake-of-an-idle-cpu-is-an-ipi.md` - `issues/the-timer-is-a-lapic-count-re-armed-every-pass-never-a-tsc-deadline.md` -- `issues/user-programs-use-avx-under-xsave.md` - `issues/no-gate-decodes-kernel-elfs-instructions.md` - `issues/kernel-forward-copies-and-fills-are-one-rep-movsb-or-stosb-on-every-cpu.md` - `issues/the-direct-map-uses-1-gib-leaves-where-the-cpu-has-them.md` diff --git a/issues/user-programs-have-no-protection-keys.md b/issues/user-programs-have-no-protection-keys.md index 38ba663b6c6..69fe0f7b97c 100644 --- a/issues/user-programs-have-no-protection-keys.md +++ b/issues/user-programs-have-no-protection-keys.md @@ -10,9 +10,8 @@ opened: 2026-09-29 part of its own memory off from itself. Linux at `Ubuntu-6.8.0-142.142` builds with `X86_INTEL_MEMORY_PROTECTION_KEYS` (`debian.master/config/annotations:15334`). Its ABI is approved in principle -and shaped in this issue's pull request, which lands after -`issues/user-programs-use-avx-under-xsave.md`, since PKRU is XSAVE -state. +and shaped in this issue's pull request. PKRU is XSAVE state, which +`kernel::xsave::xcr0` does not yet enable. **Exit**: under TCG with `+pku` (`target/i386/cpu.c:992` at QEMU v11.1.1), on the T14, and on each EPYC guest that enumerates PKU: a write under a key the diff --git a/issues/user-programs-use-avx-under-xsave.md b/issues/user-programs-use-avx-under-xsave.md deleted file mode 100644 index f8791f03396..00000000000 --- a/issues/user-programs-use-avx-under-xsave.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -status: open -kind: track -opened: 2026-09-29 ---- - -# User programs use AVX to AVX-512 under XSAVE - -`CR4.OSXSAVE` stays clear and a thread's state is its `FXSAVE64` image -(`kernel/src/arch/x86_64/fpu.rs:6-7`), so AVX is `#UD` in every user program. -The kernel enables every vector component the CPU enumerates and saves them -by one path on every CPU, XSAVEOPT, refusing by name a CPU without it: TCG -implements neither XSAVEC nor XSAVES (`target/i386/cpu.c:1012-1014` at QEMU -v11.1.1), and the CET state of -`issues/user-programs-run-without-a-shadow-stack.md` is switched by -`wrmsr`, as `fs_base` is. The TCG model gains `+xsave,+xsaveopt,+avx,+avx2`, -which TCG implements (`cpu.c:905,907,981,1012`). - -**Exit**: on every proving machine and under TCG, threads that fill their -vector registers differently each find their own after a switch; the T14's -ChaCha20-Poly1305 figure. **Mutation**: a save mask short of Hi16_ZMM, and -under TCG of YMM. **Oracle**: TCG and the T14 on SDM Vol. 1 §13.