[Japabu]

issues/user-programs-use-avx-under-xsave.md is done and this deletes it. The track landed in #859 and #871, but the issue was never closed. It still said CR4.OSXSAVE stays clear and AVX is #UD. This PR changes records only, no code.

What landed (on main before this PR)

• kernel/src/arch/x86_64/control_regs.rs declares CR4.OSXSAVE (in CR4_REQUIRED) and XCR0 as a single declaration. Each CPU computes XCR0 from its own CPUID (kernel::xsave::xcr0: x87, SSE and AVX required, all of AVX-512 or none), the BSP publishes it, and every CPU asserts it.
• kernel/src/arch/x86_64/fpu.rs saves at the switch and restores there. The area is sized from CPUID leaf 0xD and sits above the kernel stack's top. A new thread starts by load_initial.
• The x86-64 QEMU models have +xsave,+xsaveopt,+avx,+avx2 (build/arch.rs).
• The save form differs from the issue text. The issue says XSAVEOPT on every CPU. In User programs use AVX to AVX-512, saved by the best XSAVE form each CPU offers #859 the owner ruled that each CPU uses the best form it offers: XSAVES, then XSAVEC, then XSAVEOPT, and a CPU without XSAVEOPT is refused by name. The T14 saves with XSAVES, the AMD bench with XSAVEC (erratum 1386), and TCG with XSAVEOPT. This PR does not reopen that ruling.

The issue's exit, item by item

exit item	evidence	at
T14: threads with different vector contents each keep their own across a switch	vector_state judge exit 0, Width(Avx512), XSAVES (#871 comment)	cdd090baa, #871's merged head
AMD bench: the same	vector_state judge exit 0, Width(Avx), XSAVEC (same comment)	cdd090baa
TCG: the same	vector_state_survives_a_switch exit 0, "5 threads each held Width(Avx) through 3 rounds" (log L147)	4b8684f6f (this PR's base), run for this PR
EPYC KVM guest: the same	vector_state_survives_a_switch PASS, "5 threads each held Width(Avx512) through 3 rounds", under KVM on an AMD EPYC 9V45 (merge-queue run 38725606232, job 116394005489, L452 CPU, L699–L700 result). This is one EPYC form only: the 9V45 is outside erratum 1386's families, and the job log does not record its save form. A family-19h EPYC guest (7763, 9V74) has saved with XSAVEC since #915 and has no reading here.	45b66f8d6 (#945's queue commit); git diff 45b66f8d6 4b8684f6f is empty over fpu.rs, switch.rs, entry.rs, xsave.rs, control_regs.rs, sched/, build/arch.rs and the test
Mutation: save mask short of YMM, TCG	XSAVEOPT eax=3 (patch), applied, run and restored in one command: exit 1, "ymm0 changed while it was switched out" in four threads (log L15)	4b8684f6f + patch, run for this PR
Mutation: save mask short of Hi16_ZMM, T14	XSAVES eax=0x67: EXIT=1, seven threads red at zmm16 (#859 body, Checks)	365bd213b (#859)
The T14's ChaCha20-Poly1305 figure	#932's parity table: ChaCha20-Poly1305 13 rows, T14 median ratio 4.74 against ring, recorded ISA avx,avx2,…,avx512f,avx512vl (issues/toyos-crypto-is-in-the-tree-on-par-with-ring-and-secure.md)	#932

Why the earlier metal readings still hold. Between cdd090baa and 4b8684f6f, git diff shows no change to fpu.rs, switch.rs, entry.rs, vector_state.rs or arch/vector.rs. xsave.rs and control_regs.rs changed in #873 (CR4.PGE) and #915 (the erratum moved in-tree and widened to more Zen models). Neither change alters either bench's XCR0 or save form. #915's body says so for the AMD bench (model 60h, already XSAVEC), and the T14 is Intel, where the erratum does not apply. sched/ changed only in keyboard-controller polling and a task's release list.

One mutation reading is older and was not re-run. The Hi16_ZMM mutation went red on the T14 when the save was still in the entry bracket, before #871 moved it to the switch. The mask is the same operand of the same instruction, now in fpu::save. #871's stack-slot mutant also went red on the T14 at the current save site ("zmm21 changed"). I judge the Hi16_ZMM control still valid; that is an inference, not a new measurement. If the orchestrator wants it re-measured, it is a single T14 boot of vector_state with eax = 0x67 in the Form::Xsaves arm of fpu::save.

Changed records

• Deleted issues/user-programs-use-avx-under-xsave.md. It carries no durable rule that is not already in fpu.rs's and control_regs.rs's headers.
• issues/toyos-uses-what-modern-hardware-offers-for-speed.md: removed it from the exit list.
• issues/toyos-crypto-is-in-the-tree-on-par-with-ring-and-secure.md: removed the "remaining exit" sentence.
• issues/user-programs-have-no-protection-keys.md: the ordering dependency is replaced by the remaining fact: PKRU is XSAVE state that kernel::xsave::xcr0 does not yet enable.
• issues/ap-control-registers-inherit-init.md: it said OSXSAVE is in neither CR4_REQUIRED nor CR4_OPTIONAL and is asserted clear. It is now required and asserted set, and the divergence is still unrepresentable. Its count of the CR4 bits the gate names is dropped.

git grep user-programs-use-avx at the head finds nothing.

User-space feature detection (report, nothing changed)

With OSXSAVE set, these paths are open on ToyOS. They have been since #859 (f7b497231), not since this PR:

• std_detect (library/std_detect/src/detect/os/x86.rs, not OS-specific): reads XCR0 through xgetbv. xcr0 & 6 == 6 admits AVX, AVX2, FMA, F16C, VAES and VPCLMULQDQ. xcr0 & 0xe0 == 0xe0 admits the AVX-512 family on the T14. AMX stays unreported, because XCR0 bits 17 and 18 are clear.
• ring 0.17.14 fork (crypto/cpu_intel.c, not OS-specific): (xcr0 & 6) == 6 keeps AVX and everything gated on it, and (xcr0 & 0xe6) == 0xe6 keeps AVX-512. The paths taken: ChaCha20 ChaCha20_ctr32_avx2, the AVX2 and BMI2 integrated ChaCha20-Poly1305, AES-GCM's clmulavxmovbe and, where VAES and VPCLMULQDQ exist (the T14), VAesClMulAvx2.
• lib/crypto (another agent's branch; not touched): arch/x86_64/avx2.rs gates ChaCha on CPUID and xgetbv(0), and vaes.rs on is_x86_feature_detected!. Both are open on both benches. The ring comparator: toyos-crypto beside ring in a guest, on metal rows (crypto stage 2) #932 recorded those ISAs.

Checks

check	revision	exit	log
cargo test --test toyos-build -- vector_state_survives_a_switch (x86-64 under TCG on an AArch64 host)	4b8684f6f	0	tcg-main.log
the same with the YMM mask mutation, restored after	4b8684f6f + patch	1 (expected red)	tcg-ymm-mutation.log
cargo run -- --ci host	a3a5fe1e6 tree (the amend to e42dd7ef7 changes one markdown line and the commit message)	0, "87 step(s), all green"	ci-host.log L11613

The PR changes only markdown under issues/, so no image build and no metal run is needed.

Risks

• The EPYC KVM reading covers only the 9V45. Family-19h EPYC guests (XSAVEC since toyos-cpuvuln leaves the tree; what the kernel read of it stands on Intel's and AMD's documents #915) have no vector_state reading. The nightly's kvm / suite lane is new with ci: the merge queue builds both images, the guest suite runs nightly, documentation alone builds nothing #954 (2026-10-20); its first runs give the next EPYC reading, on whatever EPYC form the runner draws.

• The Hi16_ZMM metal control is from before the save moved to the switch (above).

• The ChaCha "figure" item is read as satisfied by The ring comparator: toyos-crypto beside ring in a guest, on metal rows (crypto stage 2) #932's measured figure with AVX-512 state enabled. The issue set no threshold for it.

🤖 Generated with Claude Code

[Japabu]

Review of #959 at a3a5fe1e6 (base 4b8684f6f)

Assessment. This PR changes only markdown, +8/−37. It closes issues/user-programs-use-avx-under-xsave.md and drops the three citations of its slug. git grep user-programs-use-avx origin/main finds only those three files, so after a merge nothing cites the slug. The issue holds no durable rule that is missing from the code. Per-CPU form selection, the refusal of a CPU without XSAVEOPT and the area's invariants are in kernel/pure/xsave.rs, fpu.rs and control_regs.rs. Deletion is the right close under issues/README.md, and the PR correctly does not reopen #859's owner ruling on the save form. Every edited sentence is true against origin/main:

• ap-control-registers-inherit-init.md: OSXSAVE is in CR4_REQUIRED (control_regs.rs:147). self_check asserts every CPU's CR4 and XCR0 against the BSP's declaration (DECLARED_XCR0, xsave_declaration). The control_regs gate in tests/toyos.rs names (18, "OSXSAVE", true), and its named arm refuses any bit it does not name.
• user-programs-have-no-protection-keys.md: kernel::xsave::xcr0 (kernel/pure/xsave.rs:138) does not take PKRU. The module header and the test at :284 both say so.
• The speed-track and crypto-track deletions leave the text around them coherent.

I checked the evidence myself:

• User FP/vector state is saved only at a switch, the preempt count drops lock, and the build refuses stray vector instructions #871's metal logs at evidence 8c11775aa: the T14 has xcr0=0xe7 save=xsaves and "17 threads each held Width(Avx512)", exit 0. The AMD bench has xcr0=0x7 save=xsavec and "Width(Avx)", exit 0.
• Evidence db52c393c: tcg/revision.txt is 4b8684f6f. The main run says "5 threads each held Width(Avx)", EXIT=0. The YMM mutant (eax 3 in the Xsaveopt arm only) gives ymm0 changed in four threads, EXIT=1.
• The ring comparator: toyos-crypto beside ring in a guest, on metal rows (crypto stage 2) #932's table in the crypto track gives the T14's ChaCha20-Poly1305 at 4.74, with avx512f,avx512vl recorded.

The Hi16_ZMM control. Citing it is sound, and no fresh T14 boot is needed. Under the mutant, XSAVES writes XSTATE_BV = XINUSE & RFBM (SDM Vol. 1 §13.11), so bit 7 is clear in the area. fpu::restore still runs xrstors64 with a full RFBM, and §13.12 then puts Hi16_ZMM in its initial state. The thread therefore reads zmm16 changed wherever the save sits. The red depends only on the save's operand and the restore's mask, and #871 changed neither. #871's stack-slot mutant also went red at zmm21 on the T14 at the current save site. That shows vector_state reaches Hi16_ZMM through today's switch path. The PR body correctly labels this an inference. Stating the instruction-semantics reason would make it stronger than "same operand of the same instruction", but that is optional.

Findings

BLOCKER — PR body "The issue's exit, item by item" table, and the commit message's exit list: the EPYC KVM proving machines are missing. The exit reads "on every proving machine and under TCG". This issue was a stage of issues/toyos-uses-what-modern-hardware-offers-for-speed.md, and that track's lines 15–17 make "AMD EPYC KVM guests" proving machines and say "a stage's correctness test runs on each that has its feature". The table and the commit only cover the T14, the AMD bench and TCG, so as written the closing record does not show the exit is met.

The evidence exists, so this is a citation fix, not a new run. Merge-queue run 38725606232, job 116394005489 (guest / suite, kvm: true, at 45b66f8d6, #945's queue commit on top of this PR's base):

• L452: "AMD EPYC 9V45 96-Core Processor"
• L699: [claims] vector_state: 5 threads each held Width(Avx512) through 3 rounds
• L700: PASS vector_state_survives_a_switch

git diff 4b8684f6f b00e91506 shows no change to fpu.rs, switch.rs, kernel/pure/xsave.rs, control_regs.rs, vector_state.rs, sched/ or build/arch.rs. Add that row to the PR body (the orchestrator's) and to the commit message (the implementer's). Note that #915 widened erratum 1386 to families 17h–19h, so a family-19h EPYC guest (7763 or 9V74) now saves with XSAVEC, while the 9V45 above is outside that range. One KVM row is enough for the stage. It is just not a reading for every EPYC form.

SUGGESTION — issues/ap-control-registers-inherit-init.md:15. "the control_regs gate holds every bit of CR0 and eight of CR4" is stale. CR4_BITS now names ten bits (DE, PAE, MCE, PGE, OSFXSR, OSXMMEXCPT, LA57, FSGSBASE, OSXSAVE, SMEP), plus three in CR4_MAY. The PR touched the OSXSAVE paragraph of this same record. Dropping the count ("holds every bit of CR0 and CR4 that it names") avoids having to recount next time.

Observation, outside this PR's scope. The last nightly runs (39390441510, 39221595037, 39108067326 and 38379461041) list no kvm / suite job, although nightly.yml declares one. After #954 the merge queue no longer runs the guest suite either. So no run since 2026-10-15 has given fresh EPYC evidence. That is worth a look by whoever owns CI, but it does not block this PR.

Evidence sufficiency. The PR changes only markdown, so no image build or metal run is needed. Reusing the metal and KVM readings is justified by the unchanged diffs above. The --ci host run on the staged tree is enough for the markdown checks.

SEND BACK. The one blocker is to cite the EPYC KVM row in the commit message and the PR body.

[Japabu]

Review round 1, now at head e42dd7ef7 (the commit was amended and force-pushed to this draft's own branch):

• BLOCKER (EPYC KVM proving machines): fixed. The commit message and the PR body's exit table now cite merge-queue run 38725606232, job 116394005489 (KVM, AMD EPYC 9V45, vector_state: 5 threads each held Width(Avx512) through 3 rounds, PASS). I confirmed that git diff 45b66f8d6 4b8684f6f is empty over fpu.rs, switch.rs, entry.rs, xsave.rs, control_regs.rs, sched/, build/arch.rs and the test. Both places carry the caveat: this is one EPYC form, the 9V45 lies outside erratum 1386's families, its save form is not in the job log, and family-19h guests (XSAVEC since toyos-cpuvuln leaves the tree; what the kernel read of it stands on Intel's and AMD's documents #915) have no reading. The missing nightly kvm / suite job is listed under Risks.
• SUGGESTION (CR4 count): taken. Line 15 now reads "holds every bit of CR0 and CR4 it names".

No re-run: the amend changes one markdown line, and the --ci host result from before it still covers the markdown checks.

🤖 Generated with Claude Code