Repository navigation
vtd: every unit is handed over inside protected memory regions, and the T14 iGPU fault is measured - #1086
Conversation
Review r1: igpu-fault at 30a7071Spec used: VT-d Architecture Specification Rev. 4.1 (D51397-016), the revision the narrowed issue cites. I checked it against Intel's published PDF: §6.6, §6.5.2.11, §8.1 Flags, §3.16, §11.4.4.1 (GCMD.TE/SRTP), §11.4.5 and Table 26. Architectural assessmentThe hand-over belongs in
Size: +639/−62. Production Required findings1. BLOCKER:
Consequence: translation stays on through firmware's tables on a machine where the premise was never checked. This firmware input should fail closed at the boundary. Clear the flag unless:
Adopting proposal (2) below would delete the condition altogether. 2. BLOCKER: comments and records that state the inference, or the wrong section, as specification fact.
3. BLOCKER: the reproducer's design and the new issue's exit can misattribute the result (
4. BLOCKER for landing (the metal run is not affected): the measurement code must not land.
No durable claim needs this code:
Under root policy, exploratory comparisons are evidence, not regression tests. Put the counts and lines in the evidence repo and the issue. Evidence
SuggestionsS1. Proposal (2): drop the TE-off path for legacy-mode roots. Engineering read: in favour, owner to approve the §6.6 departure.
S2. Proposal (1): contain faults on unmapped addresses from undriven functions. Engineering read: sound in principle, but simpler than proposed, and not part of this PR.
S3. Record a related exposure. S4. Smaller notes.
SEND BACK |
r1 response (head 5fb75fc)
Evidence: https://github.com/ToyOSOrg/evidence/tree/42a790b5eccac6e66a1f1911aa1c511b3fa6db07 |
Review r2: igpu-fault at 5fb75fcBase: Architectural assessmentThe r1 fixes are done as asked. They do not make the keep-on branch landable, and the spec does not support it. The question that matters now is the one the architect raised: is it legitimate to keep translation on through tables firmware left behind? Here is what Rev. 4.1 actually says about a unit firmware left translating.
So nothing in the spec states that a firmware-left root is identity over its RMRRs. The head never reads firmware's structures, so its No machine runs the branch against real firmware tables.
That makes it production machinery whose only exercise is a test scenario, and whose premise in production is an inference. What the spec supports for a firmware-left unit:
If the measurement shows
Size. The head is +282/−43, of which about +150 is production kernel. Roughly 110 of those lines belong to the keep-on branch ( Previous blockers
Required findingsN1. BLOCKER:
N2. BLOCKER: the records describe the keep-on as landed, and the new issue's exit presumes it is the remedy.
Evidence
SuggestionS1. Move the DMAR walk into SEND BACK |
…, and a reproducer for the T14's iGPU fault One T14 shared-debug boot under iommu-firmware-left halted on a read by the iGPU (00:02.0) at 0x9cea7000, inside the RMRR firmware reserves for it, with reason 6h: a second-stage entry with R clear (VT-d 5.0 Table 30, LGN.3). The identity domain maps that address read-write, so unit0 did not translate by the tables it had been given. Under the actuator, and only there, unit0 goes on, off and on while the firmware's scanout reads through it. The orchestrator's T14 logs hold 787 boots with a single on and no iGPU fault, and 46 cycled boots, one of which faulted. That points at translation going off under live DMA, but does not establish it. The hand-over now keeps translation on where §6.6 allows it. With DMA_CTRL_PLATFORM_OPT_IN (§8.1), firmware's DMA reaches only its RMRRs, and §3.16 has those mapped identity and read-write. Where the identity domain holds every RMRR and every device they name is enumerated, this kernel's root translates every request in flight as firmware's did. The root switch then happens under translation (RTADDR.RTA may change while TES is set, §11.4.5), with the global invalidation immediately after SRTP. The T14 declares the flag. QEMU does not, so its unit still goes off first and says why; the new vtd_firmware_left machine test holds that branch. iommu-handover-cycles moves every unit between its root and a copy of it 200 times by each route: off-on and the switch under translation. Fault events are held throughout, every record is counted against the cycle it followed, and the first sixteen are printed with the walk the tables give from memory. Its row, iommu_handover_cycles, runs only by name, on a boot of its own. The iommu_firmware_left judge now requires, on a declaring machine, that every unit kept translation on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… infers; the reproducer leaves the PR The declared branch now holds only for a DMAR read to its end, and only where every RMRR is whole, has base <= limit, is on segment 0 and names endpoints alone. A walk stopped by a malformed structure or the unit ceiling leaves regions unread. A short RMRR reads its fields as zero. A bridge scope's requesters are the devices below it. Each of these now clears it. The comments and records stop presenting the premise as specification. §8.1 reports platform DMA confined to RMRRs. §3.16 expects identity mappings of system software's tables, not firmware's, so "firmware's tables translate alike" is labelled an inference. The root-pointer permission is §6.6's, and TTM's guard is §6.6 and §11.4.5. The DMA_CTRL_PLATFORM_OPT_IN constant said something false of the flag. The iommu-handover-cycles measurement moves to a branch of its own and does not land. Its next run takes every switch first and every off-on after, so off-on cannot leave the unit faulting into the switch phase. The issue's exit covers all three outcomes, calls zero faults both ways inconclusive, and stays open while a path known to fault takes translation off. The narrowed hand-over issue drops its Linux citation and records the pmm exposure both routes share. The IOMMU track records the containment proposal. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… memory regions; the keep-on branch goes VT-d gives system software no condition under which it may switch firmware's tables for its own under translation. §6.6 asks for identical remapping results, and nothing documented tells this kernel firmware's mappings. DMA_CTRL_PLATFORM_OPT_IN (§8.1) says only where platform DMA goes. So the keep-on branch and everything it alone used are deleted: InFlight, regions_identity, identity_mapped, SCOPE_ENDPOINT, Rmrr::whole, TRANSLATION_TABLE_MODE, and the judge's declared and kept. Every hand-over now takes translation off. While it is off, a unit's protected memory regions block DMA to the memory they cover (§11.4.8.1). Where every unit reports PLMR and PHMR, vtd/pmr.rs programs one pair on all of them, as §11.4.8.1 requires. They cover the largest stretch of firmware's RAM from 16 MiB to 4 GiB and the largest above it, each clear of every RMRR (kernel::dmaguard). The same alignment, probed and logged, applies to every unit. The regions go on before the first unit goes off and are released once the last translates. A unit without them logs that its hand-over is unguarded; QEMU's is one. Inside the regions, the hand-over invalidates through the unit's registers (§6.5.1). The invalidation queue and its status write are listed apart from the remapping structures the regions exempt (Appendix A, ECAP.C), so nothing depends on the queue's memory being reachable. The queue and interrupt remapping are programmed after the release. That makes Queue::invalidate_all dead: the actuator's leave_on invalidates through the registers too. A unit of version 6 or later refuses register-based invalidation (§11.4.6.1), so plan leaves it unprogrammed, as it does a unit whose IOTLB registers lie past its window. The steps run in kernel::dmaguard::order, whose host test reads arm, then every translate, then release, then every finish. It goes red with the arm step removed. Records: - The two hand-over issues become one: a unit without protected memory regions passes DMA untranslated while it is handed over. - The T14 iGPU fault is the existing 2026-10-04 issue, renamed because its claim is refused: no period of refused reads is needed. It carries #1086's measurement: on the T14, a switch under translation faulted in 0 of 600 cycles, and translation off and back on in 22 of 600, the unit refusing reads its tables grant. Intel's 631123-034 lists no erratum for it. Its exit is this kernel's sequence fixed, or a documented erratum's workaround applied. This change does not cure it: under iommu-firmware-left the T14 still takes translation off. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
r2 response (head 902bffb)N1: done, by the approved protected-memory round.
N2:
Evidence: https://github.com/ToyOSOrg/evidence/tree/103ec6294d868d70ea94ae01674112ac6d552950 |
Review r3: igpu-fault at 902bffbBase. Read.
Spec. VT-d Rev. 4.1 (D51397-016), re-read from Intel's PDF: §3.4.4, §6.5, §6.5.1, §11.4.3 ( Architectural assessmentProtected memory regions are the right mechanism for the T14, and the keep-on branch is fully gone.
Two structural problems remain:
The refusals added to Size.
Answers to the questions putPMEN protocol: correct.
Alignment probe: correct for the bases.
What the regions protect.
Register-based invalidation with the queue off: valid.
Ordering against interrupt remapping and the queue: correct.
Can legitimate DMA be silently blocked in the window? I found none on the T14.
Is "largest stretch" enough for the protection claim? No, see B1. Does the order test test behaviour? No, see B4. Is the deadlock fix right? Yes.
Is the merged issue's exit sound? Not as written, see B7. The owner question needs correcting before it goes out; see the last section. Required findingsB1. BLOCKER:
B2. BLOCKER:
B3. BLOCKER:
B4. BLOCKER:
B5. BLOCKER:
B6. BLOCKER: false, stale or unclear comments.
B7. BLOCKER: the records.
Evidence
Suggestions
The owner question (for the orchestrator)The question as framed needs four corrections:
SEND BACK |
…l, and a unit the window cannot serve is unguarded rather than refused B1: the identity domains and every root are built before the regions are chosen. Each region is the largest stretch of the OS's RAM, clear of the RMRRs, that holds every 2 MiB page those tables were carved from and this kernel's image, which iommu::init now takes. The boot logs what the regions cover. Where they cannot hold all of it, it logs the hand-over unguarded and does not arm them. The RAM is usable or ACPI-reclaimable, never NVS, and the 16 MiB floor goes: the map marks the legacy ranges as non-RAM. B2: plan refuses no unit for the hand-over's sake. A unit that cannot take register-based invalidation (major version 6 or later, or IOTLB registers outside the window) makes the machine's hand-over unguarded, and an unguarded hand-over invalidates through the queue, enabled before translation comes on, as main does. leave_on is main's again. B3: protected memory firmware left on goes off before anything is probed or programmed (§11.4.8.2 to §11.4.8.5). B4: the step interpreter and its tests go; init runs arm, every translate, release and every finish in sequence, the units moving through by value. largest keeps its tests, now with the ranges a region must hold. B5: Guard::arm logs, and the judge requires that line before translation first goes off and the release before the first unit is programmed. It treats a machine as guarded only when it has VT-d unit lines, so an AMD-Vi machine is not. Judged on an r3 T14 boot, which armed without that line, it is red. B6 and B7: comments and records. The merged issue is renamed to what it holds, a unit passing DMA untranslated outside its regions, and its exit names its four cases. It records that a machine whose firmware leaves translation off is open to every function from ExitBootServices until the regions go on. The iGPU issue labels its rate an estimate. Suggestions taken: the limits are probed with the bases (S1); a hand-over with nothing covered or fewer units programmed than described is unguarded (S2, S3); identity_domain takes the held tables, so the self-deadlock cannot be written (S4); the RAM set as above (S5); the functions mastering the bus are logged as the regions go on (S7). The orchestrator's ruling: the iGPU fault stays an open defect only the actuator reaches. Its row rides a boot of its own, iommu-firmware-left, so a halt costs only that boot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
r3 response (head c6a9c99)
Evidence: https://github.com/ToyOSOrg/evidence/tree/d900761951df99a25bdc1b3e7c4e73786b6be84b |
Review r4: igpu-fault at c6a9c99Base. Read.
Size. The branch is now +829/−241 against main. Kernel and lib account for +577/−77; that includes AssessmentThe structure is now the right one.
No blocker remains in the code. Landing waits on the running T14 batch, with the pass criteria below. Previous blockersB1 (coverage): resolved.
B2 (refusals): resolved.
B3 (
B4 (step interpreter): resolved. B5 (judge): resolved.
B6 (comments): resolved.
B7 (records): resolved.
Suggestions takenAll verified:
Owner rulingImplemented as ruled.
Evidence
Suggestions (optional)
LAND, provided the T14 batch meets the criteria above. |
|
Orchestrator: landing at c6a9c99. Review r4 LAND, conditional on the T14 batch; the batch meets every condition. iommu-firmware-left ×3: covered, armed on 4 units, no unguarded line, judge passes, no 00:02.0 halt. shared-debug ×3: only main's 5 shared-member reds. testcases and testcases-watchdog: the same lines, 22 passing plus the known reds (each shown independent of this branch). Linux's NIC was ok after all 8 boots. The 00:02.0 off/on fault stays an open, unattributed defect that only the actuator reaches, in its own boot (orchestrator ruling). |
Problem
One T14 shared-debug boot (7ef0a48) halted 1 ms after
Boot: peripherals ready. The iGPU (00:02.0) read 0x9cea7000, inside its RMRR, with reason 6h (read permission). The identity domain grants that address. The same defect was first recorded on 2026-10-04 (issues/the-t14s-igpu-faults-read-permission-after-translation-goes-off-and-on-under-its-scanout.md).Measured on the T14. The measurement branch
igpu-fault-measure(never lands) ran 3 boots of 200 cycles of each kind, with fault events held:00:02.0, a read, reason 6h, inside the firmware framebuffer in data stolen memory.Change
The keep-on branch is gone. VT-d gives no condition under which system software may switch firmware's tables under translation: §6.6 demands identical results, and nothing documented tells this kernel firmware's mappings.
Every unit is handed over with translation off, inside protected memory regions (§11.4.8), where the hand-over can be guarded:
ECAP.C), so it may lie under the regions.Unguarded otherwise, never refused. The boot logs the reason when:
Such a machine invalidates through the queue before translation comes on, as main does.
The judge.
iommu_firmware_leftrequires the arm line before translation first goes off and the release before the first unit is programmed, on VT-d machines whose units all have the regions. Its row rides a boot of its own,iommu-firmware-left, because a halt there is a known outcome.shared-debugno longer arms the actuator.Records:
a-unit-passes-dma-untranslated-outside-its-protected-memory-regions-while-it-is-handed-over.md, with four named exit cases;a-unit-this-kernel-does-not-program-…is updated;Checks (c6a9c99)
cargo run -- --ci host lint: exit 0.dmaguardhost tests: exit 0.vtd_firmware_left: unguarded, queue route, actuator;iommu_virtio_platform: shipping kernel;hda_plays_to_the_controller_that_reaches_an_output;a_drives_row_holds_one_process_per_function.iommu_firmware_leftis red.--ci hostfrom r3 stands for the host suites. They do not compilevtd/; lint compiles the kernel.Boot: complete):low 0x100000..0x8ef00000 high 0x100000000..0x45f800000, 1 MiB alignment), their coverage of the tables page (0x1a00000..0x1c00000) and the kernel image, the bus masters at arm, then arm and release. Nohand-over is unguardedline appears.iommu-firmware-left×3: the judge passes, exit 0. No 00:02.0 fault.shared-debug×3: 9 rows pass; only the 5 shared-member reds that main shows.testcases, the 26 rows that ride only it: 22 pass, plus the same 4 known reds as r3.testcases, the whole boot: 33 rows pass and 225 of 225 members complete. Six rows fail: those 4, plusamdvi_usbd_drives_its_own_entry(AMD bench) andi219_messages_on_its_msi_vector. The i219 failure happens seconds after the release, and handles: only an owner ends, moves, duplicates or lends a handle #1080's boot at 141055d, without this code, failed it the same way.DMA FAULTis the test-staged00:0d.0 owner=processfault.shared-debug: the 5 shared-member reds are independent of this change, being identical on main 576bebf. r2's boots ran the deleted keep-on branch, so they say nothing about this head's hand-over.Evidence: https://github.com/ToyOSOrg/evidence/tree/0a4bcfac482940b89dc7b122a33867f09591e97e
Halting (the implementer's reasoning, not an owner decision)
A fault at an address the domain maps means the unit contradicted tables shared by all four units, so it halts.
Risks
🤖 Generated with Claude Code