Skip to content

PCI root bridges are acpiserver's declaration: each root's _CRS windows and _OSC offer, held by the kernel; the loader's window reads go (PCI apertures stage B) - #935

Draft
Japabu wants to merge 20 commits into
mainfrom
wt/toyos-aperture-b
Draft

Japabu wants to merge 20 commits into
mainfrom
wt/toyos-aperture-b

Conversation

@Japabu

@Japabu Japabu commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

Status: draft. Merged with main d7f269dbc at f9e44e0fb. It waits on the owner's item 53 (below): until it is decided, cargo run -- --arch aarch64 panics before userland. Review round 2's four items are answered (round-2 comment). Host CI, both image builds and the five QEMU rows the review asked for are green at the head. Metal at 4e32df68c: AMD passed, T14 testcases passed, T14 acpicase had no log, as predicted; its row is now unregistered (below).

Problem

cargo run -- --arch aarch64 handed over none of its four PCI claims. The loader's window list (each root bridge's Configuration() plus every unowned GCD MMIO range) counted ECAM and flash among the declared windows (#916). The owner's ruling: aperture discovery leaves the loader, AML stays in acpiserver, and a refused or missing declaration fails closed.

Owner item 53: a kernel driver's domain before the declaration

Main's IOVA allocator (K1, kernel::iova, #974) builds every domain clear of the root bridges' declared windows, and drops the old rule that placed a domain above the CPU's addresses where the unit translates wider. Option 1, an identity-placed domain for a kernel driver, was withdrawn on this PR on the premise that after step 10 no kernel driver needs a domain before the declaration. The kernel's virtio console still does: it comes up before acpiserver declares, so it gets no domain (iommu: no domain of its own for a device: no declaration of the root bridges' windows is held) and keeps the machine's addresses.

  • On VT-d those are the identity domain's: the x86 console works, without the domain main gives it.
  • On virt's SMMUv3 a stream with no domain aborts, so the console's first queue completes nothing and the kernel panics in wait_used (UART log, lines 204 to 213). The virt guest rows pass because their console bypasses the unit.

Options:

  1. Recommended: a kernel driver's domain maps its pool at the pool's physical address. RAM is never inside a declared window, because the declaration's fence refuses a window over memory, so this needs no declaration. It is a translating domain that maps the pool alone. Cost: option 1's identity placement returns for the one kernel PCI driver left.
  2. A kernel driver's domain hands out addresses only from the RAM-typed ranges of firmware's map: the same property as option 1, through the allocator, with more machinery.
  3. The console bypasses the unit: no isolation for a DMA master.
  4. The console declines a stream with no domain, and the kernel's console stays on the UART: cargo run's terminal shows nothing after the loader.

issues/an-aarch64-cargo-run-boot-hands-over-no-pci-claim.md stays open with this cause. This branch had closed it at 4f5142fea, where the boot handed over all four claims.

What this changes

  • acpiserver declares the PCI roots, once, from the initialised namespace (system/acpiserver/src/apertures.rs):
    • A root is a PNP0A03/PNP0A08 device below no other. For each present root it reads _SEG, the bus range and the memory windows of _CRS (toyos_acpi::root_bridge), including Word, DWord, QWord and Extended descriptors, and steps over I/O.
    • It queries the root's _OSC under the PCI host bridge UUID: revision 1, Support = extended config space and MSI, Control asked = native hot-plug and the PCIe capability structure. It records the offer and makes no commit; Functions arrive and leave: the device set, native PCIe hot-plug, Gone (lifecycle stage 2) #934, which services hot-plug, commits.
    • Refused by name, with no root declared: a non-zero _TRA, a consumed range, an unknown descriptor, a second or out-of-range bus range, a root with no bus range, more than 16 roots, more than 64 windows across all roots.
  • The declaration is per host bridge (toyos_abi::acpi::Apertures): segment, buses, the _OSC offer, a commit's grant, and a run of one shared window list. It travels through op::DECLARE on the acpi claim. Functions arrive and leave: the device set, native PCIe hot-plug, Gone (lifecycle stage 2) #934 consumes this shape.
  • The kernel holds it (kernel/src/pcidev): only from the acpi claim's holder and only from Unknown; every record checked against the ABI, then every window together (whole, disjoint, none over RAM). Any failure refuses by name, wakes the waiters and stands as the answer. The holder's claim ending first withholds it (Ended).
  • A PCI claim made before the declaration waits on its watch, on the claimant's syscall thread, for at most 30 s; one that waits it out withholds the declaration for the boot (Unanswered).
  • Every metal boot fails on a refused declaration (Readback::declaration_not_refused), and the AMD row requires the declared-windows record.
  • IOMMU, over main's allocator: a domain's allowed addresses are the unit's range less the backend's reserved ranges (RMRRs, IVMDs, the interrupt range, AMD-Vi's special range, the ITS doorbell), which each unit gives Domains::avoid, and less the declared windows, which each claim passes (OwnSpace::create(room, windows)). No declaration, no domain. The AMD-Vi and SMMU selftests, which run before any declaration with the test device on QEMU's root bus, pass none.
  • The supervisor starts every acpi claim holder first, before the disk drivers (One block service and a driver per disk controller; input names each device; SCSI is its own crate (usbd step 10, PR 1 of 2) #962's order), whose claims wait for it. Since boot: the kernel hands the supervisor its images and parses no filesystem #994 the list begins with SYSTEM, so the holders are picked out of the whole list; taking them from its front missed acpiserver and made the_log_lands_on_the_boot_stick wait out the deadline, a defect the merge brought and f9e44e0fb fixes.
  • Every image whose rows claim a PCI function starts acpiserver, main's benchcase and hostedclangcase now too, and a build host test holds it (every_config_that_claims_a_pci_function_starts_its_declarer; undeclaredcase is the one exception, booted to judge the refusal).
  • acpicase has no storage. Its first acpi holder is a job, so no row declares before the supervisor's own claims; main's nvme and usbd rows there would wait out the deadline and refuse every claim. The probe's held-function arm moves to the console function the kernel drives.
  • acpi_server_death is unregistered, with its judge and its job acpi_release: its job must hold the boot's first acpi claim, so nothing declares before usbd's stick claim and the T14 boot has no log (measured at 4e32df68c). No QEMU row can take its place, since QEMU's firmware hands over in ACPI mode. issues/the-acpi-servers-death-has-no-readback-on-the-t14.md, owned by the ACPI track, has re-registering all three as its exit.
  • AArch64 gets the hardware-reduced acpi claim: ECAM for configuration, firmware memory through the direct map, no registers or ports, no new privilege.
  • Deleted: loader/src/gcd.rs, loader/src/rootbridge.rs, PciRootBridgeIo, KernelArgs::root_bridge_windows and its count (KernelArgs 1504 → 472 bytes), the OVMF Configuration() fixture and its tests.
  • Issues:
  • Size: 80 files, +2220/−1133 against the merge base, tests included (measured).

What each judge establishes

  • virt_claim_lpi requires pci: root 0000:00 _OSC: native hot-plug offered; VirtIts has a second root at bus 8.
  • a_drives_row_holds_one_process_per_function requires q35's not offered.
  • a_claim_waits_out_an_undeclared_machine (q35, no acpiserver): the first claim waits 30 s, and both are refused by name.
  • acpi_mediated_access judges the Ended transition, and the console function's HeldFunction.
  • domains_map_clear_of_the_host_bridges (T14) reads every mapping against the acpi claim's declared windows.
  • amdvi_translates_the_stick_controller also requires the declaration.

Checks

Evidence: pr-935 at 415052dc7. That commit was published through the GitHub API after the clone was refused, which the owner is deciding on.

check revision result
cargo run -- --ci host f9e44e0fb exit 0, 93 steps green
--build-only, x86-64 and aarch64 f9e44e0fb exit 0, 0
a_drives_row_holds_one_process_per_function, a_claim_waits_out_an_undeclared_machine, acpi_mediated_access, virt_claim_lpi, the_log_lands_on_the_boot_stick f9e44e0fb exit 0, 5 passed
the same five at e6dd4addf, before the supervisor fix e6dd4addf 4 passed; the_log_lands_on_the_boot_stick stalled waiting for usbd, the defect fixed above
22 QEMU rows, host CI 1ffbdbaaf exit 0 each (round 3 evidence above)
cargo run x86-64 / aarch64 1ffbdbaaf boots / kernel panic, item 53
controls earlier rounds each red; none of the later merges touches the mutated lines
metal 4e32df68c AMD testcases: 3 passed. T14 testcases: passed (3 windows declared, a claim waited 67 ms, _OSC: native hot-plug offered, buses 0x00..=0x79, no commit). T14 acpicase: no log, the predicted outcome.

Logs after 1ffbdbaaf are kept locally and not published, pending the owner's word on evidence publication.

Risks and open points

  • The decision above blocks landing: aarch64 with the console behind the SMMU does not boot.
  • acpi_server_death is unregistered until its T14 boot has a log (its issue, owned by the ACPI track).
  • Every machine's claims depend on acpiserver, metal included; an x86 machine with no acpi row loses them all (its issue).
  • benchcase now runs acpiserver beside osbench's measurement, since the stick's claim needs it.
  • The AMD bench's _CRS is still unmeasured byte for byte; its next boot logs the bytes and the window count.
  • The T14's firmware offers native hot-plug (measured at 4f5142fea); Functions arrive and leave: the device set, native PCIe hot-plug, Gone (lifecycle stage 2) #934 commits it.

🤖 Generated with Claude Code

https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q

Japabu and others added 6 commits October 14, 2026 20:54
…clare, pcidev state)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016UzbDnM9m6nQWhbjGi6Lv1
# Conflicts:
#	kernel/src/pcidev/mod.rs
#	lib/acpi/tests/resource.rs
…vers map at identity

Work in progress before review; the PR body states the change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016UzbDnM9m6nQWhbjGi6Lv1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016UzbDnM9m6nQWhbjGi6Lv1
…the _OSC grant

Ruled for lifecycle stage 2, which arms hot-plug ports under a granting
root: acpiserver negotiates each root's _OSC under the PCI host bridge UUID
(PCI Firmware Specification 3.3 section 4.5) in the pass that reads its
_CRS, and the kernel keeps the records and says each grant.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016UzbDnM9m6nQWhbjGi6Lv1
# Conflicts:
#	kernel/src/arch/aarch64/smmu/domain.rs
#	kernel/src/arch/x86_64/amdvi/domain.rs
#	kernel/src/arch/x86_64/amdvi/mod.rs
#	kernel/src/pcidev/mod.rs
#	tests/toyos.rs
@Japabu

Japabu commented Oct 15, 2026

Copy link
Copy Markdown
Collaborator Author

Metal at 4f5142fea, run by the orchestrator: evidence commit 42550608c1e282100d36de07febe6b3cb9f83b80, stage-b/README.md, "Metal".

  • T14: 4 passed over 2 boots (acpi_tables_loaded, claim_reuses_its_remapping_entry, domain_ends_below_the_host_bridges, usbd_drives_the_type_c_controller). Tables loaded in 54 ms and 55 ms; acpiserver started 1-2 ms before diskserver; declared mem 0xa0000..0xc0000, mem 0xa0800000..0xc0000000, mem 0x4000000000..0x8000000000; one claim waited 37 ms. The granted 0x0 of 0x0 lines are \_SB._OSC and the processors'. The host bridge's line is pci: root 0000:00 _OSC: native hot-plug granted.
  • AMD bench: amdvi_translates_the_stick_controller passed: each kernel xHCI domain maps its one pool at identity. Tables loaded in 45 ms.
  • Finding, a defect of this PR: the AMD bench's \_SB.PCI0._CRS has more than MAX_BRIDGE_WINDOWS (8) memory windows. So acpiserver declared no root and the kernel refused the declaration. No claim was made on that boot, so no row went red, but every process claim on that bench would be refused at this head. The fix is a larger per-root bound, or one window pool shared across roots, plus a host case from that bench's _CRS. I have not changed it while the review runs.

…eclaration fails every metal boot

The AMD bench's root has more memory windows than the per-root bound of
8, so at 4f5142f its declaration was refused, every claim on it would
have been, and no row saw it: its boot claims nothing and its row judges
kernel-driven domains. The windows are now one list of 64 across every
root; acpiserver says each root's count and its _CRS bytes under OWN,
so the bench's next boot records them. Every metal boot now fails on the
kernel's refused-declaration record, and the AMD row requires the
declaration. The host bridge _OSC tests gain QEMU q35's masked shape.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016UzbDnM9m6nQWhbjGi6Lv1
@Japabu

Japabu commented Oct 15, 2026

Copy link
Copy Markdown
Collaborator Author

Review of #935 at 4f5142fea (PCI apertures stage B)

Architecture

The ownership matches #916's design and the stage-B rulings. AML stays in acpiserver. The kernel holds a typed, write-once aperture::Held. The loader no longer asks UEFI about apertures. gcd.rs, rootbridge.rs, PciRootBridgeIo and KernelArgs::root_bridge_windows are gone with no residue: the only remaining match is a historical string in build/identity.rs:208's test fixture. KernelArgs is 456 bytes in the ABI's const assertions, the loader's initializer and the kernel's reader, all in one landing.

Admission is enforced by sys_acpi's held_by check and by Held::declare accepting only from Unknown. The claim's Drop calls declarer_gone, so a claim that ends without declaring moves the state to Ended. Both paths are host-tested.

Option 1 is implemented as ruled:

  • Placement::Identity is used only by DeviceSpace::create, which serves the kernel's own drivers.
  • Its map returns Iova::translated(phys). Nothing else is mapped into such a domain on VT-d or AMD-Vi; on the SMMU the ITS doorbell page is also mapped, which MSI requires and which is MMIO.
  • It is refused when pmm::top() reaches past what the unit translates.
  • Claimed functions keep Window::new, now taking the declaration as a parameter. IommuError::Undeclared and the Option threading are gone.

The RAM fence takes firmware_map() filtered by is_read_as_memory: usable memory plus ACPI reclaim and NVS. That covers every type the pmm hands pools out of, which is what the premise needs. It is a different source from the IOMMU's reserved regions, which come from RMRRs and IVMDs, and it should be. The fence runs after the kernel drivers' domains already exist, so it is a bug fence on the declaration, not a gate on option 1. The premise itself is a hardware fact.

window.rs's header was rewritten, not appended to.

The wait is sound:

  • wait_until on DECLARATION runs with a Parkable::at_entry() on the claimant's syscall thread. For an endowed claim that is the supervisor's thread.
  • It has a 30 s Deadline and never spins.
  • A kill returns None, so the claim fails as Unusable.
  • One expired deadline answers Unanswered for the whole boot. This is stated in the pcidev header and in DECLARATION_DEADLINE's doc. The undeclared row tests it: exactly one waited line, and both claims refused. It hides no transient: a declarer slower than 30 s leaves a named refusal on every claim.

Required

  1. kernel/src/syscall/device.rs:181 / kernel/src/pcidev/mod.rs:425 / sdk/abi/src/syscall.rs:2119: a malformed record is refused silently and without becoming the answer.

    • RootBridge::decode returns None for a count > 16, a window_count > 8, an unknown _OSC word, a grant beside no answer, or first_bus > last_bus. That answers InvalidArgument, but the state stays Unknown, nothing is logged and nothing is posted.
    • acpi_declare's contract says InvalidArgument means a declaration "said in its log by name and leaving every PCI claim refused". For this path that is false.
    • Scenario: a hostile or buggy acpiserver sends count = 17. Every waiting claim keeps waiting toward its 30 s deadline with no line saying why, and a second, valid DECLARE is then admitted. That breaks "once", because a refused declaration is supposed to be the answer.
    • Fix: make a decode failure a named refusal through Held (for example a Refused::Record variant, logged and posted). decode currently has no test, so move it beside Held in toyos-pci and host-test each malformed field there.
    • Overlapping bus ranges of two roots on one segment are not refused either. That is harmless today, since only the windows are consumed and overlapping windows are caught by check, but see suggestion 2.
  2. system/acpiserver/src/osc.rs:8-11 against :157-178: ToyOS commits native PCIe hot-plug control it does not perform, and the module header now states something false.

    • The header holds _OSC to the owner's _OSI condition: "each bit set only where ToyOS does what the bit promises". negotiate_root commits NATIVE_HOT_PLUG, and nothing services it until lifecycle 2. The PR's own risk section admits that on virt firmware hands hot-plug to an OS that does not act on it.
    • The stage-B ruling asks for the grant to be negotiated and logged. It does not set aside the owner's condition. That conflict is the owner's to settle, not the implementer's.
    • Smallest compliant fix: send the query only (Query flag set) and record what would be granted, then let lifecycle 2 commit once it can service hot-plug. Otherwise, get the owner's explicit agreement and rewrite the header to state the exception.
  3. No silent debt: an x86 machine with no acpi row now loses every PCI claim.

    • kernel/src/arch/x86_64/acpi_mode.rs:234-250 mints no row for any of these: an unusable FADT, no PM1a block, no SMI_CMD, no I/O APIC carrying the SCI, an SCI line shared with another row, or HW_REDUCED_ACPI.
    • On main such a machine still handed over its NVMe and NIC claims from the loader's list. On this branch its first claim waits 30 s and every claim of the boot is refused.
    • This is a fail-closed consequence of the accepted design, but it is recorded only in the PR body. issues/a-hardware-reduced-x86-machine-has-no-power-off-and-no-power-button.md covers power alone.
    • It needs an issues/ record with its evidence, owner and exit, per root CLAUDE.md.
  4. Evidence: tests/images/acpicase.toml changed, and its metal row was not staged.

    • Dropping diskserver changes the image of acpi_server_death (tests/toyos.rs:700), a T14-only row. Its fileserver still lists the data role with no block service.
    • The guest rows on this image pass under QEMU (guests-merged.log). Nothing shows that the T14 boot still reaches its job and readback without the NVMe.
    • The staged T14 batch has only testcases and testcases-xhci-leave (metal/t14-request.txt).
    • Cheapest fix: add acpi_server_death (one acpicase boot) to the T14 run the orchestrator is already making, or explain why the change cannot affect it.
  5. kernel/src/iommu/window.rs:87: false contract. Addresses::map is documented "never handed out before". For Identity that is not true: it answers phys for any call. The sentence belongs to the Window arm alone, so say so or drop it.

Suggestions

  1. sdk/abi/src/acpi.rs:358: MAX_BRIDGE_WINDOWS = 8 is unmeasured. The old bound was 64. Exceeding it refuses the whole declaration, which on the T14 would refuse every claim on the bench.
    • Inference, not measured: Intel root _CRS lists commonly produce the sixteen-KiB legacy windows at 0xc0000..0xdffff beside VGA and the 32- and 64-bit windows, which would exceed 8.
    • Raising the bound costs 16 bytes per window per bridge. The staged acpi_tables_loaded row will show the real count either way.
  2. kernel/src/pcidev/mod.rs:417: the kernel keeps first_bus and drops last_bus. Lifecycle 2 will need the range, and it is easier to validate once at this boundary now: refuse overlapping ranges on one segment.
  3. Lost coverage. The acpi_mediated probe's HeldFunction arm now checks the kernel-driven branch of pcidev::enumerated (:512). Nothing booted exercises the claim-held branch (f.held) any more, and only acpi_mediated checks HeldFunction in a guest. The PR body should say so, and if it can be restored cheaply, it should be. One way: a probe arm that declares q35's measured windows itself as the claim's holder, then holds an edu.
  4. amdvi/selftest.rs:44, smmu/selftest.rs:62: the comment "the unit translates past the CPU's addresses" is an unchecked premise behind passing &[]. An assert! costs as little and holds it.
  5. Kernel stall at boot (inference). The supervisor blocks inside its first PCI claim until acpiserver declares. On the T14, earlier namespace initialisation asked for about 10 s of sleeps (The AML interpreter finished to ACPI 6.5: Load and _OSC, the namespace initialised, every address space, every event, and the bounds stated #875), and the staged acpi_tables_loaded row prints each claim's wait. The orchestrator should read that row as a boot-time cost, not only as a pass.

Evidence checked

  • ci-host-4.log: --ci host at 4f5142fea, EXIT=0, "86 step(s), all green". The FAILED lines in it are the proclife mutation controls reaching their verdicts.
  • guests-merged.log: 18 passed at 4f5142fea.
  • x86-undeclared-2.log: at d84d8c345, "waited 30006ms", and both edu claims refused by name. The test's lower bound of 30 s is a deadline that cannot fire early, so it is not a QEMU timing claim. The ceiling is three times the deadline.
  • arm-cargo-run-final.log: _OSC: native hot-plug granted, the declared windows 0x10000000..0x3eff0000 and 0x8000000000..0x10000000000, "waited 5ms", 00:02.0, 00:03.0, 00:05.0 and 00:06.0 handed over, diskserver: partition … at block 256, vdagent: port 1 open, and domain1 "maps each pool at identity". The closed issue's exit is met.
  • Controls: _TRA, the RAM fence, a second declaration replacing the first, and the claim not waiting are each red as stated, at revisions whose later edits do not touch the mutated lines.
  • The _CRS reader handles Word, DWord, QWord and Extended descriptors, skips zero-length ranges, refuses _TRA, consumed, unknown and second-bus descriptors, and is swept over both QEMU _CRS lists.
  • _OSC uses UUID 33DB4D5B-…, revision 1 and three DWORDs. The Control field is masked to what was asked, then to what was offered. Failure, an unrecognised UUID or revision, an absent _OSC and malformed answers are each host-tested.
  • The judges are restated as claimed: clear_of_host_bridges (identity mappings at phys, outside every declared window, with red cases) and amdvi_translates (exactly one mapping, at identity).
  • PR body correction (orchestrator's): the measured diff against origin/main is 69 files, +1940/−1113, not "+1621/−1100 over 67 files".

SEND BACK: required items 1 to 5. The T14 and AMD rows remain the orchestrator's to judge.

Japabu and others added 4 commits October 15, 2026 06:53
…ried only, the no-acpi-row loss recorded

- A declaration whose records the ABI does not name (a count or run past
  its array, an unnamed _OSC word, a reversed bus range) or whose roots
  share a bus on one segment is refused through Held like any other
  refusal: logged, posted, and the answer (toyos_pci::aperture::records).
- The root _OSC is a query alone and its offer is recorded; the hot-plug
  service that acts on native hot-plug control (#934) commits it, so the
  osc module's condition holds.
- issues/an-x86-machine-with-no-acpi-row-hands-over-no-pci-claim.md.
- window.rs: Addresses::map's contract stated per arm.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016UzbDnM9m6nQWhbjGi6Lv1
Owner ruling: the kernel stops driving PCI functions (step 10) before this
lands, so no kernel driver needs a domain before the declaration. The
window rule is the one rule again, for every domain: above the CPU where
the unit has room, and otherwise clear of the declared windows, with no
domain where nothing is declared. A claim passes its declaration; a
kernel driver, which comes up before any declaration, passes none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016UzbDnM9m6nQWhbjGi6Lv1
@Japabu Japabu changed the title PCI root bridges are acpiserver's declaration: each root's _CRS windows and _OSC grant, held by the kernel; the loader's window reads go (PCI apertures stage B) PCI root bridges are acpiserver's declaration: each root's _CRS windows and _OSC offer, held by the kernel; the loader's window reads go (PCI apertures stage B) Oct 15, 2026
@Japabu

Japabu commented Oct 15, 2026

Copy link
Copy Markdown
Collaborator Author

Round 1 at de8e9883a (main 3b77b9b2f merged). Evidence: pr-935 64646e491, stage-b/round2/README.md.

  1. Malformed records: toyos_pci::aperture::records checks every record before a window is read. It refuses a count or window run past its array, an unnamed _OSC word or a grant beside no answer, a reversed bus range, and two roots of one segment sharing a bus (your suggestion 2). The kernel keeps both bus ends. The refusal goes through Held::refuse: it is logged, the waiters are woken, and it stands as the answer, so a later declaration is late. Host-tested per field. The control that leaves the state Unknown reds the test.
  2. _OSC: a query alone, revision 1, Control bits 0 and 4; the offer is recorded and logged as offered. Functions arrive and leave: the device set, native PCIe hot-plug, Gone (lifecycle stage 2) #934 commits. The osc header states this. A host test shows the firmware is called once, with the Query flag set; a control that commits reds it. QEMU q35's 0x1F and 0x1E masks are now cases.
  3. Recorded as issues/an-x86-machine-with-no-acpi-row-hands-over-no-pci-claim.md, with each condition that leaves no row, the measured machines, owner and exit as ruled.
  4. acpi_server_death: not staged. By the owner's ruling the branch is blocked on step 10 and no bench is re-staged before it. The body says the bench boots would fail by design until then. This row joins the batch when the branch rebases.
  5. window.rs:87: gone with option 1, which the owner withdrew. window.rs is main's again, the one rule. Claims pass their declaration; kernel drivers, which come up before any declaration, pass none.

Also in this round:

  • Window bound: the AMD finding is fixed with one list of 64 windows across every root. An 18-window root is a host case, and a control with the old per-root bound reds it.
  • Shared metal check: every metal boot now fails on pcidev: the root bridges' memory is refused as declared, and the AMD row requires the declared-windows record.
  • Lost coverage: your suggestion 3 is stated in the body.
  • Size: measured at 70 files, +1972/−1047.
  • Not taken: suggestion 4 (the selftests pass None now, so the premise is the window rule's own refusal); suggestion 5 (no bench boot happens before step 10).

So #934 commits through this ABI unchanged: a root's record holds what
its query offered and what a commit (Query clear) was granted, none in
stage B. A commit beside no offer, or of bits not offered, is a record
the ABI does not name, refused by name with the declaration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016UzbDnM9m6nQWhbjGi6Lv1
@Japabu

Japabu commented Oct 15, 2026

Copy link
Copy Markdown
Collaborator Author

a18b42b88: the per-root record carries the _OSC commit beside the offer, so #934 needs no ABI change.

  • HostBridge gains commit/committed beside osc/offered (renamed from granted).
  • HostBridge::commit() answers Some(None) where no commit was made, as in all of stage B. It answers Some(Some(bits)) for a commit that granted no more than was offered. It answers None for a commit beside no offer, bits not offered, or an unnamed word.
  • toyos_pci::aperture::records refuses any of those cases by name (Refused::Record) with the whole declaration.
  • The kernel keeps the commit with the root's record and logs …, buses 0x00..=0xff, no commit (or committed 0x…).
  • acpiserver still sends the query alone and declares None for the commit. Functions arrive and leave: the device set, native PCIe hot-plug, Gone (lifecycle stage 2) #934 performs the commit beside the query in osc.rs and declares the committed bits.

Host tests: an offer with no commit decodes; a commit decodes beside an offer covering it; a commit beside no offer, beyond the offer, or under an unnamed word is none; records refuses a commit with no offer and accepts an offer with none. cargo run -- --ci host at a18b42b88: exit 0, 86 steps green (log). No guest row was rerun: the kernel's log line gained a trailing clause, and the two judges that read it match prefixes that still hold.

Japabu and others added 4 commits October 22, 2026 10:29
Conflicts resolved against main's changes since 3b77b9b:

- #947 deleted kernel/pci/src/placement.rs and
  issues/no-boot-reaches-the-retry-that-undoes-a-placement-nothing-answered.md;
  the branch's hunks there (a doc comment and an import, and the issue's
  exit) were about the placement that no longer exists, so both stay
  deleted. issues/an-aarch64-cargo-run-boot-hands-over-no-pci-claim.md
  stays deleted: main's one-line edit restated how its windows are read,
  which this branch replaces.
- K1's IOVA allocator (6cb27ce): a domain's allowed addresses are still the
  unit's range less every reserved range, but the root bridges' windows now
  come per domain from the declaration (`Domains::prepare`), beside the
  backend's own reserved ranges (RMRRs, IVMDs, the interrupt range, AMD-Vi's
  special range, the ITS doorbell), which `avoid` keeps. No declaration, no
  domain. `OwnSpace::create` takes the declared windows; the IOMMU
  selftests, which run before any declaration on QEMU's root bus, pass none.
- pcidev: main's `publish` without the loader's list, and the declaration,
  its wait and its refusals as on the branch.
- KernelArgs: main's `slot_volume_guid` follows `microcode_len` at 456;
  472 bytes.
- The supervisor (27b75c3): the `acpi` claim's holder starts before the
  disk drivers, whose claims wait for its declaration.
- acpicase: main's nvme and usbd rows claim PCI functions on a boot whose
  only `acpi` holder is a job, so it keeps no storage rows; the probe's
  held-function arm takes the console function the kernel drives, the only
  function a kernel driver still holds.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q
Main's benchcase and hostedclangcase came with nvme and usbd rows and no
`acpi` claim holder, so on this branch their first claim would wait out its
deadline and every claim of the boot would be refused. Both start
acpiserver now, and a build host test holds the rule for every config:
one whose rows claim a PCI function starts a row holding the `acpi` claim,
undeclaredcase alone excepted, since it is booted to judge that refusal.

Issues directly affected by the loader's window reads going:
- the GCD map's held MMIO descriptors: deleted, since the loader reads no
  GCD map any more;
- the hypervisor and ACPI tracks and the BarUnrouted tooling issue name
  `_CRS` where they named the loader's reads;
- the x86 no-acpi-row issue names the disk drivers, which main's block
  service split made the claimants;
- new: the ACPI server's death has no readback on the T14, since its
  boot's first `acpi` holder is a job and no row declares before usbd's
  claim.

Issues that named closed drafts as owners: the four scheduler findings
#872's S2 and S3 were to settle are the placement track's, which owns the
push, the victim choice and wake placement; the speed track's line naming
#865 as the CPU cap's owner goes, since #976 sized every per-CPU table for
the CPUs firmware names; the ACPI track's `_CST` line keeps its issue and
drops #870.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q
The merge of main left the kernel's virtio console with no domain before
the declaration: main's IOVA allocator builds every domain clear of the
root bridges' windows, and the console comes up before acpiserver declares
them. On VT-d the console keeps the identity domain and works; on virt's
SMMUv3 its stream aborts, so `cargo run -- --arch aarch64` panics in
`wait_used` before userland. The issue this branch had closed stays open
with that cause, owned by this track, since how a kernel driver gets a
domain before any declaration is the owner's to decide.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q
benchcase: main's osbench round 2 beside the ACPI server this branch
starts there.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q
Japabu added a commit that referenced this pull request Oct 22, 2026
…2d60) into wt/toyos-lifecycle-2

Lifecycle stage 2 rebuilt on main and stage B.

- pcidev: stage B's module is the base. The per-requester records, the
  device set's generation and snapshot, `Gone` on an older arrival's claim,
  the departed release and retirement, and the slot's interrupt are this
  branch's, over main's BAR records (`Decoded`); `Machine::windows` and
  the placement state are gone with #947.
- An arrived function's BARs: `place_unassigned` and `behind_bridges` are
  gone. Each BAR the device leaves unassigned is given, before the
  function is listed, the lowest address inside its port's windows that no
  other function decodes in and that the declared root bridges' memory
  holds (`toyos_pci::hotplug::{Windows, assign}`, host-tested), written
  once and kept where the register reads it back, with no probe through
  it. Its BARs enter `Machine::decoded` and leave it on retirement.
  `bar::placement` stays for that one write, refusing only an I/O BAR and
  asserting the caller's own choice.
- The slots are armed by stage B's declaration, from `pcidev::declare`,
  below a root whose `_OSC` commit was granted native hot-plug and the
  capability structure. acpiserver commits both where its query offered
  both (`osc::negotiate_root`); the declaration carries the grant. The
  boot-time arm with no bridges is gone.
- A slot's port reads its own bus numbers and windows: main deleted
  `toyos_pci::bridge` and `PciDevice::bridge`. `PciDevice::header_type`
  tells a bridge below a slot.
- Two readers of the set never enumerate one arrival twice (`REFRESHING`).
- ABI: the device set's object kind is 15 and its syscalls 135 and 136,
  past main's; the static `IrqWatch` poll returns for its watch.
- supervisor: main's `drive_where` reads the device set, says what joined
  and left, and decides over the rows asked for.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q
Japabu and others added 2 commits October 22, 2026 14:13
Its job must make the boot's first acpi claim, so no row declares the root
bridges' memory, and usbd's stick claim, which carries the log, waits out
the deadline. The T14 boot at 4e32df6 ran the job and stopped cleanly,
and its kernel.log came back empty. A row that cannot produce a readback is
a defective registration, so it goes with its judge and the helper only it
used. No QEMU row takes its place: QEMU's firmware hands over in ACPI mode,
so the release has no legacy mode to return to. `acpi_release` stays,
run by nothing. The issue's exit is the row registered again on a boot that
has a log, and the issues and the ACPI track that read the row point there.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q
- KernelArgs: main's boot images (#994) without the loader's root bridge
  windows; 504 bytes, root_image at 184.
- loader: main's payload module, no rootbridge.
- supervisor: start_service takes main's spawn (the image's source, the
  boot images) and skips SYSTEM, which starts before the manifest.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q
@Japabu

Japabu commented Oct 22, 2026

Copy link
Copy Markdown
Collaborator Author

Review of #935, round 2: pushed 4e32df68c, plus local e307e6060 and 3d6257080

I reviewed the pushed head 4e32df68c and the two local commits that are not pushed yet:

Against origin/main at 3d6257080 the diff is 81 files, +2243/−1197. Tests account for about +350/−240 of it, outside the inline unit tests. The main deletions are the loader's GCD and root-bridge readers, PciRootBridgeIo, KernelArgs' window array and its tests, and the OVMF Configuration() fixture with its sweep. The sweep moved to acpiserver over the evaluated q35 and virt _CRS lists.

Architecture

The ownership is right and matches #916's ruling:

  • AML stays in acpiserver.
  • The kernel holds one write-once aperture::Held. Its records refuse a malformed record by name and the refusal stands as the answer (round 1's item 1).
  • The loader asks UEFI nothing about apertures.
  • toyos-abi carries only the declaration that crosses the kernel boundary.

Over main's K1 allocator, the IOMMU change is the smallest that fits:

  • Domains::reserved holds only the backend's own ranges: RMRRs, IVMDs, the interrupt range, AMD-Vi's special range and the ITS doorbell.
  • Each OwnSpace::create passes the declared windows.
  • No declaration means no domain.

The supervisor's ranking puts the acpi holder first and #962's disk drivers next, which is the order the wait needs. The aarch64 hardware-reduced claim is new kernel surface. It is justified, because acpiserver must run on virt to declare there, and it grants nothing beyond ECAM and firmware memory through the direct map.

The open owner decision is recorded honestly. The PR body and issues/an-aarch64-cargo-run-boot-hands-over-no-pci-claim.md say:

  • the kernel's virtio console gets no domain before the declaration;
  • VT-d falls back to identity;
  • the SMMUv3 aborts the console's stream, so cargo run -- --arch aarch64 panics in wait_used;
  • option 1 was withdrawn on a premise that turned out false.

I have not decided it. Landing waits on it whatever else is fixed.

I checked the stale-reference sweep after the deletions:

  • No gcd.rs, rootbridge, root_bridge_windows, PciRootBridgeIo, #872, #865 or the-gcd-maps reference remains.
  • One #870 remains, in a ruling's provenance line in the ACPI track.
  • One rootbridge.rs remains, in a landed stage of the loader track.

Both are history and correct as they stand.

The repointed scheduler issues name issues/placement-has-no-migration-cost-or-wake-affinity.md, which exists on main. placement.rs and no-boot-reaches-the-retry… stay deleted.

Required

  1. kernel/src/iommu/mod.rs:169-174, kernel/src/drivers/virtio_console.rs:235-238: false comments.

    • DeviceSpace::create calls unit::domain::create(0, None), which always fails with Undeclared or NoUnit, so DeviceSpace::Own is never built at this head.
    • The doc still says a kernel driver gets the machine's addresses only "where the unit has no room above the CPU's addresses". That is the rule K1 removed.
    • The console's comment says "an address space of this device's own, holding one pool".
    • Whichever option the owner picks, these must say what the code does. If option 3 or 4 is chosen, the dead Own path in create goes too.
  2. e307e6060: acpi_release is now dead code.

  3. issues/an-aarch64-cargo-run-boot-hands-over-no-pci-claim.md:20-21: a misleading record.

    • "the console works, isolated by no domain of its own as it was on main" reads as if main gave it no domain either.
    • The PR body says the opposite, which is correct: main built the console a domain clear of the loader's list.
    • Say it loses the domain main gave it.
  4. Evidence at the head that will land.

    • 3d6257080 merges boot: the kernel hands the supervisor its images and parses no filesystem #994. That merge rewrites KernelArgs (504 bytes, root_image at 184), the loader's start_kernel and the supervisor's start path, and this branch's start_service was resolved against main's new spawn.
    • Only ci-host-3 is running there. The 22 QEMU rows and both --build-only runs are at 1ffbdbaaf.
    • The cheapest sufficient check:
      • --ci host, plus --build-only on x86-64 and aarch64, at the pushed head;
      • the rows that boot through the supervisor's new order: a_drives_row_holds_one_process_per_function, a_claim_waits_out_an_undeclared_machine, acpi_mediated_access, virt_claim_lpi, and one usbd boot-stick row.
    • The metal at 4e32df68c predates boot: the kernel hands the supervisor its images and parses no filesystem #994's boot path. Rerun it only if those QEMU rows disagree with it; that is the orchestrator's call.

Suggestions

  1. build/build.rs:3074: the test's scope is narrower than its claim.
    • It sees rows that drives or that start with a pci: device. It does not see a program that claims through the device syscap: virtclaimcase, testcases and metaldevicecase would pass it without acpiserver.
    • Either say so in its doc, or count device-syscap holders, with acpicase as a second exception.
  2. sdk/abi/src/acpi.rs:357-360, system/acpiserver/src/apertures.rs:279: history in two doc comments.
    • "the per-root bound that refused it" and "the per-root bound of 8 that refused the AMD bench's root" are investigation history.
    • The bound's justification is the measured counts, and the AMD bench's is now 16 (metal/amd readback). State that number instead.
  3. sdk/abi/src/acpi.rs:377: Osc::Offered's doc names "Stage B". Say "a query's offer", which stays true when Functions arrive and leave: the device set, native PCIe hot-plug, Gone (lifecycle stage 2) #934 commits.
  4. system/supervisor/src/main.rs:963, 979: the declarer's late HOME is made in two places. One home step, taken where the session's home is made, would remove the second loop and the homes_made threading.
  5. kernel/src/arch/aarch64/acpi_mode.rs:84 and x86_64/acpi_mode.rs:786: duplicated access code.
    • Both decode and validate an Access, make unaligned direct-map moves (Unaligned<T>) and make an ECAM config access.
    • None of that is architecture code. A generic module with arch hooks for ports and registers would keep one copy.
  6. Owners named by PR number. the-acpi-servers-death-has-no-readback-on-the-t14.md and an-x86-machine-with-no-acpi-row-hands-over-no-pci-claim.md name "the PCI apertures track (PCI root bridges are acpiserver's declaration: each root's _CRS windows and _OSC offer, held by the kernel; the loader's window reads go (PCI apertures stage B) #935)", but no apertures track file exists, and the PR closes on landing. The ACPI track, which owns acpi_server_death's exit, is a lasting owner for the first.
  7. Overlap with osbench and latency images run in ACPI mode, as a shipped image does #1009. Both PRs add acpiserver to benchcase and latencycase. Whichever lands second resolves the conflict. The supervisor ranks acpiserver first whatever the list order, so the two edits are compatible.

Evidence checked

  • Metal, metal/amd/judge-b.log at 4e32df68c: 3 passed. The bench declares 16 windows in one root, and pcidev: a claim waited 21ms.
  • Metal, metal/t14/testcases: verdict passed. The T14 declares 3 windows, a claim waited 67 ms, and the log reads _OSC: native hot-plug offered, buses 0x00..=0x79, no commit.
  • Metal, metal/t14/acpicase: refused with no Boot: complete. That is the predicted outcome, and e307e6060's issue edit records it.
  • ci-host-1.log at 1ffbdbaaf: EXIT=0, 87 steps green.
  • I read every PCI root bridges are acpiserver's declaration: each root's _CRS windows and _OSC offer, held by the kernel; the loader's window reads go (PCI apertures stage B) #935 hunk against the merge base and traced:
    • answered(): the deadline returns Ok, the stall is withheld as Unanswered, and a kill is Unusable;
    • declare: records, then windows, then the RAM fence;
    • declarer_gone;
    • the _CRS reader's Word, DWord, QWord and Extended layouts, its bus rule, its _TRA rule and its consumer rule;
    • query_root's Support (bits 0 and 4) and Control (bits 0 and 4) fields, against PCI Firmware 3.3 Tables 4-4 and 4-5.

SEND BACK on items 1 to 4. The open owner decision blocks landing separately.

- iommu::DeviceSpace::create's doc and the virtio console's comment say
  what the code does today: a driver in this kernel comes up before the
  declaration and keeps the machine's addresses. The owner's decision on
  that changes the code later.
- acpi_release goes with its row and judge. Nothing ran it, and the
  issue's exit restores all three.
- The aarch64 issue says the console loses the domain main gives it.
- The two new issues are owned by the ACPI track. The PR they were named
  after closes on landing.
- Docs: MAX_WINDOWS states the AMD bench's measured 16, `Osc::Offered`
  names no stage, and the config test says which claims it does not see.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q
Japabu added a commit that referenced this pull request Oct 22, 2026
#935's review round 2.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q
…r it is listed

Since #994, SYSTEM is the first service in the list, started before the
manifest was read. The holder count taken from the list's front therefore
found none, the disk drivers' claims were made before acpiserver ran, and
they waited out the declaration's deadline. `the_log_lands_on_the_boot_stick`
went red on it after the merge of main. The declaring rows are now
partitioned out of the whole list and started first, and the session home
step makes every declarer's home.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q
@Japabu

Japabu commented Oct 22, 2026

Copy link
Copy Markdown
Collaborator Author

Round 2 at f9e44e0fb (main d7f269dbc merged).

  1. False comments: DeviceSpace::create's doc and the virtio console's comment now say what the code does: a driver in this kernel comes up before the declaration and keeps the machine's addresses. The owner's item 53 changes the code later.
  2. acpi_release: gone with its row and judge, and out of RUST_SKIP. acpi_mediated.rs no longer names it. The issue's exit restores the job, the judge and the row, and no-t14-row-arranges-… now notes that the row is unregistered.
  3. The aarch64 issue now says the console loses the domain main gives it.
  4. Evidence at the head:
    • --ci host: exit 0, 93 steps green.
    • --build-only on x86-64 and aarch64: exit 0 each.
    • a_drives_row_holds_one_process_per_function, a_claim_waits_out_an_undeclared_machine, acpi_mediated_access, virt_claim_lpi and the_log_lands_on_the_boot_stick: 5 passed.
    • This found a defect from the boot: the kernel hands the supervisor its images and parses no filesystem #994 merge. At e6dd4addf, the_log_lands_on_the_boot_stick stalled waiting for usbd. boot: the kernel hands the supervisor its images and parses no filesystem #994 puts SYSTEM first in the service list, so taking the acpi holders from the list's front found none, and the disk drivers' claims waited out the declaration's deadline.
    • f9e44e0fb picks the holders out of the whole list. Its home step makes every holder's home, which also removes the [..at] half of suggestion 4.

Suggestions:

  • Taken:
    • 1: the config test's doc says it does not see run-time device-syscap claims.
    • 2 and 3: the doc history is gone, and MAX_WINDOWS states the AMD bench's 16.
    • 6: both new issues are owned by the ACPI track.
  • Not taken:

The rows were checked against logs kept locally. Evidence publication waits on the owner.

@Japabu

Japabu commented Oct 22, 2026

Copy link
Copy Markdown
Collaborator Author

Review of #935, round 3, at f9e44e0fb

f9e44e0fb is pushed and merges cleanly with origin/main c2fa26319. Since 3d6257080 this round adds two commits:

  • e6dd4addf, the round 2 answers;
  • f9e44e0fb, the start-order fix.

Round 2 items

  1. False comments: resolved.
    • DeviceSpace::create's doc now says what the code does. A driver in this kernel is given the machine's addresses, and no domain of its own, because it comes up before any declaration.
    • The virtio console's comment says the same and points at the aarch64 issue.
    • The unreachable Own arm in create stays until the owner decides item 53. Whether it goes depends on that decision.
  2. acpi_release: resolved. The binary and its RUST_SKIP entry are deleted, and acpi_mediated.rs:352 no longer names it. The issue's exit now restores the row with both its job and its judge. no-t14-row-arranges-an-acpi-disable-asked-off-the-boot-processor.md now says that its row is unregistered, and why.
  3. Aarch64 issue wording: resolved. It now says the console loses the domain main gives it.
  4. Evidence at the merged head: resolved for host and QEMU. All of these ran at f9e44e0fb. The logs carry no revision line, but their timestamps (14:51–14:58) all fall after the commit (14:51:41) and before the next one:
    • r3-ci-host.log: EXIT=0, 93 steps green.
    • r3-build-x86.log and r3-build-arm.log: both EXIT=0.
    • r3-guests.log: 5 passed (a_drives_row_holds_one_process_per_function, acpi_mediated_access, virt_claim_lpi, the_log_lands_on_the_boot_stick, a_claim_waits_out_an_undeclared_machine, the last waiting 30006 ms).

Suggestions 1, 2, 3 and 6 are taken. The build test's doc states its scope, the two history comments state measured counts, Osc::Offered no longer names a stage, and both new issues name the ACPI track as owner.

The start-order fix (f9e44e0fb)

The defect was real:

The fix partitions the whole list into declarers and the rest, keeping their order, and is correct. I traced what it does to SYSTEM:

  • SYSTEM is the first entry of the rest.
  • is_storage holds for it, because it has a role, so it triggers no home making.
  • start_service returns early for it, because child.is_some().

start_service's home loop now walks the whole list. That is correct, since every declarer is started before the first non-storage row.

The fix changes every metal boot's start order relative to the 4e32df68c readbacks, which predate #994's boot path. The metal re-run you plan after item 53 is required.

Suggestion

  1. A guard for the start order. The order bug was caught by a QEMU row reaching the deadline, not by anything that names the order. A host test of the boot ranking would hold the invariant cheaply: every declarer is started before drive_where(is_disk_driver), wherever SYSTEM sits. Examples are undeclaredcase's refusal and the boot-stick row's 30 s. The boot logic is not host-tested today, so this is optional.

Verdict

No review blocker remains. Landing is still held by owner item 53 (aarch64 cargo run panics on its console's first queue) and by the metal re-run at the head that will land. Both are already planned.

LAND on the review's findings, once item 53 is answered and metal passes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant