hw-iommu: unmap notifiers on the SMMUv3

[tamnd]

Part of #9.

What

IOMMU notifiers on the SMMUv3, on the notifier API from #330, as in QEMU 11.1's hw/arm/smmuv3.c and smmu-common.c.

• The IOMMU region of each device behind the SMMU now has a notifier list. smmuv3_notify_flag_changed() is ported: a notifier asking for MAP or DEVIOTLB_UNMAP events is refused with QEMU's message, and the device goes on or off devices_with_notifiers as its first notifier comes or its last one goes.
• The TLB invalidation commands send unmap events, as smmuv3_inv_notifiers_iova(), smmuv3_notify_iova() and smmu_inv_notifiers_all() do:
  • TLBI_NH_VA, TLBI_NH_VAA and TLBI_S2_IPA unmap their range. Without a range (TG = 0) that is one page of the device's granule, and only when the ASID and VMID match its configuration. A range is split into the same powers of two as the IOTLB invalidation.
  • With nested translation a stage 2 invalidation unmaps nothing, since only stage 1 addresses are IOVAs.
  • TLBI_NH_ASID, TLBI_NSNH_ALL, TLBI_NH_ALL without stage 2, and TLBI_S12_VMALL unmap the whole range of each notifier. TLBI_NH_ALL with stage 2 sends nothing, as in QEMU.
• The stream ID of a device comes from its bus number, which the PCI side holds, so the stream IDs of the devices with notifiers are worked out before the SMMU state is locked. The unmaps go out once the lock is dropped and before the CMD_SYNC interrupt, the same way intel-iommu defers its events.

Replay

There is no SMMUv3 replay, as in QEMU. hw/arm/smmuv3.c has no replay callback. The only callers of memory_region_iommu_replay() are VFIO and vhost-vdpa, and both register MAP notifiers, which the SMMUv3 refuses. So the default IommuOps::replay of #330, which does nothing, is the right one here.

What it changes for guests

Nothing in ruvm registers an unmap-only notifier behind the SMMU yet: vfio-pci is on q35 only, and it needs MAP anyway. So this is the device model side, tested at the unit level. To check that the translation path still works, the root port boot from #336 ran on QEMU 11.1 and on this build: an Alpine 6.12 guest with -M virt,iommu=smmuv3,gic-version=3 -cpu max -m 1G -smp 1 and a virtio-blk-pci with iommu_platform=on behind a pcie-root-port. It takes the md5 of /dev/vda, writes a sector, drops caches, reads it back and takes the md5 again.

Boot	QEMU 11.1	this build
DT	md5 95934ad2... then 269d53a2..., sector read back	the same
ACPI (edk2)	md5 95934ad2... then 269d53a2..., sector read back	the same, and the same IORT bytes

The host image md5 after the run was 269d53a2fae97c0c7de47a8a3d6776b2 on all four boots. The guest's PCI addresses differ by the default NIC (a virtio-net at 00:01.0), which ruvm does not add. The ruvm boots used a release build of this diff on e7c7db9; #342, which came in after, migrates the ACPI GED and does not touch hw-iommu.

Not done here

• RMR, ATS, secure state and VMState come next.
• Accelerated nested mode (accel=on, CMDQV) needs host SMMU hardware we do not have, so it comes last.

Tests

• notifiers_take_unmap_only: MAP and DEVIOTLB_UNMAP notifiers are refused with QEMU's messages and leave nothing on the list. An UNMAP notifier goes on the list and comes off it again.
• stage1_invalidations_unmap_for_the_notifiers: one page for a matching ASID and nothing for another, a range split into its powers of two, whole-range unmaps for TLBI_NH_ASID and TLBI_NSNH_ALL, nothing for CFGI_STE, and nothing once the notifier is gone.
• stage2_invalidations_unmap_for_the_notifiers: one page for a matching VMID and nothing for another, nothing for TLBI_NH_ALL with stage 2, a whole-range unmap for TLBI_S12_VMALL, and with nesting nothing for TLBI_S2_IPA but a page for TLBI_NH_VA.
• On 8fb1fee plus this diff, on Linux x86_64: cargo fmt --check -p ruvm-hw-iommu; clippy -D warnings on hw-iommu, machine-arm and system; cargo check --workspace --all-targets; cargo test -p ruvm-system --lib (113 passed); the tests of hw-iommu, mem, machine-arm, system, hw-pci and hw-virtio, and the linux-user x86 suite (user-common, linux-user, jit-x86_64, jit-core), 770 passed and 0 failed in all; the provenance, unsafe-audit, layers and style xtasks; cargo check --target x86_64-pc-windows-msvc on hw-iommu, machine-arm and system with no warnings.