vfio-pci: mmap the BARs into the guest

[tamnd]

Part of #9.

vfio-pci now maps the BARs of the device into the guest, like vfio_region_mmap() in QEMU. Reads and writes to a mapped BAR go to the device memory without a pread or pwrite per access. While INTx is pending the mmaps are off, so the guest's next BAR access traps and serves as the EOI, and a timer on the virtual clock turns them back on once INTx is quiet, as vfio_intx_mmap_enable() does.

What is in it:

• ruvm-mem: a RAM device region type for host device memory, made with memory_region_init_ram_device_ptr(), that the vfio listener maps for DMA like RAM.
• ruvm-sys: the region info capabilities (sparse mmap areas and MSIX_MAPPABLE), VFIO_DEVICE_FEATURE_DMA_BUF, and HostMemory::map_device(), which reserves the whole region and maps each area into it.
• vfio-pci: the sparse mmap areas as vfio_setup_region_sparse_mmaps() reads them, with size 0 areas dropped, the areas sorted, and QEMU's error for overlapping areas. Each area becomes a "NAME BAR n mmaps[i]" subregion of the trapped BAR, then the dma-buf probe runs with QEMU's two warnings. A failed mmap prints "Failed to mmap NAME BAR n. Performance may be slow" and the BAR stays trapped. Without MSIX_MAPPABLE the pages of the MSI-X table are cut out of the mmaps, as vfio_pci_fixup_msix_region() does. INTx turns the mmaps off and arms the timer, the timer rearms while INTx is pending, and turning INTx off deletes the timer and turns the mmaps on. The container maps the RAM device sections for DMA, and a failure there is a warning with QEMU's peer to peer hint instead of an error.
• system: x-no-mmap and x-intx-mmap-timeout-ms now do what they do in QEMU, and the virtual clock is passed to realize for the timer.

Where it differs from QEMU:

• Under TCG, ruvm reads and writes a RAM device region through its IO callbacks, not through the TLB. There are no KVM, HVF or WHPX slots for it yet.
• vfio_sub_page_bar_update_mapping() is not ported, so a BAR smaller than a host page stays trapped.
• The page sizes for the DMA map check come from the host page size, as VFIO_IOMMU_GET_INFO is not ported.
• x-msix-relocation is not ported.

Checked:

On server2, on 9596e67, the branch with main up to #336 merged in:

• clippy with -D warnings and --all-targets for ruvm-mem, ruvm-hw-iommu, ruvm-sys, ruvm-hw-pci, ruvm-hw-vfio, ruvm-machine-x86, ruvm-system and ruvm-machine-arm.
• clippy for x86_64-pc-windows-gnu and aarch64-apple-darwin on the same crates.
• tests of ruvm-mem, ruvm-hw-iommu, ruvm-sys, ruvm-hw-pci and ruvm-hw-vfio, and the ruvm-machine-x86, ruvm-system and ruvm-machine-arm libs. New tests cover the sparse mmap areas, the MSI-X table cut and the INTx timer on a manual clock.
• xtask unsafe-audit, provenance, layers and style.

The same checks without ruvm-machine-arm, the release build and the L1 compare below ran on 7b2159e, the commit before that merge. The merge brings in #335 and #336, which touch only Arm files.

• An L1 compare on the 7b2159e release build. L1 is QEMU 11.1 with intel-iommu,caching-mode=on and an e1000e at 00:04.0 bound to vfio-pci. L2 is run with -device vfio-pci,host=0000:00:04.0,addr=2 under qtest, as QEMU 11.1 and ruvm, each with and without x-no-mmap=on, and then with x-intx-mmap-timeout-ms=0. Each batch waits for all of its answers, and the syscall counts are the syscr deltas of the L2 process from /proc.

Part A, with -S, where the virtual clock does not move. qtest places the BARs, reads and writes BAR0, BAR1, the IO BAR and the MSI-X BAR, sends a frame by DMA, programs the IOAPIC and raises INTx.

Step	QEMU 11.1	ruvm	QEMU x-no-mmap	ruvm x-no-mmap
mmaps of the device fd	BAR0 0x20000, BAR1 0x20000, BAR3 0x4000	same	none	none
300 STATUS reads	10	5	310	305
300 STATUS reads after INTx	310	305	310	305

All 71 answers are the same in the four runs. Both print "VFIO dma-buf not supported in kernel, using mmap fallback, P2P DMA will not work", as the L1 kernel has no dma-buf for vfio. ruvm prints it without QEMU's -device vfio-pci,host=0000:00:04.0,addr=2: prefix. The MSI-X BAR is mapped whole in both, since the kernel reports MSIX_MAPPABLE.

Part B, in a guest that runs, so the timer can fire. SeaBIOS places BAR0 at 0xfeba0000 in all four runs. qtest raises INTx, waits 3 seconds, does one STATUS read, then reads ICR, masks the interrupt and does 300 reads, waits 3 seconds and does 300 reads more.

Step	QEMU 11.1	ruvm	QEMU timeout 0	ruvm timeout 0
300 reads before INTx	16	6	16	6
one read 3 s after INTx	5, trapped	3, trapped	4, trapped	3, trapped
ICR and 300 reads	319	307	318	307
300 reads 3 s later	14	5	314	305

The INTx step itself is two writes at 4, 2, 3 and 2, so the one read adds a pread in every run. ICR reads 0x80000001 in all four. The mmaps are off while INTx is pending, back on after the timeout, and stay off with a timeout of 0, in QEMU and in ruvm. How many of the 300 reads right after the EOI trap depends on when the next timer tick lands, and was between 74 and 319 over the debug runs for both.