Repository navigation
cputopology, hardware: report the CPU topology in the hardware inventory - #6946
Conversation
96d522a to
04c8512
Compare
|
@claude please review and include comparing with the text/comments in the eve-api defining the information reported by this PR. |
|
Claude finished @eriknordmark's task in 6m 30s —— View job Review: CPU topology in the hardware inventoryReviewed the
eve-api alignment — matches the contract
The tests are strong — notably Minor, non-blocking observations
ConclusionNo correctness, security, or performance issues found. The model→protobuf mapping is faithful to the eve-api field semantics, the degraded/error paths behave as the comments require, and the single-source-of-truth for cache ids keeps the |
04c8512 to
80b3a1d
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #6946 +/- ##
==========================================
+ Coverage 28.47% 29.06% +0.59%
==========================================
Files 540 554 +14
Lines 98258 100005 +1747
==========================================
+ Hits 27975 29065 +1090
- Misses 68009 68715 +706
+ Partials 2274 2225 -49 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
evetest skip decisionThis PR only changes Whole suites unrelated to CPU/hardware-inventory reporting:
Individual tests within
An engineer can comment |
evetest: skip decisionThis PR only changes Skipped as unrelated (Check A) — whole suites:
Skipped as unrelated within a related suite (Check A) — individual tests in
Kept: An engineer can comment |
A new package builds, from the facts ghw reads, what placing workloads on whole cores needs: the physical cores, each with its SMT siblings, NUMA node and the L2 and L3 instances it uses, and the offline CPUs. A core is told by its socket and ghw's core id, never by a shared cache: the four cores of an Intel E-core module share one L2 and stay four cores. Cache instances are named by the kernel's id when every instance of their level and type has one, otherwise by the lowest CPU sharing them, so caches an ARM device tree does not number are still told apart and the two kinds of name never mix. When ghw cannot list the CPUs, or an online CPU's socket, core or NUMA node is unknown, Discover returns a degraded model, each online CPU its own core, and the error. It can be reported but must not be placed on. The ghw facts stay attached for reporting whenever ghw listed the CPUs. FromInfo builds the model from facts already read, as Discover does once ghw has read them. KernelIsolation reads the CPUs the running kernel isolates and runs tickless, under the sysfs root a ghw context sets, as ghw does. It is how the kernel was booted rather than what the CPUs are, so it stays outside the model. Signed-off-by: Mikhail Malyshev <mike.malyshev@gmail.com>
The CPU inventory emitted one entry per physical core with the core id in the field meant for a logical CPU id, and no topology. The ids were not the ones CPU affinities are expressed in, and the SMT structure, which a consumer reasoning about CPU placement needs most, was absent. It now reports one entry per logical CPU with its socket, physical core, NUMA node and L2 and L3 cache ids as cputopology models them, the model EVE places workloads on, so the report and the placement cannot drift. Each entry also carries its core class, its base and maximum frequency where the kernel exposes them, and the model and vendor of its own package. A value EVE cannot read is left unset, never reported as zero. When the topology is degraded, the CPUs are still listed with what was read about each, but without coordinates or caches: unset is how the API says the topology is unknown, where zeros would describe one socket of single-thread cores. Cache instances are reported with the CPUs sharing each, which tells a consumer which workloads contend for the same cache: level 1 as its data and instruction caches, levels 2 and 3 by their unified, else data, cache. An instance is named by the kernel's id when every instance of its level and type has one, otherwise, as on many ARM64 platforms, by the lowest CPU sharing it; a CPU's l2_id and l3_id name the instance that lists it. Kernel CPU isolation is reported as a node fact (KernelCPUIsolation): the CPUs the kernel isolates (isolcpus) and runs without the periodic tick (nohz_full), read from sysfs rather than the command line so it describes what the kernel does. A set that cannot be read is an error and leaves the report unset: a node whose isolated set is unreadable is not one without isolation. PCI devices report their local CPUs. The facts come from ghw through cputopology; hardware no longer calls ghw.CPU itself. Signed-off-by: Mikhail Malyshev <mike.malyshev@gmail.com>
80b3a1d to
da556f3
Compare
|
Based on the scope of this PR (a new Unrelated suites (skipped entirely):
Unrelated individual tests (within
An engineer can comment |
Description
The hardware inventory reports one CPU entry per physical core, with the core id in the field meant for a logical CPU id, and nothing about the topology. This PR makes it report the CPU topology the eve-api has had since lf-edge/eve-api#155, from a new model of the CPUs that CPU placement (#6335) will build on, so the report and the placement cannot drift.
Two commits:
cputopology: model the CPUs EVE places workloads on: a new pillar package, built from the CPU facts zededa/ghw reads (pinned by pkg/pillar: update zededa/ghw for the CPU topology facts #6942). It groups the online CPUs into physical cores by socket and ghw's core id, never by a shared cache (the four cores of an Intel E-core module share an L2 and stay four cores), with each core's NUMA node and L2/L3 cache instances. Cache instances are named by the kernel's id when every instance of their level and type has one, otherwise by the lowest CPU sharing them, so caches an ARM device tree does not number are still told apart. When the CPUs cannot be listed, or a CPU's socket, core or NUMA node is unknown, it returns a degraded model (each CPU its own core) and the error: it can be reported, but must never be placed on.KernelIsolationreads the CPUs the running kernel isolates (isolcpus) and runs without the tick (nohz_full) from sysfs.hardware: report the CPU topology, caches and kernel isolation: the inventory reports one entry per logical CPU with its socket, core, NUMA node, L2 and L3 ids, core class, base and maximum frequency where the kernel exposes them, and its own package's model and vendor; the cache instances (L1d, L1i, L2, L3) with the CPUs sharing each;KernelCPUIsolation; and each PCI device's local CPUs. A value EVE cannot read is left unset, never reported as zero. A degraded topology still lists every CPU, without coordinates or caches.hardwareno longer callsghw.CPUitself.Nothing in pillar consumes the CPU inventory; it only goes to the controller.
How to test and validate this PR
Automated:
make -C pkg/pillar TESTPKGS="./hardware/... ./cputopology/..." testandmake pillar-vet HV=kvm.cputopologyis tested on sysfs fixture trees read through ghw: an SMT host with two sockets and two NUMA nodes, ARM clusters whosecore_idrestarts per cluster and whose caches have no ids, an unknown NUMA node (degraded), no CPUs listed, offline CPUs, and the isolation files (missing, empty, malformed).hardwaremaps a synthetic two-package host, its degraded form and the no-facts case to the expected protobuf.TestCreateInventorychecks the inventory of the machine the test runs on against sysfs: every online CPU is listed once; CPUs reporting the same socket and core are exactly the sysfs thread siblings; each CPU's NUMA node lists it; each CPU'sl2_id/l3_idis the id of the cache domain listing it; every cache domain lists online CPUs only.On a device, compare the hardware inventory it reports with the host:
lscpu -e=CPU,SOCKET,CORE,NODE: every logical CPU appears with the same socket and NUMA node, and CPUslscpuputs on one core sharesocket_idandcore_id.kernel_cpu_isolationmatches/sys/devices/system/cpu/isolatedandnohz_full(boot withisolcpus=to see a non-empty set).local_cpu_idsmatch/sys/bus/pci/devices/<address>/local_cpulist.Changelog notes
The hardware inventory reports the CPU topology: one entry per logical CPU (previously one per physical core) with its socket, core, NUMA node, L2 and L3 cache, core class and frequencies; the CPU caches with the CPUs sharing each; the CPUs the kernel isolates or runs without the scheduler tick; and the CPUs local to each PCI device.
PR Backports
Checklist
I've provided a proper description
I've added the proper documentation
I've tested my PR on amd64 device
I've tested my PR on arm64 device
I've written the test verification instructions
I've set the proper labels to this PR
I've checked the boxes above, or I've provided a good reason why I didn't check them.