Skip to content

cputopology, hardware: report the CPU topology in the hardware inventory - #6946

Merged
eriknordmark merged 2 commits into
lf-edge:masterfrom
rucoder:rucoder/hwinventory-cpu
Oct 30, 2026
Merged

eriknordmark merged 2 commits into
lf-edge:masterfrom
rucoder:rucoder/hwinventory-cpu

Conversation

@rucoder

@rucoder rucoder commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Description

The hardware inventory reports one CPU entry per physical core, with the core id in the field meant for a logical CPU id, and nothing about the topology. This PR makes it report the CPU topology the eve-api has had since lf-edge/eve-api#155, from a new model of the CPUs that CPU placement (#6335) will build on, so the report and the placement cannot drift.

Two commits:

  1. cputopology: model the CPUs EVE places workloads on: a new pillar package, built from the CPU facts zededa/ghw reads (pinned by pkg/pillar: update zededa/ghw for the CPU topology facts #6942). It groups the online CPUs into physical cores by socket and ghw's core id, never by a shared cache (the four cores of an Intel E-core module share an L2 and stay four cores), with each core's NUMA node and L2/L3 cache instances. Cache instances are named by the kernel's id when every instance of their level and type has one, otherwise by the lowest CPU sharing them, so caches an ARM device tree does not number are still told apart. When the CPUs cannot be listed, or a CPU's socket, core or NUMA node is unknown, it returns a degraded model (each CPU its own core) and the error: it can be reported, but must never be placed on. KernelIsolation reads the CPUs the running kernel isolates (isolcpus) and runs without the tick (nohz_full) from sysfs.
  2. hardware: report the CPU topology, caches and kernel isolation: the inventory reports one entry per logical CPU with its socket, core, NUMA node, L2 and L3 ids, core class, base and maximum frequency where the kernel exposes them, and its own package's model and vendor; the cache instances (L1d, L1i, L2, L3) with the CPUs sharing each; KernelCPUIsolation; and each PCI device's local CPUs. A value EVE cannot read is left unset, never reported as zero. A degraded topology still lists every CPU, without coordinates or caches. hardware no longer calls ghw.CPU itself.

Nothing in pillar consumes the CPU inventory; it only goes to the controller.

How to test and validate this PR

Automated:

  • make -C pkg/pillar TESTPKGS="./hardware/... ./cputopology/..." test and make pillar-vet HV=kvm.
  • cputopology is tested on sysfs fixture trees read through ghw: an SMT host with two sockets and two NUMA nodes, ARM clusters whose core_id restarts per cluster and whose caches have no ids, an unknown NUMA node (degraded), no CPUs listed, offline CPUs, and the isolation files (missing, empty, malformed).
  • hardware maps a synthetic two-package host, its degraded form and the no-facts case to the expected protobuf.
  • TestCreateInventory checks the inventory of the machine the test runs on against sysfs: every online CPU is listed once; CPUs reporting the same socket and core are exactly the sysfs thread siblings; each CPU's NUMA node lists it; each CPU's l2_id/l3_id is the id of the cache domain listing it; every cache domain lists online CPUs only.

On a device, compare the hardware inventory it reports with the host:

  • lscpu -e=CPU,SOCKET,CORE,NODE: every logical CPU appears with the same socket and NUMA node, and CPUs lscpu puts on one core share socket_id and core_id.
  • kernel_cpu_isolation matches /sys/devices/system/cpu/isolated and nohz_full (boot with isolcpus= to see a non-empty set).
  • A PCI device's local_cpu_ids match /sys/bus/pci/devices/<address>/local_cpulist.

Changelog notes

The hardware inventory reports the CPU topology: one entry per logical CPU (previously one per physical core) with its socket, core, NUMA node, L2 and L3 cache, core class and frequencies; the CPU caches with the CPUs sharing each; the CPUs the kernel isolates or runs without the scheduler tick; and the CPUs local to each PCI device.

PR Backports

  • 17.0-stable: No, a new feature.
  • 16.0-stable: No, a new feature.
  • 14.5-stable: No, a new feature.
  • 13.4-stable: No, a new feature.

Checklist

  • I've provided a proper description

  • I've added the proper documentation

  • I've tested my PR on amd64 device

  • I've tested my PR on arm64 device

  • I've written the test verification instructions

  • I've set the proper labels to this PR

  • I've checked the boxes above, or I've provided a good reason why I didn't check them.

@eriknordmark

Copy link
Copy Markdown
Contributor

@claude please review and include comparing with the text/comments in the eve-api defining the information reported by this PR.

@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @eriknordmark's task in 6m 30s —— View job


Review: CPU topology in the hardware inventory

Reviewed the cputopology and hardware commits (not the vendored ghw / usbmanager commits), cross-checking the mapping against the field comments in eve-api info/hardware.proto (vendored hardware.pb.go).

  • Read cputopology (discover, topology, isolation) + tests
  • Read hardware/inventory.go changes + tests
  • Compare field-by-field with the eve-api comments
  • Check the degraded / error paths and the caller in zedagent

eve-api alignment — matches the contract

eve-api field (comment) implementation verdict
CPU — one entry per online logical CPU iterates topo.HW.LogicalProcessors (ascending) ✅
socket_id/core_id "SMT siblings share (socket_id, core_id)" core.Socket/core.ID; siblings grouped by them, never by a shared cache (E-core module test) ✅
"topology known only when every CPU carries socket_id and core_id … else leave unset and report no caches" any CPU with unknown socket/core/NUMA → Degraded, all coordinates withheld, cacheIDs nil → no caches ✅ all-or-nothing
numa_node "0 on a kernel without NUMA" ghw returns 0 when no node dir; <0 (partial) degrades ✅
l2_id/l3_id "CacheDomain.id of its L2/L3; unset if none" cacheID() → unset unless Known; same ids map feeds both sides ✅ consistent
CacheDomain.id "opaque key; kernel id or … " kernel id when every instance of a level+type has one, else lowest CPU — uniform per level, so CPU↔cache ids never mix ✅
core_class, base_freq_khz, max_freq_khz "unset when kernel doesn't expose" coreClasses map + known() (0 → unset) ✅
PCIDevice.local_cpu_ids "cross-references CPU.id" pci.LocalCPUs passed through ✅
KernelCPUIsolation "empty list = none; whole thing = booted state" reads isolated/nohz_full; missing file → empty, unreadable/malformed → error → message unset ✅
degraded report is still sent zedagent.go:431 keeps hwInventory and only logs the error ✅ matches "can be reported, never placed on"

The tests are strong — notably TestCreateInventory/checkCPUInfo validate the live host against sysfs (siblings, NUMA membership, and CPU l2_id/l3_id ↔ CacheDomain reciprocity), which is exactly the cross-field consistency the API comments promise.

Minor, non-blocking observations

  • Caches the API enum can't represent are silently omitted. The cache-level mapping drops a unified L1 and any L4/eDRAM (the CacheLevel enum only has L1D/L1I/L2/L3), so such a cache appears nowhere in the inventory. This is the only correct behavior given the API, and such caches are rare on EVE targets — just flagging that it's a deliberate, silent drop rather than an oversight.
  • Degrade blast radius. A single CPU missing its socket/core/NUMA in sysfs (e.g. a hotplug race) blanks the coordinates and all cache domains for the whole host, not just that CPU. This is intentional and required by the API's all-or-nothing contract (and by placement safety), so no change requested — noting it so it's a conscious tradeoff.

Conclusion

No correctness, security, or performance issues found. The model→protobuf mapping is faithful to the eve-api field semantics, the degraded/error paths behave as the comments require, and the single-source-of-truth for cache ids keeps the CPU.l*_id ↔ CacheDomain.id cross-references consistent. The PR-dependency caveats (rebase after #6942/#6943) are already called out by the author.
· pr-6946

@rucoder
rucoder force-pushed the rucoder/hwinventory-cpu branch from 04c8512 to 80b3a1d Compare October 29, 2026 10:35
@codecov

codecov Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.75758% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 29.06%. Comparing base (a273c3e) to head (da556f3).
⚠️ Report is 11 commits behind head on master.

Files with missing lines Patch % Lines
pkg/pillar/hardware/inventory.go 95.16% 1 Missing and 2 partials ⚠️
pkg/pillar/cputopology/discover.go 96.72% 1 Missing and 1 partial ⚠️
pkg/pillar/cputopology/isolation.go 85.71% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6946      +/-   ##
==========================================
+ Coverage   28.47%   29.06%   +0.59%     
==========================================
  Files         540      554      +14     
  Lines       98258   100005    +1747     
==========================================
+ Hits        27975    29065    +1090     
- Misses      68009    68715     +706     
+ Partials     2274     2225      -49     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@rene rene left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions

Copy link
Copy Markdown
Contributor

evetest skip decision

This PR only changes pkg/pillar/hardware/inventory.go (the CPU topology/cache/PCI-local-CPU fields of the ZInfoHardware report) and adds a new, pillar-unit-tested pkg/pillar/cputopology package that nothing else in pillar consumes yet. The one evetest test whose purpose is to validate ZInfoHardware (TestHardwareInventory, in TestTelemetrySuite) is kept. Skipping the following as unrelated to this change:

Whole suites unrelated to CPU/hardware-inventory reporting:

  • TestApplicationConnectivitySuite — app networking
  • TestAppsSuite — app lifecycle (including TestContainerdCPUs, which checks cpuset cgroup assignment via ctrd_max_vcpus, not topology reporting)
  • TestBootstrapSuite — device bootstrap/proxy/VLAN config
  • TestControllerFaultsSuite — controller connectivity fault handling
  • TestDatastoreSuite — image datastores
  • TestDeviceConnectivitySuite — network connectivity
  • TestDiagSuite — diag output
  • TestHardwareSuite — USB hotplug enumeration, not CPU/PCI topology
  • TestKonvertSuite — KVM-to-kubevirt conversion
  • TestKubeRegistrationSuite — kube cluster registration
  • TestLPSSuite — local profile server
  • TestNodeClusterResilienceSuite / TestNodeClusterSuite — clustering
  • TestPcibackErrorSuite — assignable-adapter (ZioBundle) config-mismatch errors, not ZInfoHardware
  • TestPortScanSuite / TestExternalPortScan — port scanning
  • TestSecuritySuite — security features (AppArmor, vTPM, vault, certs)
  • TestStorageSuite — volumes/storage
  • TestUpgradeSuite — EVE upgrades

Individual tests within TestTelemetrySuite (kept TestHardwareInventory, the dedicated test for this change):

  • TestDeviceInfo, TestDeviceMetrics — assert on ZInfoDevice/ZInfoMetric fields untouched by this diff
  • TestDeviceLogs, TestWorkingConfig, TestFaultyConfig, TestEmptyConfig, TestInvalidBase64Config, TestRemoteLogLevelNone, TestKernelPanicReport — exercise the newlogd/vector logging and config-apply pipeline, unrelated to CPU topology reporting

An engineer can comment /evetest all to force the full suite to run instead, or /evetest <name> <name> ... to run only specific tests/suites.

@github-actions

Copy link
Copy Markdown
Contributor

evetest: skip decision

This PR only changes pkg/pillar/cputopology (new package) and pkg/pillar/hardware/inventory.go's getCPUInfo/getKernelCPUIsolation/PCI-local-CPUs reporting — a pure hardware-inventory (ZInfoHardware) reporting change that nothing else in pillar consumes (per the PR description).

Skipped as unrelated (Check A) — whole suites:

  • TestApplicationConnectivitySuite, TestDeviceConnectivitySuite — networking/connectivity, untouched by this PR.
  • TestAppsSuite — app lifecycle, untouched.
  • TestBootstrapSuite — device bootstrap/config, untouched.
  • TestControllerFaultsSuite — controller fault handling, untouched.
  • TestDatastoreSuite — datastore/downloads, untouched.
  • TestDiagSuite — diag output, untouched.
  • TestHardwareSuite — USB hotplug (not CPU/inventory related despite the name), untouched.
  • TestKonvertSuite, TestKubeRegistrationSuite — kubevirt conversion/registration, untouched.
  • TestLPSSuite — local profile server, untouched.
  • TestNodeClusterResilienceSuite, TestNodeClusterSuite — clustering, untouched.
  • TestPcibackErrorSuite — PCI passthrough assignment-error reporting (not the hardware inventory's PCI device list), untouched.
  • TestPortScanSuite — port scanning, untouched.
  • TestSecuritySuite — AppArmor/vTPM/vault/certs, untouched.
  • TestStorageSuite — storage, untouched.
  • TestUpgradeSuite — EVE upgrade, untouched.

Skipped as unrelated within a related suite (Check A) — individual tests in TestTelemetrySuite:

  • TestDeviceInfo, TestDeviceMetrics, TestDeviceLogs, TestWorkingConfig, TestFaultyConfig, TestEmptyConfig, TestInvalidBase64Config, TestRemoteLogLevelNone, TestKernelPanicReport — exercise other telemetry paths, not the CPU topology/hardware inventory this PR changes.

Kept: TestHardwareInventory (in TestTelemetrySuite) — the dedicated test asserting ZInfoHardware/CpuInfo fields against the device's real hardware, i.e. the primary/most direct coverage for this change. No other suite or subtest references CpuInfo/HardwareInventory/cputopology, so Check B found no additional redundant coverage to prune.

An engineer can comment /evetest all to force the full suite to run instead, or /evetest <name> <name> ... to run only specific tests/suites.

A new package builds, from the facts ghw reads, what placing workloads on
whole cores needs: the physical cores, each with its SMT siblings, NUMA node
and the L2 and L3 instances it uses, and the offline CPUs. A core is told by
its socket and ghw's core id, never by a shared cache: the four cores of an
Intel E-core module share one L2 and stay four cores.

Cache instances are named by the kernel's id when every instance of their
level and type has one, otherwise by the lowest CPU sharing them, so caches
an ARM device tree does not number are still told apart and the two kinds of
name never mix.

When ghw cannot list the CPUs, or an online CPU's socket, core or NUMA node
is unknown, Discover returns a degraded model, each online CPU its own core,
and the error. It can be reported but must not be placed on. The ghw facts
stay attached for reporting whenever ghw listed the CPUs. FromInfo builds
the model from facts already read, as Discover does once ghw has read them.

KernelIsolation reads the CPUs the running kernel isolates and runs tickless,
under the sysfs root a ghw context sets, as ghw does. It is how the kernel
was booted rather than what the CPUs are, so it stays outside the model.

Signed-off-by: Mikhail Malyshev <mike.malyshev@gmail.com>
The CPU inventory emitted one entry per physical core with the core id in
the field meant for a logical CPU id, and no topology. The ids were not
the ones CPU affinities are expressed in, and the SMT structure, which a
consumer reasoning about CPU placement needs most, was absent.

It now reports one entry per logical CPU with its socket, physical core,
NUMA node and L2 and L3 cache ids as cputopology models them, the model
EVE places workloads on, so the report and the placement cannot drift.
Each entry also carries its core class, its base and maximum frequency
where the kernel exposes them, and the model and vendor of its own
package. A value EVE cannot read is left unset, never reported as zero.
When the topology is degraded, the CPUs are still listed with what was
read about each, but without coordinates or caches: unset is how the API
says the topology is unknown, where zeros would describe one socket of
single-thread cores.

Cache instances are reported with the CPUs sharing each, which tells a
consumer which workloads contend for the same cache: level 1 as its data
and instruction caches, levels 2 and 3 by their unified, else data,
cache. An instance is named by the kernel's id when every instance of its
level and type has one, otherwise, as on many ARM64 platforms, by the
lowest CPU sharing it; a CPU's l2_id and l3_id name the instance that
lists it.

Kernel CPU isolation is reported as a node fact (KernelCPUIsolation): the
CPUs the kernel isolates (isolcpus) and runs without the periodic tick
(nohz_full), read from sysfs rather than the command line so it describes
what the kernel does. A set that cannot be read is an error and leaves
the report unset: a node whose isolated set is unreadable is not one
without isolation.

PCI devices report their local CPUs.

The facts come from ghw through cputopology; hardware no longer calls
ghw.CPU itself.

Signed-off-by: Mikhail Malyshev <mike.malyshev@gmail.com>
@rucoder
rucoder force-pushed the rucoder/hwinventory-cpu branch from 80b3a1d to da556f3 Compare October 29, 2026 18:06
@github-actions

Copy link
Copy Markdown
Contributor

Based on the scope of this PR (a new cputopology package + hardware/inventory.go reporting CPU topology/cache/isolation data to the controller, with no consumers elsewhere in pillar), evetest will skip the following as unrelated to the changed behavior:

Unrelated suites (skipped entirely):

  • TestApplicationConnectivitySuite — app/NI networking, unrelated.
  • TestAppsSuite — app lifecycle/VNC/cgroups/containerd CPU cgroup assignment, not CPU topology/inventory reporting.
  • TestBootstrapSuite — onboarding/bootstrap config paths, unrelated.
  • TestControllerFaultsSuite — controller connectivity fault handling, unrelated.
  • TestDatastoreSuite — image download/datastore handling, unrelated.
  • TestDeviceConnectivitySuite — network port/connectivity behavior, unrelated.
  • TestDiagSuite — diag output tool, unrelated.
  • TestHardwareSuite — USB hotplug tests; despite the name, unrelated to CPU/hardware-inventory.
  • TestKonvertSuite — KVM-to-eve-k conversion, unrelated.
  • TestKubeRegistrationSuite — cluster registration, unrelated.
  • TestLPSSuite — local profile server, unrelated.
  • TestNodeClusterResilienceSuite / TestNodeClusterSuite — kube cluster behavior, unrelated.
  • TestPcibackErrorSuite — PhysicalIO/ZioBundle assignment-error reporting, a different PCI data path than the new local_cpu_ids field.
  • TestPortScanSuite — external port scanning, unrelated.
  • TestSecuritySuite — apparmor/vTPM/certs, unrelated.
  • TestStorageSuite — volumes/disks, unrelated.
  • TestUpgradeSuite — full EVE upgrade flow, unrelated to inventory content.

Unrelated individual tests (within TestTelemetrySuite, which also contains the directly relevant TestHardwareInventory and TestDeviceInfo — kept):

  • TestDeviceMetrics — DeviceMetric CPU/memory/network counters, not CPU topology data.
  • TestDeviceLogs — log reporting, unrelated.
  • TestWorkingConfig, TestFaultyConfig, TestEmptyConfig, TestInvalidBase64Config — config-application outcome tests, unrelated.
  • TestRemoteLogLevelNone — remote log level handling, unrelated.
  • TestKernelPanicReport — kernel panic reporting, unrelated.

TestHardwareInventory is kept as the dedicated test for the CPU topology/hardware inventory this PR changes, and TestDeviceInfo is kept since it separately validates ZInfoDevice's own hardware-inventory floor checks alongside its primary network/HSM focus.

An engineer can comment /evetest all to force the full suite to run instead, or /evetest <name> <name> ... to run only specific tests/suites.

@eriknordmark
eriknordmark merged commit ea360bb into lf-edge:master Oct 30, 2026
76 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants