test(orchestrator): adopt per-test logger contract in pxeboot suites by balajikumaran-c-s · Pull Request #5497 · dell/omnia

balajikumaran-c-s · 2026-10-09T19:33:30Z

PR Title

test(orchestrator): adopt per-test logger contract in pxeboot suites

Description of the Solution

This is part 3 of 3 of the Orchestrator test automation update. It moves every pxeboot suite to the per-test logger contract and its order block. It also makes reboot and node remove/add tests opt-in through their own markers, and moves the external LDAP checks from prepare to the pxeboot slurm_ldap suite.

Related Issue

Related to [Feature request] Omnia Modernization #4849

Depends on parts 1 (#5495) and 2 (#5496) and contains their commits. Until they merge, review only the last commit here (test(orchestrator): adopt per-test logger contract in pxeboot suites).

Changes

pxeboot suites (`test/orchestrator/fvt/pxeboot`)

Every suite creates its own TestLogger, resolves its registry entry, and calls verify_pxeboot(test_log, case, host, checker).
Suites move into the 40100-42400 order blocks in dependency order: reachability and node state, cluster health, temporary workloads, then reboot and node-removal suites.
Negative cases leave the sanity marker set; feature markers are function-level.
Kubernetes: local etcd checks require the boot script DONE marker from the current boot, a GPT etcd disk, and the configured disk media; control-plane reboot recovery waits for etcd health.
The PowerVault and HPC benchmark suite conftests no longer register markers; the root conftest registers them.

Opt-in marker gating (`test/orchestrator/conftest.py`, `_run.py`, `library/vars/domain_vars.py`)

Reboot tests carry reboot and node remove/add tests carry the new node_lifecycle marker. Both are deselected unless --marker selects them, even when the suite is named; the disruptive marker is removed.
With no marker every other test runs: the runner no longer forces a sanity default for pxeboot, and full verify runs no longer exclude negative cases.
Skips raised before a test starts are logged with the test ID, title and reason, and reach the summary table.
Registered the benchmark, boot_image and powervault_* markers; SUITES lists oim_readiness and places the reboot and lifecycle suites last.

External LDAP move (`library/functions`, `fvt/prepare/openldap`, `fvt/pxeboot/slurm_ldap`)

external_ldap_pxeboot_func.py reconciles and verifies the LDAP proxy and backend; the new test_external_ldap.py runs them ahead of the LDAP login tests (ORCH_FVT_PXEBOOT_V524, V525).
Removed the external LDAP cases and helpers from the prepare OpenLDAP suite; openldap_enabled is now public for reuse.

Result contract (`test/orchestrator/fvt/result.py`)

verify_pxeboot takes the test's TestLogger and registry case, completing the contract from part 2.

Documentation (`test/orchestrator/README.md`, `test/orchestrator/docs/test_run_config.md`)

Documents that markers narrow a run, and that reboot and node remove/add tests need --marker reboot or --marker node_lifecycle.

Files Changed

File	Change Type	Description
`test/orchestrator/README.md`	Modified	Marker and opt-in rules
`test/orchestrator/_run.py`	Modified	Remove forced pxeboot `sanity` default
`test/orchestrator/conftest.py`	Modified	Opt-in gating, marker registry, pre-test skip logging
`test/orchestrator/docs/test_run_config.md`	Modified	Marker reference
`test/orchestrator/fvt/result.py`	Modified	Final `verify_pxeboot` contract
`test/orchestrator/fvt/prepare/openldap/test_openldap.py`	Modified	Drop external LDAP cases
`test/orchestrator/fvt/pxeboot/test_playbook.py`	Modified	Root order
`test/orchestrator/fvt/pxeboot/additional_cloud_init/` (1 file)	Modified	Contract and order block
`test/orchestrator/fvt/pxeboot/cloudinit/` (1 file)	Modified	Order block
`test/orchestrator/fvt/pxeboot/connectivity/` (1 file)	Modified	Contract and order block
`test/orchestrator/fvt/pxeboot/coredns_coredhcp/` (3 files)	Modified	Contract and order block
`test/orchestrator/fvt/pxeboot/kubernetes_cluster/` (3 files)	Modified	Contract and order block
`test/orchestrator/fvt/pxeboot/kubernetes_etcd/` (2 files)	Modified	Contract, order block, local etcd media case
`test/orchestrator/fvt/pxeboot/kubernetes_recovery/` (1 file)	Modified	Contract, `reboot` marker
`test/orchestrator/fvt/pxeboot/kubernetes_storage/` (1 file)	Modified	Contract and order block
`test/orchestrator/fvt/pxeboot/minimal_os/` (1 file)	Modified	Contract and order block
`test/orchestrator/fvt/pxeboot/mount_config/` (1 file)	Modified	Contract and order block
`test/orchestrator/fvt/pxeboot/powervault/conftest.py`	Deleted	Markers registered by the root conftest
`test/orchestrator/fvt/pxeboot/powervault/` (4 test files)	Modified	Contract and order block
`test/orchestrator/fvt/pxeboot/slurm_apptainer/` (6 files)	Modified	Contract, order block, `reboot` marker
`test/orchestrator/fvt/pxeboot/slurm_cluster/` (3 files)	Modified	Contract and order block
`test/orchestrator/fvt/pxeboot/slurm_dcgm/` (2 files)	Modified	Contract and order block
`test/orchestrator/fvt/pxeboot/slurm_gpu/` (1 file)	Modified	Contract and order block
`test/orchestrator/fvt/pxeboot/slurm_hpc_benchmarks/` (5 files)	Modified	Contract, order block, conftest marker registration removed
`test/orchestrator/fvt/pxeboot/slurm_infiniband/` (1 file)	Modified	Contract and order block
`test/orchestrator/fvt/pxeboot/slurm_jobs/` (1 file)	Modified	Contract; drain case keeps `scheduler_state`, drops `disruptive`
`test/orchestrator/fvt/pxeboot/slurm_ldap/test_external_ldap.py`	Added	External LDAP proxy and backend cases
`test/orchestrator/fvt/pxeboot/slurm_ldap/` (2 other files)	Modified	Contract, order block, LDAP identity gating
`test/orchestrator/fvt/pxeboot/slurm_lifecycle/` (1 file)	Modified	Contract, `node_lifecycle` marker
`test/orchestrator/fvt/pxeboot/slurm_openmpi/` (1 file)	Modified	Contract and order block
`test/orchestrator/fvt/pxeboot/slurm_recovery/` (1 file)	Modified	Contract, `reboot` marker
`test/orchestrator/fvt/pxeboot/slurm_ucx/` (1 file)	Modified	Contract and order block
`test/orchestrator/fvt/pxeboot/vast_storage/` (5 files)	Modified	Contract and order block
`test/orchestrator/library/functions/__init__.py`	Modified	Export external LDAP checks
`test/orchestrator/library/functions/external_ldap_pxeboot_func.py`	Added	External LDAP proxy and backend checks
`test/orchestrator/library/functions/openldap_prepare_func.py`	Modified	Remove moved LDAP checks; public `openldap_enabled`
`test/orchestrator/library/vars/domain_vars.py`	Modified	Marker list, suite order, no implicit exclusions
`test/orchestrator/ut/test_prepare_contracts.py`	Modified	External LDAP contracts against the pxeboot module

Testing

python3 -m pytest --confcutdir=ut ut -q (from test/orchestrator): 143 passed.
python3 -m pytest fvt/pxeboot nft -m 'not deploy' --collect-only -q with each marker choice:
- no marker: 244 of 253 collected; the 8 reboot and node remove/add tests and the deploy case are deselected
- --marker reboot: 6 collected
- --marker node_lifecycle: 2 collected
- --marker reboot,node_lifecycle: 8 collected
Not run: FVT against a cluster for this branch.

Backward Compatibility

Reboot and node remove/add tests no longer run without --marker reboot or --marker node_lifecycle. Runs that selected --marker disruptive must switch to those markers.
A pxeboot verify run without a marker now runs every other test, including functional, negative and image-download cases, instead of only sanity.
External LDAP cases move from ORCH_FVT_PREPARE_V008/V012 to ORCH_FVT_PXEBOOT_V524/V525.
No changes to playbooks, inputs or other domains.

Suggested Reviewers

Reviewer	Review Focus
`@sujit-jadhav`	Orchestrator pxeboot suites and LDAP move
`@abhishek-sa1`	Marker gating and runner behavior
`@snarthan`	Kubernetes etcd and reboot recovery coverage

Prepare the shared layer that the lifecycle suites build on. - omnia_auto lists skip reasons under the summary table, drops the expected/unexpected/framework skip classification, and the wheel is rebuilt. add_session_result takes an optional reason, so other domains are unchanged. - The test case registry gains the OIM negative cases, the local etcd media case, renumbered duplicate IDs, and per-suite order blocks. - The verification library adds local etcd media, control-plane reboot recovery, OIM negative inputs, and LDAP identity gating. Node removal keeps the heredoc delimiter off the PXE mapping and needs at least two Slurm compute nodes. Recovery helpers check the reboot authorization. Host-key cleanup runs through run_on_host, and progress and cleanup messages use omnia_auto log. Unused imports are removed. - Unit tests cover the etcd contracts, heredoc writes, the two-node removal minimum, host-key cleanup and progress logging. - The test automation standard and test_config documentation describe the new rules and settings. Refs dell#4849 Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com>

Each test now resolves its registry entry, creates its own TestLogger, and passes both to verify_precheck, verify_precheck_rejection, verify_cleanup or verify_nft. The result helpers only render the final record. Tests move into their suite order blocks (precheck 10000, prepare 20000, provision 30000, cleanup 50000, NFT 60000). Precheck adds the OIM negative cases: CPU, memory and disk thresholds, a missing admin NIC, PXE/public NIC overlap, air-gapped internet, an unreachable SSH target and an OS mismatch. They use verify_precheck_rejection, which passes only when the precheck rejects the input with an error. The provision boot image checks carry a boot_image marker so they run by default. verify_pxeboot keeps its (host, key, checker) form for now so the pxeboot suites are unchanged; the next change moves them to the same contract. The FVT README lists the new order blocks and test inventory. Refs dell#4849 Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com>

The pxeboot suites now create their own TestLogger, resolve their registry entry, and call verify_pxeboot(test_log, case, host, checker). Each suite moves into its order block (40100 to 42400) so cases run in dependency order, with reboot and node removal suites last. Negative cases leave the sanity marker set. Reboot and node remove/add tests carry the reboot and node_lifecycle markers and are deselected unless --marker selects them, even when the suite is named. Without a marker every other test runs; the runner no longer forces a sanity default. Register the benchmark, boot_image, powervault_* and node_lifecycle markers and drop the disruptive marker. Restore the Kubernetes local etcd media check and control-plane reboot recovery. Move the external LDAP proxy and backend checks from prepare to the slurm_ldap suite ahead of the LDAP login tests, and skip LDAP identity checks when validate_external_ldap is false. Update the README and run configuration reference for the marker rules. Refs dell#4849 Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com>

balajikumaran-c-s added 3 commits October 10, 2026 00:59

balajikumaran-c-s requested review from RvishankarOMnia, VenkateswaraVatam, abhishek-sa1, snarthan and sujit-jadhav as code owners October 9, 2026 19:33

This was referenced Oct 9, 2026

test(orchestrator): update verification library and unit tests #5495

Open

test(orchestrator): adopt per-test logger contract outside pxeboot #5496

Open

abhishek-sa1 approved these changes Oct 10, 2026

View reviewed changes

Navigation Menu

Repository navigation

test(orchestrator): adopt per-test logger contract in pxeboot suites - #5497

test(orchestrator): adopt per-test logger contract in pxeboot suites#5497
balajikumaran-c-s wants to merge 3 commits into
dell:pub/rhel10.2from
balajikumaran-c-s:orch-test-split-3

balajikumaran-c-s commented Oct 9, 2026 •

edited

Loading

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

balajikumaran-c-s commented Oct 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Title

Description of the Solution

Related Issue

Changes

pxeboot suites (test/orchestrator/fvt/pxeboot)

Opt-in marker gating (test/orchestrator/conftest.py, _run.py, library/vars/domain_vars.py)

External LDAP move (library/functions, fvt/prepare/openldap, fvt/pxeboot/slurm_ldap)

Result contract (test/orchestrator/fvt/result.py)

Documentation (test/orchestrator/README.md, test/orchestrator/docs/test_run_config.md)

Files Changed

Testing

Backward Compatibility

Suggested Reviewers

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

balajikumaran-c-s commented Oct 9, 2026 •

edited

Loading

pxeboot suites (`test/orchestrator/fvt/pxeboot`)

Opt-in marker gating (`test/orchestrator/conftest.py`, `_run.py`, `library/vars/domain_vars.py`)

External LDAP move (`library/functions`, `fvt/prepare/openldap`, `fvt/pxeboot/slurm_ldap`)

Result contract (`test/orchestrator/fvt/result.py`)

Documentation (`test/orchestrator/README.md`, `test/orchestrator/docs/test_run_config.md`)