Repository navigation
test(orchestrator): restore etcd and reboot coverage, fix LDAP gating and test ordering - #5478
Draft
balajikumaran-c-s wants to merge 14 commits into
Draft
balajikumaran-c-s wants to merge 14 commits into
balajikumaran-c-s wants to merge 14 commits into
Conversation
… and ordering Restore the Kubernetes local-disk etcd and control-plane reboot coverage that existed in the 2.2 suite (Omnia_Spec dell#953): - local etcd checks require the boot script DONE marker from the current boot, a GPT etcd disk, and the configured disk media - control-plane reboot recovery waits for etcd health and confirms the fstab update ran while disk setup did not rerun and the UUID is kept Move the external LDAP proxy and backend checks from prepare to the pxeboot slurm_ldap suite, ahead of the LDAP login tests, and skip every LDAP identity check when validate_external_ldap is false. Give each suite its own 100-number order block so orders are unique and run in dependency order. Tests create their own TestLogger, resolve the registry entry, and carry function-level feature markers. Keep negative tests out of the sanity marker set; sanity holds only the baseline positive checks. Fix the Slurm drain-state command template, whose unescaped sinfo %T broke string formatting, and its unit test. Remove the unexpected-skip classification from the omnia_auto report and rebuild the wheel. Update the test automation standard with tag folder, test_playbook.py, naming, credential scope, README, UT, and omnia_auto rules. Drop the UT that only checked layout, and fix duplicate UT IDs. Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com>
The provision boot image checks were marked functional, which the orchestrator conftest skips when no marker is selected. The skip happened before the test body, so no reason was printed. Give them a dedicated boot_image marker so they run by default as before. List every skip reason under the omnia_auto summary table, and log skips raised before a test starts with its ID, title, and reason. Parse the pytest skip reason without its tuple wrapper. Rebuild the omnia_auto wheel; the new add_session_result reason argument is optional, so other domains are unchanged. Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com>
A run without --marker now executes all collected tests. Remove the pxeboot sanity default from the runner entry point, drop the sanity default and the disruptive/negative exclusions from untagged lifecycle runs, and stop the conftest auto-skip of functional and disruptive tests. With no marker, the runtime authorization covers every gate a test carries. Lifecycle execution cases still run only in the exec phase, and marker expressions keep narrowing a run as before. Document the behavior in the README, the run configuration reference, and the test automation standard. Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com>
Resolve conflicts with the HPC benchmark and CoreDNS/CoreDHCP test updates (dell#5479). Keep the per-suite order blocks and the test-created TestLogger style, and add the seven new upstream tests in upstream's sequence inside their suite blocks. Keep upstream's markers, docstrings, and registry IDs. Regenerate the FVT README tables and update the pxeboot ID ranges. Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com>
Signed-off-by: Abhishek S A <abhishek.sa3@dell.com>
Signed-off-by: balajikumaran.cs <balajikumaran.cs@dell.com>
balajikumaran-c-s
changed the title
Resolve conflicts with the HPC benchmark prerequisite, idempotency, and concurrency tests (dell#5491). Keep the per-suite order blocks, function-level markers, and the test-created TestLogger style. Place the five new tests in upstream's sequence inside the slurm_hpc_benchmarks block and drop the removed per-tool staging report test. Upstream registered the three new benchmark tests as V095-V097, which already belong to two additional_cloud_init tests and node_architecture. Keep the first registrants and move the benchmark tests to V225-V227. Record V207 as retired and the three reassignments in the FVT README. Register the benchmark marker in MARKERS and the root conftest so the runner accepts it, and regenerate the slurm_hpc_benchmarks README table, the pxeboot ID ranges, and the marker list in test_run_config.md. Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com>
… design Convert the minimal_os and vast_storage tests to the verify_pxeboot(test_log, case, host, checker) contract, import TestLogger, and move them into the lifecycle order-block scheme (minimal_os 40501-40515, vast_storage 42101-42117 with recovery and lifecycle suites shifted after it). Register the vast_* markers, remove duplicate coredns_coredhcp/powervault SUITES entries, and document the new tests, blocks, and markers. Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com>
…elected An untagged run, or a run that only names a suite, rebooted cluster nodes because every collected test ran without a marker. Reboot tests now carry only the reboot marker and are skipped unless --marker selects reboot. The redundant disruptive marker is removed; scheduler-drain and node remove/add tests run by default. The shared standard and orchestrator docs are updated to match. Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com>
…marked Recovery (reboot) and Slurm node remove/add tests disrupt the cluster and should not appear in a normal run. They are now deselected unless --marker selects reboot or the new node_lifecycle marker, even with --suite. Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com>
When the mapping file had no trailing newline, the node remove/add restore appended the OMNIA_EOF delimiter to the last row, corrupting the mapping and failing later provisions. Always end the content with a newline and drop the quote escaping, which a quoted heredoc writes literally. Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com>
… nodes Removing the only Slurm compute node leaves the cluster with no compute capacity. Node removal now skips unless the PXE mapping has at least two slurm_node entries, and a skipped removal discards any stale backup so the re-addition test cannot restore an old mapping. Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com>
The upstream merge left imports that no module uses. Remove them and correct two README statements: markers other than reboot and node_lifecycle only narrow test selection, and scheduler_state needs no opt-in. Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com>
…h omnia_auto Stale SSH host-key removal called subprocess directly, and polling and cleanup helpers printed to stdout. Run ssh-keygen through run_on_host with a PXEBOOT_COMMANDS entry, and send progress and cleanup warnings through omnia_auto log. Add ORCH_UT_093 and ORCH_UT_094 to cover both paths. Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR Description
Description of the Solution
Summary: Restores Kubernetes local-disk etcd and control-plane reboot coverage that existed in the 2.2 suite, moves external LDAP checks to the appropriate pxeboot suite, reorganizes test suite ordering for dependency correctness, and fixes multiple test framework issues including Slurm drain-state command template, omnia_auto report classification, and test automation standards.
Changes
Kubernetes Etcd and Reboot Coverage
LDAP Test Reorganization
Test Suite Ordering
Test Framework Improvements
Bug Fixes
Test Automation Standards
Files Changed
docs/code-style/test_automation.mdtest/orchestrator/README.mdtest/orchestrator/_run.pytest/orchestrator/conftest.pytest/orchestrator/docs/test_config.mdtest/orchestrator/docs/test_run_config.mdtest/orchestrator/fvt/README.mdtest/orchestrator/fvt/cleanup/*/test_*.pytest/orchestrator/fvt/precheck/*/test_*.pytest/orchestrator/fvt/prepare/*/test_*.pytest/orchestrator/fvt/provision/*/test_*.pytest/orchestrator/fvt/pxeboot/*/test_*.pytest/orchestrator/fvt/pxeboot/kubernetes_etcd/test_local_etcd.pytest/orchestrator/fvt/pxeboot/kubernetes_recovery/test_reboot.pytest/orchestrator/fvt/pxeboot/slurm_ldap/test_external_ldap.pytest/orchestrator/fvt/pxeboot/slurm_ldap/test_authentication.pytest/orchestrator/fvt/result.pytest/orchestrator/library/functions/__init__.pytest/orchestrator/library/functions/_workload_helpers.pytest/orchestrator/library/functions/apptainer_*.pytest/orchestrator/library/functions/external_ldap_pxeboot_func.pytest/orchestrator/library/functions/kubernetes_recovery_pxeboot_func.pytest/orchestrator/library/functions/kubernetes_runtime_pxeboot_func.pytest/orchestrator/library/functions/oim_readiness_precheck_func.pytest/orchestrator/library/functions/openldap_prepare_func.pytest/orchestrator/library/functions/slurm_auth_pxeboot_func.pytest/orchestrator/library/functions/slurm_recovery_pxeboot_func.pytest/orchestrator/library/messages/precheck_msgs.pytest/orchestrator/library/vars/__init__.pytest/orchestrator/library/vars/domain_vars.pytest/orchestrator/library/vars/precheck_vars.pytest/orchestrator/library/vars/pxeboot_vars.pytest/orchestrator/library/vars/test_case_vars.pytest/orchestrator/nft/README.mdtest/orchestrator/nft/result.pytest/orchestrator/nft/test_*.pytest/orchestrator/test_config.ymlTesting
Backward Compatibility
Suggested Reviewers
@orchestrator-test-team