Repository navigation
spank_qrmi: set QRMI_QPU for srun too - #212
Merged
Merged
Conversation
srun runs SPANK plugins in S_CTX_LOCAL, so a job submitted with 'srun --qpu=...' reached slurmctld without QRMI_QPU in its spank_job_env. spank_job_control_setenv() accepts the local context (spank(8) documents it as local only), and srun's allocation request carries spank_job_env like sbatch and salloc do.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of Change
Follow-up to #211.
QRMI_QPUis only set inS_CTX_ALLOCATOR(sbatch, salloc).srunruns SPANK plugins inS_CTX_LOCAL, so a job submitted withsrun --qpu=...reaches slurmctld withoutQRMI_QPU, and a job_submit plugin cannot see its QPU. This sets it in both contexts.Evidence (Slurm 25.05.3)
spank_init(NULL)(srun_job.c#L689), which usesS_TYPE_LOCAL(spank.c#L834-L840), reported to plugins asS_CTX_LOCAL(spank.c#L1911-L1912).spank_job_control_setenv()only rejects remote and slurmd contexts (spank.c#L2370-L2382). The spank(8) docs list thespank_job_control_*functions as available "from local context only".slurm_opt_create_job_desc()(allocate.c#L705), which copiesspank_job_envinto the job description sent to slurmctld (slurm_opt.c#L5795-L5802), as for sbatch and salloc.Testing
On a Slurm 25.05.3 cluster, slurmctld at
debug3(which logsspank_job_envper submission), each command run with--qpu=PASQAL_LOCAL:srunspank_job_env="_SLURM_SPANK_OPTION_spank_qrmi_qpu=PASQAL_LOCAL"..., QRMI_QPU=PASQAL_LOCALsalloc..., QRMI_QPU=PASQAL_LOCAL..., QRMI_QPU=PASQAL_LOCALsbatch..., QRMI_QPU=PASQAL_LOCAL..., QRMI_QPU=PASQAL_LOCALIn
sruninside an existing allocation the call only updates the step'sspank_job_env; the job runs as before.Note for job_submit.lua
Lua sees
job_desc.spank_job_envas a table with 0-based keys (job_submit_lua.c#L723-L737), so theipairsloop in the #211 description skips the first entry. Usepairs: