Repository navigation
Conversation
Since commit 84eacf1 ("io_uring/io-wq: inherit cpuset of cgroup in io worker"), io_wq_create() copies the cpuset of the task that creates the io-wq into wq->cpu_mask, and io_init_new_worker() sets the affinity of every new worker to that copy. The copy is never refreshed, but workers are started for as long as the io-wq exists: whenever a request has to block and no worker is free, while idle ones exit after WORKER_IDLE_TIMEOUT. When the cpuset of the cgroup changes, or the task moves to another cgroup, cpuset updates the affinity of the workers that exist. Workers started later get the stale mask, which set_cpus_allowed_ptr() does not check against their cpuset, and run on CPUs their cgroup no longer has. A worker that cpuset moved is also retired at its next idle timeout, even when it is the last one, as it now runs outside wq->cpu_mask, and its replacement starts on the old CPUs. With a task in a cgroup whose cpuset.cpus goes from 2-3 to 4-5 while one request is blocked in a worker, and three more blocking requests issued afterwards: worker started before the change Cpus_allowed_list: 4-5 3 workers started after the change Cpus_allowed_list: 2-3 So io-wq workers keep running on CPUs that were taken away from their cgroup, for instance to give them to latency sensitive tasks. Keep wq->cpu_mask for the affinity of the io-wq itself, all CPUs or the mask set with IORING_REGISTER_IOWQ_AFF, and intersect it with the cpuset of the worker when the worker starts, using the cpuset alone if the two have no CPU in common. That is how the scheduler treats the affinity a task requested when its cpuset changes, see __set_cpus_allowed_ptr(). Use the same mask in the idle timeout test of a worker, and when the registered affinity is removed, go back to all CPUs rather than to the cpuset of that moment. A mask being registered must still lie within the cpuset, as required since commit 0997aa5 ("io_uring/io-wq: do not allow pinning outside of cpuset"). Fixes: 84eacf1 ("io_uring/io-wq: inherit cpuset of cgroup in io worker") Cc: stable@vger.kernel.org Signed-off-by: Mikhail Malyshev <mike.malyshev@gmail.com>
rucoder
marked this pull request as draft
November 12, 2026 11:01
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
io_uring's io-wq workers started after a cgroup's cpuset changes, or after the task moves to another cgroup, run on the CPUs the cgroup had when the io-wq was created. Since 6.12 (
84eacf177faa"io_uring/io-wq: inherit cpuset of cgroup in io worker", in 6.18 too)io_wq_create()copies the cpuset once andio_init_new_worker()gives every new worker that copy. cpuset re-masks the workers that exist; workers started later, and the replacement of an idle worker that cpuset moved, land on the old CPUs.This is why lf-edge/eve#6335 gives
aio=io_uringonly to VMs whose non-vCPU threads never move. With this fix, QEMU's io_uring workers follow the cpuset like every other thread of the VM.Root cause / mechanism
wq->cpu_maskmixed two things: the io-wq's own affinity (all CPUs, orIORING_REGISTER_IOWQ_AFF) and a snapshot of the cpuset. The fix keeps only the former inwq->cpu_maskand intersects it with the worker's cpuset when each worker starts, falling back to the cpuset when the two don't intersect (the rule__set_cpus_allowed_ptr()applies to a task's requested affinity). The idle worker test uses the same mask, and unregistering the affinity goes back to all CPUs rather than to the cpuset of that moment.Cost: one
cpuset_cpus_allowed()per worker start and per idle timeout, not per request. Measured with the function graph tracer: about 0.3-0.4 us against 57-62 us for a whole worker start (create_io_worker).upstreaming
UPDATE: the finding was confirmed valid by maintainer
Validation
A QEMU/KVM test (8 CPUs, cgroup v2, lockdep and
CPUMASK_OFFSTACKon) that blocks requests in io-wq workers around cpuset changes and checks every worker's affinity:IORING_REGISTER_IOWQ_AFF2-3, cpuset 3-6 / 4-5 / unregisterSame results on 6.18.55, 6.18.63 and mainline
io_uring-7.4. New liburing testtest/iowq-cpuset.cfails without the fix and passes with it; the whole liburing suite gives the same results with and without (the only differences,wq-affandiowait, are flaky on both: 20 runs each). No lockdep or sleep-in-atomic reports.Upstream status
The same commit is prepared for Jens Axboe's
io_uring-7.4(Cc stable), with the liburing test. Not posted yet; this PR carries it until it lands in stable 6.18.y.Backport / other branches
Also affected:
eve-kernel-arm64-v6.17.13-nvidia-gb10,eve-kernel-arm64-v7.0.14-nvidia-gb,eve-kernel-amd64-next. Not affected:eve-kernel-arm64-v6.1.155-generic(6.1 predates the bug).🤖 Generated with Claude Code