Repository navigation
Run each task at most once at a time (unique WorkManager work per task) - #423
Open
whatgoodarewords wants to merge 1 commit into
Open
whatgoodarewords wants to merge 1 commit into
whatgoodarewords wants to merge 1 commit into
Conversation
SyncManager.queue() used plain WorkManager.enqueue(), so a trigger firing, Start Task, boot catch-up or WorkManager resuming interrupted work could start a second SyncWorker for a task that was still running. Enqueue task runs as unique work named task-<id> with KEEP, so extra requests are dropped while a run is pending or running. Follow-ups use APPEND_OR_REPLACE so a task that follows up on itself runs again after the current run instead of being dropped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SyncManager.queue(taskID)calls plainWorkManager.enqueue(), so every request for a task adds anotherSyncWorkerfor it, even when one is already running. Requests come from several places:TriggerService.queueTrigger)BootRecieverre-queues triggers)WorkManager also resumes interrupted work after a reboot, so a reboot during a long sync gives you the resumed run plus the boot catch-up run.
I hit this with a 6-hour trigger on a first full copy of about 26 GB over a slow uplink. The copy takes longer than six hours, so the trigger started a second
rclone copyof the same task against the same remote while the first one was still going. Two identical copies then compete for the same bandwidth and write the same files.Change
queue(taskID)now usesenqueueUniqueWork("task-<id>", ExistingWorkPolicy.KEEP, request). If that task already has a run queued or running, the new request is dropped. Once the run finishes, fails or is cancelled, the next request starts a new run as before.queueFollowup(taskID)withExistingWorkPolicy.APPEND_OR_REPLACE. A worker queues its follow-up while it is still running, so withKEEPa task that names itself as its own follow-up (selectable in the task editor, e.g. "on fail, run this task again") would be dropped. Appending runs it once after the current run ends, which keeps the current behaviour for that setup.queueEphemeralandEphemeralTaskManagerare unchanged. They use random ids, so they never collide.Different tasks still run in parallel. This PR does not limit concurrency across tasks (#402).
Cancelling still works
SyncManager.cancel(tag)/cancelAllWorkByTagcancels it as before.createCancelPendingIntent(workId). The worker's id is the same UUID whether or not the work is unique, so this path does not change.Testing
./gradlew :app:compileOssDebugKotlin :app:testOssDebugUnitTest -x :rclone:buildAllpasses (existing 14 unit tests).work-testingtest. It is not in this PR because it adds four test dependencies (robolectric, work-testing, androidx.test:core, concurrent-futures); I can add it as a second commit if you want it. The test uses a worker that never finishes, standing in for a long copy:queue(42)three times:masterleaves 3 RUNNING workers for the task (the test fails there), this branch leaves 1.queue(1)andqueue(2): both tasks run.queue(7)thenqueueFollowup(7): one RUNNING, one BLOCKED behind it.cancel("9")cancels the run, and a laterqueue(9)starts a new one.cancelWorkById(what the notification's Cancel action calls) cancels the run.🤖 Generated with Claude Code