Repairer shutdown races with background processing under race tests

Cloud delivery baseline validation at commit 4c7c685 found an existing data race in internal/execution. The delivery change only touches scripts/docs; runtime Go sources are unchanged.

Evidence level: Test. Full make test-short on Linux Go 1.26.4 runs go test -short -race -count=1 -shuffle=on -timeout 5m ./....

Repairer.Shutdown invokes drain/processDue while the goroutine started by Repairer.Start can still be in tryProcess. The race report points to repairer.go lines 232, 244, 269 and 278. TestRepairer_MergeTerminalPreferred reports the race; other concurrent tests are also marked failed by the race detector.

Excerpt:

WARNING: DATA RACE
Read at 0x00c00013c268 by goroutine 59:
  github.com/hrygo/hotplex/internal/execution.(*Repairer).processDue()
      /workspace/hotplex/internal/execution/repairer.go:232 +0x5c4
  github.com/hrygo/hotplex/internal/execution.(*Repairer).drain()
      /workspace/hotplex/internal/execution/repairer.go:343 +0xbc
  github.com/hrygo/hotplex/internal/execution.(*Repairer).Shutdown()
      /workspace/hotplex/internal/execution/repairer.go:329 +0x189
  github.com/hrygo/hotplex/internal/execution.TestRepairer_MergeTerminalPreferred()
      /workspace/hotplex/internal/execution/repairer_test.go:263 +0x8f7
  testing.tRunner()
      /workspace/cloud-setup/go/src/testing/testing.go:2036 +0x21c
  testing.(*T).Run.gowrap1()
      /workspace/cloud-setup/go/src/testing/testing.go:2101 +0x38

Previous write at 0x00c00013c268 by goroutine 6466:
  github.com/hrygo/hotplex/internal/execution.(*Repairer).tryProcess()
      /workspace/hotplex/internal/execution/repairer.go:278 +0x494
  github.com/hrygo/hotplex/internal/execution.(*Repairer).processDue()
      /workspace/hotplex/internal/execution/repairer.go:239 +0x725
  github.com/hrygo/hotplex/internal/execution.(*Repairer).loop()
      /workspace/hotplex/internal/execution/repairer.go:209 +0x16f
  github.com/hrygo/hotplex/internal/execution.TestRepairer_MergeTerminalPreferred.(*Repairer).Start.func1.gowrap1()
      /workspace/hotplex/internal/execution/repairer.go:110 +0x46

Goroutine 59 (running) created at:
  testing.(*T).Run()
      /workspace/cloud-setup/go/src/testing/testing.go:2101 +0xb12
  testing.runTests.func1()
      /workspace/cloud-setup/go/src/testing/testing.go:2585 +0x84
  testing.tRunner()
      /workspace/cloud-setup/go/src/testing/testing.go:2036 +0x21c
  testing.runTests()
      /workspace/cloud-setup/go/src/testing/testing.go:2583 +0x9e9
  testing.(*M).Run()
      /workspace/cloud-setup/go/src/testing/testing.go:2443 +0xf4b
  main.main()
      _testmain.go:254 +0x164

Goroutine 6466 (running) created at:
  github.com/hrygo/hotplex/internal/execution.TestRepairer_MergeTerminalPreferred.(*Repairer).Start.func1()
      /workspace/hotplex/internal/execution/repairer.go:110 +0xf9
  sync.(*Once).doSlow()
      /workspace/cloud-setup/go/src/sync/once.go:78 +0xa1
  testing.tRunner()
      /workspace/cloud-setup/go/src/testing/testing.go:2036 +0x21c
  testing.(*T).Run.gowrap1()
      /workspace/cloud-setup/go/src/testing/testing.go:2101 +0x38
==================
==================
WARNING: DATA RACE
Write at 0x00c00013c260 by goroutine 59:
  github.com/hrygo/hotplex/internal/execution.(*Repairer).tryProcess()
      /workspace/hotplex/internal/execution/repairer.go:244 +0x88
  github.com/hrygo/hotplex/internal/execution.(*Repairer).processDue()
      /workspace/hotplex/internal/execution/repairer.go:239 +0x725
  github.com/hrygo/hotplex/internal/execution.(*Repairer).drain()
      /workspace/hotplex/internal/execution/repairer.go:343 +0xbc
  github.com/hrygo/hotplex/internal/execution.(*Repairer).Shutdown()
      /workspace/hotplex/internal/execution/repairer.go:329 +0x189
  github.com/hrygo/hotplex/internal/execution.TestRepairer_MergeTerminalPreferred()
      /workspace/hotplex/internal/execution/repairer_test.go:263 +0x8f7
  testing.tRunner()
      /workspace/cloud-setup/go/src/testing/testing.go:2036 +0x21c
  testing.(*T).Run.gowrap1()
      /workspace/cloud-setup/go/src/testing/testing.go:2101 +0x38

Previous read at 0x00c00013c260 by goroutine 6466:
  github.com/hrygo/hotplex/internal/execution.(*Repairer).tryProcess()
      /workspace/hotplex/internal/execution/repairer.go:269 +0x23c
  github.com/hrygo/hotplex/internal/execution.(*Repairer).processDue()
      /workspace/hotplex/internal/execution/repairer.go:239 +0x725
  github.com/hrygo/hotplex/internal/execution.(*Repairer).loop()
      /workspace/hotplex/internal/execution/repairer.go:209 +0x16f
  github.com/hrygo/hotplex/internal/execution.TestRepairer_MergeTerminalPreferred.(*Repairer).Start.func1.gowrap1()
      /workspace/hotplex/internal/execution/repairer.go:110 +0x46

Goroutine 59 (running) created at:
  testing.(*T).Run()
      /workspace/cloud-setup/go/src/testing/testing.go:2101 +0xb12
  testing

Acceptance: add a deterministic regression for concurrent shutdown/processing, fix the shutdown ownership or synchronization, and pass targeted execution race tests plus the unchanged full quality gates. Do not disable race detection or bypass Git hooks.

Refs #1033. This blocks the first cloud delivery validation.

PR #1035 is merged on October 9, 2026.

Verified PR HEAD: 839f6bf3ca829227338347efd5a68fefcf517816.
CI run 37876271883 and both PR Checks runs completed successfully for this exact HEAD. All checks passed, including race/coverage tests, PostgreSQL, WebChat browser contracts, Coverage Check and Validation Gate.
The main branch still pointed to the already integrated lifecycle merge 34240394301fb400af3f73c7cc9ec9359284f3e0 at the final check.
With the user's explicit authorization to use administrator merge after green CI, PR Fix Repairer shutdown races and add cloud delivery pipeline #1035 was squash-merged as 4119b288f4ebd8539c8c1255fce32dc8e39f6132.
No production deployment or data migration was performed. Existing uncommitted changes in the primary checkout were preserved.

Local review and validation details are recorded in the PR review and updated PR description.

Navigation Menu

Repository navigation

Repairer shutdown races with background processing under race tests #1034

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Repairer shutdown races with background processing under race tests #1034

Description

Activity

hrygo commented on Oct 9, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions