test(suites): cut the race run's critical path without dropping a case - #439
Merged
Merged
Conversation
Contributor
Author
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
Co-Authored-By: jason.han <hanhuijun@gmail.com>
devin-ai-integration
Bot
force-pushed
the
test/parallel-slow-suites
branch
from
September 19, 2026 18:35
6938c01 to
82b5d33
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
make testspends its wall time on a handful of packages whose tests run serially inside the package, so the race run is bounded by the longest package rather than by the core count. Measured on an 8-core box with every corpus present: 10m45s, with the runtime (506s), smt (359s) and workspace model (351s) packages on the critical path.Two kinds of change, no test or assertion removed:
TestRuntimeRobustness/nested_flow_that_never_ends(91s) andunguarded_loop_through_a_merge(53s) ininternal/exec/runtimeran the default 1,000,000 action steps under the race detector only to reachErrActionStepLimitExceeded. They now setctx.maxActionSteps = 1000, as other robustness cases already do; the code path and the typed error are the same.TestRefereeCorpus(internal/exec/smt): the corpus cases run under acasesgroup witht.Parallel(). Each case gets a value copy of the solver (asolve.Solveris configuration; every check starts its own process) and its ownrefereeTally, merged into the shared tally under a mutex; the group boundary keepstally.logand the "encoded no case" check after every case has finished. Subtest names gain acases/segment; nothing in the tree refers to the old names.TestIncrementalEqualsFresh/TestIncrementalEqualsFreshCorpora(tests/model): each replay owns its workspace and RNG.TestCompiledCalcsAgreeWithInterpreter(internal/frontend/repl): the Go and C targets build into separateTempDirs from separate sessions.TestExprTypeCheckNoStdlibFalsePositives/TestExprTypeCheckPublishedStdlibDefects(internal/workspace/model): each opens its own workspace over a read-only library source.Per-package race times before → after (measured before the package reorganization; the packages are unchanged in content): runtime 506s → 194s, smt 359s → 200s, model 351s → 235s, repl 232s → 144s, tests/model 220s → 134s.
TestRuntimeRobustness179s → 19s,TestRefereeCorpus328s → 167s.How it was verified
go test -race -pgo=off -count=1 ./internal/exec/runtime ./internal/exec/smt ./internal/workspace/model ./internal/frontend/repl ./tests/modelon top of currentdevelop, with the corpora andOPENSYSML_REQUIRE_SMT=1: passes, zero race reports.-race -count=2ontests/model -run TestIncrementalEqualsFresh$and the model package's-run TestExprTypeCheck.gofmt -l .empty,go vetclean on the touched packages.Checklist
make testandmake lintpass locally (touched packages; full run in CI)Link to Devin session: https://nasa-jpl-demo.devinenterprise.com/sessions/9c85d2f2974848dbb25a944ea7212e1e
Open in Devin Desktop: https://nasa-jpl-demo.devinenterprise.com/desktop/session/9c85d2f2974848dbb25a944ea7212e1e?variant=devin
Requested by: @HuiJun