Skip to content

test(suites): cut the race run's critical path without dropping a case - #439

Merged
HuiJun merged 1 commit into
developfrom
test/parallel-slow-suites
Sep 19, 2026
Merged

HuiJun merged 1 commit into
developfrom
test/parallel-slow-suites

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

What and why

make test spends its wall time on a handful of packages whose tests run serially inside the package, so the race run is bounded by the longest package rather than by the core count. Measured on an 8-core box with every corpus present: 10m45s, with the runtime (506s), smt (359s) and workspace model (351s) packages on the critical path.

Two kinds of change, no test or assertion removed:

  • Budget-exhaustion cases spin less. TestRuntimeRobustness/nested_flow_that_never_ends (91s) and unguarded_loop_through_a_merge (53s) in internal/exec/runtime ran the default 1,000,000 action steps under the race detector only to reach ErrActionStepLimitExceeded. They now set ctx.maxActionSteps = 1000, as other robustness cases already do; the code path and the typed error are the same.
  • Independent subtests run in parallel.
    • TestRefereeCorpus (internal/exec/smt): the corpus cases run under a cases group with t.Parallel(). Each case gets a value copy of the solver (a solve.Solver is configuration; every check starts its own process) and its own refereeTally, merged into the shared tally under a mutex; the group boundary keeps tally.log and the "encoded no case" check after every case has finished. Subtest names gain a cases/ segment; nothing in the tree refers to the old names.
    • TestIncrementalEqualsFresh / TestIncrementalEqualsFreshCorpora (tests/model): each replay owns its workspace and RNG.
    • TestCompiledCalcsAgreeWithInterpreter (internal/frontend/repl): the Go and C targets build into separate TempDirs from separate sessions.
    • TestExprTypeCheckNoStdlibFalsePositives / TestExprTypeCheckPublishedStdlibDefects (internal/workspace/model): each opens its own workspace over a read-only library source.

Per-package race times before → after (measured before the package reorganization; the packages are unchanged in content): runtime 506s → 194s, smt 359s → 200s, model 351s → 235s, repl 232s → 144s, tests/model 220s → 134s. TestRuntimeRobustness 179s → 19s, TestRefereeCorpus 328s → 167s.

How it was verified

  • go test -race -pgo=off -count=1 ./internal/exec/runtime ./internal/exec/smt ./internal/workspace/model ./internal/frontend/repl ./tests/model on top of current develop, with the corpora and OPENSYSML_REQUIRE_SMT=1: passes, zero race reports.
  • -race -count=2 on tests/model -run TestIncrementalEqualsFresh$ and the model package's -run TestExprTypeCheck.
  • gofmt -l . empty, go vet clean on the touched packages.

Checklist

  • make test and make lint pass locally (touched packages; full run in CI)
  • Tests added or updated for the change
  • Documentation extended where it already covers the surface (not applicable)
  • Changelog entry added (test-only change)
  • baselines regenerated (no gate count moved)
  • No internal work-item labels in the body, docs, or changelog

Link to Devin session: https://nasa-jpl-demo.devinenterprise.com/sessions/9c85d2f2974848dbb25a944ea7212e1e
Open in Devin Desktop: https://nasa-jpl-demo.devinenterprise.com/desktop/session/9c85d2f2974848dbb25a944ea7212e1e?variant=devin
Requested by: @HuiJun

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@devin-ai-integration
devin-ai-integration Bot marked this pull request as ready for review September 19, 2026 02:59

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

Co-Authored-By: jason.han <hanhuijun@gmail.com>
@devin-ai-integration
devin-ai-integration Bot force-pushed the test/parallel-slow-suites branch from 6938c01 to 82b5d33 Compare September 19, 2026 18:35
@HuiJun
HuiJun merged commit bffbc94 into develop Sep 19, 2026
12 checks passed
@HuiJun
HuiJun deleted the test/parallel-slow-suites branch September 19, 2026 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant