Repository navigation
concurrency: process-wide mutable state is not safe under concurrent in-process use — two compilations in one process gave 5 vs 1 diagnostics; module-level random shares one System.Random; crash bundles check-then-create #2250
Description
Class
Process-wide mutable state is not safe under concurrent in-process use. Before P41 (#2179) every test project ran on one thread, which hid this. Now Compiler.Tests runs 2 threads (a24387b) and Stdlib.Tests runs 4 (8df7f32). Two compilations, two stdlib calls, or two crash reports in one process can therefore overlap. The contract: a compile is a pure function of its inputs; a stdlib module function that CPython makes safe to call from several threads is safe here too; and a per-event filesystem artefact is created exclusively. The seam is unmeasured; this issue records the cells.
Cells
| cell | program / site | observed @ sha | expected | lane | reach | found by |
|---|---|---|---|---|---|---|
| two in-process compilations at once | Fuzz.SemanticPropertyTests.Compilation_IsDeterministic(seed: 2025) iter 18: from snake_case import __dunder__ · def count() -> tuple: · if "self": for __dunder__ in None: pass · z(8), compiled twice in one process |
5 vs 1 diagnostics for the same input, measured once, @ af6f309 (whole Compiler.Tests at MaxParallelThreads=8; green at 1/2/4) |
identical diagnostics on every compile | silent-wrong | probe (the LSP compiles in-process too) | P41 P5.1 matrix |
module-level random from two threads |
RandomModuleModule._Random is one unsynchronised static System.Random (src/Sharpy.Stdlib/Random/Random.cs:18, generated from spy/random_module.spy); seed() swaps the instance |
code-read, not run, @ 116c6e2. System.Random is documented non-thread-safe; concurrent use can leave it returning 0 | CPython's module-level random functions are safe to call from several threads |
silent-wrong | latent | P41 Phase 2 audit |
| two crash bundles in the same millisecond | CrashBundleWriter.TryWrite: while (Directory.Exists(candidate)) suffix++ then CreateDirectory (src/Sharpy.Compiler/Diagnostics/CrashBundleWriter.cs:108–116) |
code-read, @ 0aecaa1: check-then-create, so both writers can pick one directory and interleave files | each crash gets its own directory (create-exclusive) | tooling | benign | P41 Phase 4.1 audit |
Siblings read, not affected: NumpyRandom is [ThreadStatic]. Plain statics calendar._moduleFirstweekday and socket._DefaultTimeout follow the CPython global model.
Acceptance
- The compile cell gets a reproduction first (a deterministic two-thread harness compiling the cell program while another compile runs), then the seam.
StaticStateConformanceTests(roadmap(A3): determinism harness — shuffled-order double-compile + static-state analyzer #1036) bans mutable statics in the compiler assembly, so the state is in an allowlisted cache, the module registry, or Stdlib. Find it, don't guess. - A concurrency matrix: {compile × compile, compile × stdlib module load, stdlib module fn × fn} × {1, 2, 8 threads} becomes a standing test.
- Every cell above is green or carries an owner ruling.
Labels: class-tracker, lane:silent-wrong, reach:probe. Found while implementing P41 (plan-5c21c8). Tracker for the parallelism change: #2179.
Activity
antonsynd commentedon Oct 5, 2026
Cells (P41 standalone verification, @ 8df7f32, code-read): more stdlib module-level process globals that tests now reach concurrently.
socket.setdefaulttimeoutwrites the public staticSocketModule._DefaultTimeout(src/Sharpy.Stdlib/Socket/SocketModule.cs:38,86).socket_module_testscalls it at 10 sites and does not run alone.calendar.setfirstweekdaywrites_moduleFirstweekday(CalendarFunctions.cs:62).random.seedis covered only by the mutual-exclusionPrngStatecollection (Stdlib.Tests/Spy/SpyTestCollections.cs).ProcessGlobalStateConformanceTestsdoes not check it.Today every reader of each global is in the same test class, so nothing collides. A reader added in another class would race while the guard stays green.
ProcessGlobalStateConformanceTests, test: restore measured parallelism — maxParallelThreads=1 and the blanket HeavyCompilation collection date from a 9.9k-test suite; 11.5k independent CompileAndExecute tests run serially (28 min) #2179._closed_loopback_port(Stdlib.Tests/Spy/socket/socket_module_tests.spy:627) binds port 0, closes it, then expects connection refused. AtmaxParallelThreads4, another test can take that ephemeral port in between. Low probability, not seen in P5.2's 4 runs.Lane silent-wrong, reach latent. Found by the P41 verification-expert.
antonsynd commentedon Oct 6, 2026
/verify-implementation of plan-5c21c8 (P41) — two more cells of this class (process-wide mutable state under concurrent in-process use; lane:silent-wrong, reach:probe; found by the class-cure auditor @ 5b98c8e), next to the L1 trio (
socket.setdefaulttimeout,calendar.setfirstweekday,random.seed):xml.register_namespaceXmlFunctions._registeredNamespaces, a non-thread-safeDictionarylogging.basicConfig_loggersBoth become live the moment two concurrently running test classes (Stdlib maxParallelThreads is 4 since 8df7f32) or two in-process programs touch the module.
antonsynd commentedon Oct 6, 2026
Batching 2026-10-05 @ 1e5d54c — charter
.claude/plans/remediation-round-2026-09-20-batching.md§3.49, P54 — process-wide state under concurrent in-process use (order N38, after P46; Compiler + Stdlib + tooling; generator FIRST). Ruling R-EX (owner, 2026-10-05): (a) own group with the concurrency harness FIRST.Contract as you wrote it: a compile is a pure function of its inputs under concurrent in-process use; a stdlib module function CPython makes safe to call from several threads is safe here; a per-event filesystem artefact is created exclusively. Lane / reach: silent-wrong · probe (the LSP compiles in-process concurrently, so the compile cell can reach an editor as nondeterministic diagnostics). First task: the harness — {compile × compile, compile × module load, stdlib fn × fn, crash × crash} × {1, 2, 8 threads} × {diagnostics identical, state not corrupted, one directory per event} with an allowlist ratchet; then the seams (
StaticStateConformanceTestsbans mutable statics in the compiler assembly, so the compile cell's state is in an allowlisted cache, the module registry or Stdlib — find it, don't guess). Cells: your eight (compile determinism at 8 threads;random,socket,calendar,xml,loggingmodule globals;CrashBundleWriter; the ephemeral-port test). Rejected: (b) split — the compile cell on P28's warm ≡ cold harness, the Stdlib globals one chore, the crash bundle one chore (P28's harness would carry two contracts: state across restores vs across threads); (c) chores without a harness (the compile seam is unknown, nothing to commit against). Ordering only, two-way. Class tracker: this issue.antonsynd commentedon Oct 12, 2026
New cells: warm state on ONE CompilerApi (order as well as concurrency), found by the P47 code-action sweep (#2232)
Same contract as this tracker: an analysis, and every semantic query a handler makes on it, is a function of the document alone. This adds an order cell. A warm instance used sequentially also answers differently from a cold one, so the cure is not a lock.
SharpyCodeActionHandleronfstrings/format_iformattable_class_1988(cr twin) selection [12:0-19:24], on a CompilerApi that has served the sweep's earlier documents. The extracted signature spells builtinsformat/print/strtaken as parametersSharpy.Lspprocess. Same forformat_no_dunder_format_refused_1988[8:0-16:23] (8,49) vs (8,59), anddeferred_overload_annotated_pin_1589[4:0-9:21] (5,148) vs (5,149). Two DOP 1 runs agree with each otherCompilerApi.Analyzeof the IFormattable fixtures' source gained/lost SPY0220 between two identical FULL runs at one sha. Coldsharpyc emit diagnosticsreports 0 SPY0220Not minimized to two programs. A plain
CompilerApi.Analyzeof every corpus source, sequential or DOP 4, with path-less orForLsp()options, never moved the target's identifier types or SPY0220: 3,445 of 3,445 fingerprints matched cold. So the trigger is in the sweep's OTHER work on the instance: the code-action handler's semantic queries and the analyses of applied (edited) texts. Candidate seam: the instance's cached master builtin/module registry (builtin overload-set types forformat/print, the IFormattable format-operand fact).MasterRegistryImmutabilityTestsdoes not cover the handler-query path.Workaround in the sweep, not a cure.
CodeActionShapeSweepTestsnow builds a fresh CompilerApi and driver per (stem, twin). That gave byte-identical pins and cells across three isolated FULL runs and at DOP 1 vs DOP 4, at a cost of 2× sweep time. The LSP server itself still holds one instance across documents.test(lsp): the code-action sweep gives every (stem, twin) its own com…
antonsynd commentedon Oct 14, 2026
Batching 2026-10-14 @ 27e2616 — charter
.claude/plans/remediation-round-2026-09-20-batching.md§3.6, P54 — process-wide state under concurrent in-process use (order 6; placement unchanged). The P47 sweep's ORDER cell (a warmCompilerApiused sequentially answers differently from cold; DOP 4 gains/loses SPY0220; a plainAnalyzeof every corpus source matches cold 3,445/3,445, so the trigger is the handler queries and applied-text analyses on the same instance) is recorded: the harness gains {cold, warm-sequential, warm-concurrent} × {analysis, handler query, applied-text analysis}; the sweep's fresh-instance workaround is not the cure.antonsynd commentedon Oct 14, 2026
Cell (P46 final verifier @ 739ae54; pre-existing, same on the pre-P46 binary c245ece): the compile server × a reference across requests.
sharpyc server, request 1 isrun main.spywith no-r(main importsGreeter). Request 2 is the same with-r Greeter.dll. Request 3 is again without-r.SPY0908 … CS0400, exit 2: the assembly loaded by request 2 satisfies request 3's import at the registry, and the Roslyn step then has no reference.antonsynd commentedon Oct 15, 2026
Batching 2026-10-14 @ 3f16f26 — charter
.claude/plans/remediation-round-2026-09-20-batching.md§3.6, P54 — process-wide state under concurrent in-process use (order 6; placement unchanged). Cell placed: the compile server × a reference across requests (run main.spy→-r Greeter.dll→ no-ron onesharpyc server: request 3 is SPY0908 CS0400 instead of SPY0300; P46 final verifier @ 739ae54, same @ c245ece) — the long-lived-process family; the harness gains a {server request sequence} row: {no-r,-r, no-r} must answer as three fresh processes do. Lane ice, reach program. Dependency recorded (R-HI on #2182): P42's door widening (every SPY0306 refuses at every door) waits for this cell's cure — the registry's SPY0306 "already loaded a same-named assembly from another path, then succeeded against the loaded copy" arm (ModuleRegistry.cs:47-50, read @ 3f16f26) is the same leak seen from the registry side, and P54 removes it rather than P42 filtering it.antonsynd commentedon Oct 15, 2026
P54 close-out: implementation evidence @ 7ae6ed7
Plan P54 (plan-9ef825, verified PASS WITH CORRECTIONS / CLASS / one group / generator first) landed on
devas 24 commits, 1a41b05..7ae6ed7, from base 486c9a8. Contract: a compile is a pure function of its inputs; no process-wide state crosses compiles, registries, threads or hosts.Close criterion, bullet by bullet (what ran @ sha)
in-process-isolation-allowlist.txtempty; sweep at R = 8 shows 0 driftInProcessIsolationSweepTests.Sweep_EightThreads_EightRepetitions(8 threads × R=8) andSweep_RegularSlicepass in the whole-solution gate @ 7ae6ed7RequestSequence_NoRef_Ref_NoRef…green (request 3 = SPY0300)TwoRegistries_SameNamed…greenWarmInstanceParityTestsgreen on the three P47 stemsStdlibModuleGlobalConcurrencyTestsgreen on six rowsCrashBundleExclusivityTestsgreenCallTimingCoverageTests.EveryExecuteArm_AttributesWithinFivePercentOfItsWallClock(wall-clock ratio 0.930 against the 0.95 threshold under full-suite load). Run alone @ 7ae6ed7 it passed 3/3; a whole Compiler.Tests re-run @ 7ae6ed7 gave 49253/0/3 (measured), timing test included. So every project is green @ 7ae6ed7: 80350/0/4 across the two runs. Earlier full gates: 80324/0/4 @ effba77, P1 80313/0/4 @ e3a987e; baseline 80185/0/4 @ 3f16f26format whitespace --verify-no-changesclean,check_spy_stalenessandcheck_spy_tests_stalenessOK, stdlib docs regenerate with no diff,check_allowlist_issue_stateOK, all @ 7ae6ed7devThe seams (what was cured)
ProcessWideResolutionScanTestspins this.AssemblyIdentity.FromAssemblyPathare deleted.defdeclares its own symbol. A type definition in a function body or a block is refused with SPY0202, so it never writes a builtin or an enclosing type..ownermarker and the pid in the name.Retention and warm cost
RegistryIsolationTests.CompileServer_FiftyRequests_LoadContextsCollect: 0 static-cache keys, 0 of 50 contexts alive after GC (ceiling 2), 2 share entries.FrontEndParity_AllFixturesat 13m51s. With the keyed share it takes 1m13s, against 1m14s at base.Direction record (base 486c9a8 → HEAD)
len/formatretyping (P47 cells); every crash gets its own bundle.from system.reactive.linq import Observablecompiled at base with a SPY0452 warning, and any use crashed at run. It is now SPY0301. Its base answer depended on sharpyc shipping System.Reactive.system.command_line,microsoft.code_analysis*.microsoft.extensions.primitivesgives SPY0300, as at base.from microsoft.visual_basic import Stringswent from SPY0300 to running (it is in the shared framework).try: the misleading SPY0220 is gone, and the SPY0510 underneath is a semantic: type-body statement totality — executable statements in a class/struct/interface/union body reach the emitter as SPY0510 ICEs instead of a semantic refusal #2148 cell.Filed by class this round
#2314, #2315, #2316, #2317 (new class trackers); cell comments on #2170, #2148, #2313, #2114, #2185 and #2042.
Residuals kept as ledger rows (latent or probe-only):
Listmembers reachable from frozen masters (no writer exists).-mpaths? Unchanged from base.-rlibrary binds to the host copy (same as base).ReplSessioninstance is repl: an expression statement whose value is void is echoed as a value — a call of a function/method with no return value is SPY0908 CS1503 at the prompt #2310/P72's domain.IsClrTypeGenericInNamespacereflection (Rule-2 residue).R-HI's dependency for #2182 (P42's door widening) is satisfied by 32ea542; noted there.
fix(cli): the failure exit kind is returned by CompileToBinary, not r…
test(stdlib): the closed-port socket cells retry a re-taken ephemeral…
test(server): {no -r, -r, no -r} on one server must answer as three f…
18 remaining items
perf(discovery): record the warm-compile cost of the P54 seams — with…
fix(discovery): the framework a compilation sees is the shared framew…
fix(semantic): every type definition declares its own symbol and a fr…
chore(spec-blocks): two wrapped overload examples are refused, not co…
test(build-tools): the stdlib-docs type roster and mapping citations …
antonsynd commentedon Oct 15, 2026
Closing on evidence. The last bullet of the close criterion was the CI run with the sixth sweep step. PR #2318 (dev → mainline) @ 045ce6b passed every check, including Sweep (in-process-isolation) (https://github.com/antonsynd/sharpy/actions/runs/38769686501/job/116543520330), all 7 Compiler shards, CLI/Core/Stdlib/LSP, and the other five sweeps.
Two follow-up commits were needed after the close-out comment (issuecomment-6194337667):
main()are now refused SPY0220 instead of compiling. At base they reached a CS0128 internal error when run. Both are allowlisted against semantic: a nested def that redefines a name in the same function scope is refused with a misleading SPY0220 at the call (was a CS0128 ICE) — rebind, overload or a clean refusal is undecided #2314; the third example's allowlist entry was stale and is deleted. The cell is posted on semantic: a nested def that redefines a name in the same function scope is refused with a misleading SPY0220 at the call (was a CS0128 ICE) — rebind, overload or a clean refusal is undecided #2314.Register(...). The test citations now match the new spelling.Each acceptance bullet with its sha, the direction record, retention and warm cost: see the close-out comment above.
Merge pull request #2318 from antonsynd/dev
antonsynd commentedon Oct 15, 2026
/verify-implementation plan-9ef825record (2026-10-15), scope 486c9a8..045ce6b → remediation d54e902 + f8c37f9. Gate @ 045ce6b: 80350/0/4; gate @ f8c37f9: 80351/0/4 (both measured, whole solution,Category!=Benchmark). Five auditors; the class claim held on every cell of the plan's matrix (route parity on 5 routes, server sequences, 8 concurrent clients, cross-process crash bundles 480/480 ×3 vs 255/480 at base, stdlib cross-thread cells = CPython/numpy, 849 builtin-shadowing triggers × 10 typed probes: 0 divergence on a warmCompilerApiat HEAD vs 3330 at base) — and was refuted in two places:InProcessIsolationSweepTestsreported 0 red at base and stayed green (9216 observations) with the visibility filter reverted AND with the master freeze reverted. Fixed in f8c37f9: builtin-shadowing triggers + typed probes with oracle controls, a per-run referenced-library pair, oneCompilerper reference set; now red under both reverts (12/21 regular rows, 6/38 GapDiscovery rows, 8/8) and at base (33 / 44 rows), green at HEAD with 0 allowlist rows; a row naming no corpus program is refused.ReferenceLoadContext.Load→Default;AssemblyCompiler.GetMetadataReferenceshands Roslyn the whole host TPA):-r <lone Microsoft.CodeAnalysis.CSharp.dll>compiles, then exit 134 FileNotFoundException at run — at BOTH shas (pre-existing, not a regression). Filed by class on interop: a -r assembly with an unloadable type aborts the compile (ReflectionTypeLoadException) — UnityEditor.CoreModule, UnityEngine.UI, TextCore, UIElements #2182 (with the-r O -r Aorder cell and the framework-named-reference cell); the seam move is the next plan's work, not this round's.Also fixed (d54e902): the five harnesses required allowlist/KnownRed rows to cite exactly
#2250— closed — so the first future red could not be allowlisted without reddeningcheck_allowlist_issue_state.sh; now the repo's#\d+convention. Direction sibling recorded: an unusedfrom system.composition import ExportAttributeran (SPY0452) at base and is SPY0301 now — the same mechanism as the owner-ruledsystem.reactive.linqcell. Two roster texts still described the pre-F3 framework set; the emitter's Rule-2 reflection site now cites #2320.Latent, in-contract, no observed cell (recorded here, no fix): master symbols created after the constructor (
_clrTypeCache,_interfaceSymbols) are never frozen — no writer found;ProcessWideResolutionScanTestsscanssrc/Sharpy.Compileronly and does not nameAssemblyLoadContext.Allor instance ALC loads as candidates; the own-symbol gate keys on_controlFlowDepth, which for/whileelse:bodies do not raise (no master write results; such a def has no symbol); the stdlib module lock is a public non-readonly static an importer can rebind (#2131). Filed elsewhere: #2319 (type int = strin a def accepted), #2320 (emitter reflection site), cells on #2114 (server with a rebuilt reference — refutes the "stayed correct" note above), #2314 (call-the-first nested def: SPY0220 → CS0128, refusal → ICE in kind), #2151, #2131; enum-in-def and the SPY0202 text stay on #2316.test(conformance): the P54 ratchets cite an issue by number, not the c…
test(conformance): the in-process isolation sweep is load-bearing — b…
Merge pull request #2323 from antonsynd/dev
test(property): TypeSoundness, ErrorCompleteness's base generator, Ty…
test(property): the #2212 explicit-instantiation pin is spelled `new …
feat(semantic): TypeSymbol.RefusedDeclarations records every property…