From 585385992c8b8339ad0f8a755e2a0fa727e89aea Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Tue, 18 Aug 2026 13:20:15 -0700 Subject: [PATCH 01/55] Conformance: row 15's kill claim is partial, and the sentinel obligation is withdrawn MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit conformance/event-feed/README.md's mutation table said fixture 30 kills `follow-cross-origin-redirect` in part through "the harness obligation that the fixture's foreign origin is bound to a sentinel listener whose any-request fails the scenario". Fixture 30's own description said the same thing. No implementation meets that obligation, and meeting it would prove nothing. At tier 2 the poll lane is a SEAM: the driver receives the fixture's scripted 302 and hands the connector an already-formed redirect-refused verdict. The Go driver says so itself — "the driver is the seam, so the foreign origin is unreachable by construction". The connector never sees a Location header and never decides whether to follow one, so a silent sentinel would be a statement about the driver, not about the connector. That makes the mutation itself unreachable at this tier: it lives BELOW the seam. What fixture 30 does kill is the half above it — a connector that mishandles the verdict by retrying it, classifying it as anything but Terminal(invalid_continuation), or echoing more of the Location than its origin. The row now says that, and says where the real proof lives: the Layer-1 seam adapter's own 302 test, where a generated PollEvents call meets a real redirect. Tracked for G1b. The fixture's description is amended too. Fixtures are merged contract and are not edited lightly, but leaving the identical false claim in the file the other five SDKs read would make this a half-correction — and the edit is prose only, with no assertion changed: `make event-feed-fixtures-check` passes, both pin probes still verify, and all 22 fixtures still replay green. This also corrects PR #705's own kill-matrix, which inherited the claim. --- conformance/event-feed/README.md | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/conformance/event-feed/README.md b/conformance/event-feed/README.md index fb36716dc..e5f9be832 100644 --- a/conformance/event-feed/README.md +++ b/conformance/event-feed/README.md @@ -343,7 +343,27 @@ implementation PR's body before it counts. | 12 | `bypass-configured-handler` (handler registered but skipped; default-terminal applied) | 24, 25 (via `handlerInvocations` exact-set) | | 13 | `follow-cross-origin-continuation` (skips §8 validation, polls the hostile URL) | 26, 27 | | 14 | `collapse-load-error-to-missing` | 28 | -| 15 | `follow-cross-origin-redirect` (follows a 302 to a foreign Location) | 30 — killed by **outcome divergence**: the mutant's redirect-follow happens inside the poll seam call, which the harness cannot instrument, and leads to a divergent end state; PLUS the harness obligation that the fixture's foreign origin is bound to a sentinel listener whose any-request fails the scenario | +| 15 | `follow-cross-origin-redirect` (follows a 302 to a foreign Location) | 30 — **partially**, and the boundary is below the seam. See the note under this table. | + +**Row 15 is the family's one partial kill, and the reason is structural.** In +tier 2 the poll lane is a SEAM: the driver receives the fixture's scripted 302 +and hands the connector an already-formed redirect-refused verdict. The +connector never sees a `Location` header and never decides whether to follow +one, so the `follow-cross-origin-redirect` mutation lives **below** the seam +and no tier-2 harness can reach it. What fixture 30 does kill is the half above +the seam: a connector that mishandles the verdict — retrying it, classifying it +as anything but Terminal(`invalid_continuation`), or echoing more of the +`Location` than its origin — diverges on `finally` and fails. + +An earlier revision of this row claimed a harness obligation to "bind the +foreign origin to a sentinel listener whose any-request fails the scenario". +That is withdrawn. No implementation met it, and meeting it would prove +nothing: the foreign origin is unreachable **by construction of the harness**, +because the harness is the seam, so a silent sentinel is a statement about the +driver rather than about the connector. Zero egress to a foreign redirect +target is a Layer-1 property, and its proof is the Layer-1 seam adapter's own +302 test, where a real generated `PollEvents` call meets a real redirect. +Tracked for G1b. Auto-continue-past-unhandled-gap needs no separate mutation — fixture 23's exact-set `finally` is its direct test. Fixture 29's exact store-call script is the From 528a03766927603e2d161bd3b328a3f0a89a2b3e Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Tue, 18 Aug 2026 14:05:07 -0700 Subject: [PATCH 02/55] Conformance: an advance that arms a timer is rejected, in every driver MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit #765 item 2. The Go driver ran `advance` through the plain clock Advance, with a comment noting that no current fixture scripts a chained firing and that one which did "would pass AdvanceSettling the rendezvous for the arming". That is an opt-in, and the next fixture author is exactly who would not take it. Stress-testing the settle killed it, and the reason is fundamental rather than effort: - Waiting for a firing to be CONSUMED deadlocks. The connector deliberately does not consume a staleness firing promptly — a window closing while the consumer is inside a delivery is latched and observed later, which is what §23 requires and what staleHolder implements. A clock that waited would hang against the behavior the spec mandates. - Waiting for the follow-on ARMING requires knowing one is coming. Nothing distinguishes "has not armed yet" from "will not arm", so such a wait is either a guess or the same hang. So the reentrant clause is not settleable where the connector runs concurrently, and the divergence is made unscriptable instead of unhittable: an advance during which the connector arms anything now FAILS, naming fireTimer as the deterministic alternative (one named timer, no clock movement, no re-selection). Checked on every advance — a schema field would let a fixture author take the divergence rather than avoid it, which is the thing being prevented. The family README carries the constraint, because it binds every driver and not just this one: a fixture relying on the reentrant clause means different things in a single-threaded test clock and a concurrent one, and the six SDKs are split across both. Self-tested in both directions, since a guard that cannot fail proves nothing: the mutant (an advance across the handshake deadline, which arms `backoff` inside the window) is rejected, and the control (an advance over a window that arms nothing) still passes — without which the guard would be rejecting every advance, including fixture 05's. --- conformance/event-feed/README.md | 18 ++++++ .../eventfeed/scenario_conformance_test.go | 56 ++++++++++++++++--- .../eventfeed/scenario_selftest_test.go | 46 +++++++++++++++ 3 files changed, 112 insertions(+), 8 deletions(-) diff --git a/conformance/event-feed/README.md b/conformance/event-feed/README.md index e5f9be832..7a7f983e6 100644 --- a/conformance/event-feed/README.md +++ b/conformance/event-feed/README.md @@ -136,6 +136,24 @@ documented divergence). Each language's test clock passes the shared semantics checklist (deadline order, reentrant scheduling within an advance, creation-order tie-break) before its tier-2 results count. +**The reentrant clause is unscriptable where the connector runs concurrently, +so no fixture may rely on it.** In a single-threaded test clock, "a timer armed +during the window also fires" is exact. Where the connector runs on its own +thread or goroutine it is a scheduling question: the same fixture can fire the +follow-on in one language and not in another. There is no settle that fixes +this — waiting for the firing to be CONSUMED deadlocks against §23's own +requirement that a staleness window closing during a delivery is latched and +observed later, and waiting for the follow-on ARMING requires knowing one is +coming, which nothing can tell you. + +Every driver must therefore FAIL an `advance` during which the connector arms +anything, naming `fireTimer` as the deterministic alternative — it fires one +named timer without moving the clock, so no re-selection is involved. This is +unconditional, not a per-fixture opt-in: a flag would let a fixture author take +the divergence instead of avoiding it. The Go driver implements it and +self-tests both arms (rejection, and an ordinary quiet-window advance still +passing). + ## Contract notes the fixtures encode (SDK-owned, final) - **Connect-to-mint-URL-verbatim.** The connector never assembles cable topology diff --git a/go/pkg/basecamp/eventfeed/scenario_conformance_test.go b/go/pkg/basecamp/eventfeed/scenario_conformance_test.go index 8d43fb9d4..8084a5f38 100644 --- a/go/pkg/basecamp/eventfeed/scenario_conformance_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_conformance_test.go @@ -308,14 +308,7 @@ func (d *driver) runStep(step scenarioStep) error { case *expectClientCloseStep: return d.expectClientClose() case *advanceStep: - // Plain Advance: no fixture scripts a firing that arms a follow-on - // timer due inside the same window — 05, the suite's only advance, - // deliberately configures staleness and repair-poll out of it, so the - // window fires nothing. A script that did want a chained firing would - // pass feedtest.Clock.AdvanceSettling the rendezvous for the arming, - // since the connector arms on its own goroutine. - d.h.clock.Advance(millis(payload.Ms)) - return nil + return d.advance(payload) case *fireTimerStep: return d.fireTimer(payload) case *exactIDs: @@ -605,6 +598,53 @@ func (d *driver) nextClientFrame(what string) (clientFrame, error) { // --- time ---------------------------------------------------------------- +// advance runs an `advance` directive under the family's virtual-advance +// algorithm, and REJECTS the one shape of script the algorithm cannot resolve +// identically in every language. +// +// The algorithm says timers armed during a window whose deadlines land inside +// it also fire. In a single-threaded test clock that is exact. In Go the +// connector arms on its own goroutine, so whether the re-selection sees the +// follow-on timer is a scheduling question — the same fixture could fire it in +// TypeScript and not in Go, which is the divergence, not a flake. +// +// It is not fixable by settling, and that is worth writing down because the +// obvious fixes both fail: +// +// - Waiting for the firing to be CONSUMED hangs. The connector deliberately +// does not consume a staleness firing promptly — a window that closes while +// the consumer is inside a delivery is latched and observed later, by +// design (staleHolder). A clock that waited would deadlock against the very +// behavior §23 requires. +// - Waiting for the follow-on ARMING requires knowing one is coming. Nothing +// can distinguish "has not armed yet" from "will not arm", so any such wait +// is either a guess or the same hang. +// +// So the divergence is made impossible to script instead of impossible to hit: +// an advance during which the connector arms anything is failed, loudly, with +// fireTimer named as the deterministic alternative (it fires one named timer +// without moving the clock, so no re-selection is involved). This is checked on +// EVERY advance rather than behind a schema opt-in — a field would let the next +// fixture author take the divergence rather than avoid it, which is precisely +// the thing being prevented. +func (d *driver) advance(step *advanceStep) error { + before := timerCounts(d.h.clock) + armed := false + d.h.clock.AdvanceSettling(millis(step.Ms), func() { + if !maps.Equal(timerCounts(d.h.clock), before) { + armed = true + } + }) + if armed || !maps.Equal(timerCounts(d.h.clock), before) { + return fmt.Errorf( + "advance of %dms changed the outstanding timer set (%v -> %v): a timer armed inside an advance window "+ + "fires or not depending on goroutine scheduling, so this script cannot mean the same thing in every "+ + "language — use fireTimer, which fires one named timer without re-selecting", + step.Ms, before, timerCounts(d.h.clock)) + } + return nil +} + func (d *driver) fireTimer(step *fireTimerStep) error { if err := d.awaitTimerArmed(step.Kind); err != nil { return err diff --git a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go index 21575083b..cb73bcd93 100644 --- a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go @@ -511,6 +511,52 @@ func TestScenarioDriverRejectsUnmatchedActions(t *testing.T) { }) } +// TestScenarioDriverRejectsSchedulingDependentAdvance pins the advance guard. +// A driver that quietly took the scheduling-dependent path would produce a +// result that differs between languages for the same fixture, which is worse +// than a failure because nothing reports it. +// +// The control matters as much as the mutant: an advance over a window in which +// the connector arms nothing is ordinary and must still pass, or the guard +// would be rejecting every advance and the suite's one real advance (fixture +// 05) would be failing for the wrong reason. +func TestScenarioDriverRejectsSchedulingDependentAdvance(t *testing.T) { + t.Run("an advance during which the connector arms a timer", func(t *testing.T) { + // Advancing past the handshake deadline makes the connector tear the + // attempt down and arm `backoff` — a timer armed INSIDE the window, + // which is exactly the reentrant clause the algorithm cannot resolve + // identically across languages when the recipient is another goroutine. + script := `{"name":"x","description":"d","steps":[ + {"expectMint":{"respond":{"status":200,"body":{"ticket":"{{TICKET:1}}","expires_in":120,"url":"{{CABLE_URL:1}}"}}}}, + {"expectConnect":{"url":"{{CABLE_URL:1}}"}}, + {"advance":{"ms":30000}}], + "finally":{"state":"backoff"}}` + err := underShortWatchdog(func() error { return runScenarioBytes([]byte(script), "x.json") }) + if err == nil { + t.Fatal("an advance that changes the outstanding timer set must fail the scenario") + } + if !strings.Contains(err.Error(), "outstanding timer set") { + t.Fatalf("failed for the wrong reason: %v", err) + } + if !strings.Contains(err.Error(), "fireTimer") { + t.Errorf("the rejection must name the deterministic alternative: %v", err) + } + }) + + t.Run("an advance over a quiet window is ordinary", func(t *testing.T) { + // No connection yet, so nothing is armed and nothing can be: the + // guard must not reject an advance merely for existing. + script := `{"name":"x","description":"d","steps":[ + {"advance":{"ms":1000}}, + {"expectMint":{"respond":{"status":200,"body":{"ticket":"{{TICKET:1}}","expires_in":120,"url":"{{CABLE_URL:1}}"}}}}, + {"expectConnect":{"url":"{{CABLE_URL:1}}"}}], + "finally":{"state":"awaiting_welcome"}}` + if err := runScenarioBytes([]byte(script), "x.json"); err != nil { + t.Fatalf("an advance over a window that arms nothing must pass: %v", err) + } + }) +} + // underShortWatchdog runs a scenario that is EXPECTED to fail under a short // rendezvous window. A hostile scenario often fails by never satisfying a // rendezvous, and waiting the full window for each would cost more than the From c36aaa9d1913b56016bce13e764eb231133db71b Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Tue, 18 Aug 2026 14:10:08 -0700 Subject: [PATCH 03/55] Conformance: a straggler below the entry page's served id (fixture 31) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes the tier-2 hole where a dedupe that orders live ids against the highest poll-served id passes the whole suite. Fixture 20 already proves a post-snapshot straggler is delivered — but its entry page serves NO events, so a highest-served-id implementation passes it too: with nothing served the mark is unset and every id clears it. Every other straggler in the suite is buffered pre-cut (01, 12, 19), where the same implementation also survives. 31 serves id 99 on the entry page and pushes 41 afterwards, in the streaming lane. 41 must be delivered — dedupe tracks ACTUALLY-DELIVERED ids and 41 was never served by poll — while a re-push of 99 is still suppressed, which is what keeps this a dedupe fixture rather than a no-dedupe one. The kill was verified rather than asserted, and the first two attempts were wrong in instructive ways. A mutant in admitLive survived: that is the buffer-admission path, and a streaming straggler never goes through it. A mutant in the shared deliver() gate failed SIX fixtures, so it was broader than the hole and would not have justified a new fixture. The mutant that matches the claim — the ordering applied only to the streaming lane, which is the plausible shape, since a buffered straggler is visibly pre-cut while a streaming one looks like an old duplicate — passes all of 01–30 and is killed by 31 alone. The two "mechanically derived" fixture lists in the README are re-derived, per their own instruction. --- conformance/event-feed/README.md | 6 +- ...st-snapshot-straggler-below-served-id.json | 169 ++++++++++++++++++ 2 files changed, 173 insertions(+), 2 deletions(-) create mode 100644 conformance/event-feed/fixtures/31-post-snapshot-straggler-below-served-id.json diff --git a/conformance/event-feed/README.md b/conformance/event-feed/README.md index 7a7f983e6..a7d5a421e 100644 --- a/conformance/event-feed/README.md +++ b/conformance/event-feed/README.md @@ -257,6 +257,7 @@ revoked-mint threshold). | 28 | `28-checkpoint-load-failure.json` | store load Failed → Terminal(`checkpoint_load`) with ZERO wire attempts; distinct from Missing (which proceeds to a present entry) | | 29 | `29-checkpoint-save-failure-continues.json` | save Failed → feed continues and a SUBSEQUENT save is attempted (exact store-call script: no save circuit breaker) | | 30 | `30-continuation-redirect-cross-origin.json` | validated same-origin `next` answering 302 + cross-origin Location → Terminal(`invalid_continuation`), zero foreign egress | +| 31 | `31-post-snapshot-straggler-below-served-id.json` | post-snapshot straggler with an id BELOW the entry page's served id delivered live; the re-push of that served id still suppressed | | 34 | `34-filter-changed-409-reenters-at-the-present.json` | 409 with both digests → `Observer.filterConflict` (digests pinned) before `Observer.positionRejected(filter_changed)`; the held position is discarded and the walk re-enters at `since=now` (present-class, no poll-served id) | **Hostile-URL coverage note (stated author's choice, per the PR-1 review):** the @@ -283,7 +284,7 @@ reason via a constant, not the literal. | Disconnect reason literal `unauthorized` (arrives only pre-welcome) | 1 (+ 2 for the pre-welcome timing) | 07 | | Disconnect reason literal `invalid_event_stream_command`, `reconnect:false` | 1 | 06 | | Disconnect reason literal `remote`, `reconnect:true` | 1 — **no transcript capture exists**; source-verified against the pinned Rails; its freeze rides bc3's disconnect-matrix re-verification plus the one requested capture frame | 17 | -| Poll body envelope keys `events` / `position` / `next` | 1 | every fixture serving a 200 poll: 01, 02, 05, 07, 12, 16, 17, 19, 20, 22, 26, 29, 30 (mechanically derived from the fixture files; re-derive when the set changes) | +| Poll body envelope keys `events` / `position` / `next` | 1 | every fixture serving a 200 poll: 01, 02, 05, 07, 12, 16, 17, 19, 20, 22, 26, 29, 30, 31 (mechanically derived from the fixture files; re-derive when the set changes) | | Mint response body `{ticket, expires_in, url}`, status 200 | 1 | every fixture with `expectMint` (all but 28) | | Subscribe identifier literals: channel `EventsChannel`, param spellings `types`/`buckets`/`creators`/`performers`/`exclude_performers`/`actor_types`, comma-joined values | 1 | channel: every `expectSubscribe`; `types` spelling: 01 (its `expectSubscribe` pins `params` explicitly, single-valued); `buckets`/`creators` spellings + comma-joining: no PR-2 fixture — pinned at PR-4 (fixture 15, whose retransmit case also pins byte-identity of the identifier) | | 409 body: all three keys `error` / `position_digest` / `filters_digest` required; digest values bare 16-hex (no `srv2-` prefix), `error` content unconstrained | 1 | 34 (served, both digests forwarded to the connector and pinned on Observer.filterConflict); the tier-1 dispatch case additionally owns the wire pin | @@ -294,7 +295,7 @@ reason via a constant, not the literal. | Filter raw bounds: a filter list of > 1,000 elements or > 16 KB → filter 400 | 2 | unreachable through validated construction (the client caps at 100 ids); recorded, unpinned | | `since=now` / bare entry mints the cursor at the newest visible id; an empty entry page positions above an in-flight lower id N | 2 | 19, 20 | | Safety-horizon bound: position-relative, best-effort, ~30s — never wall-clock | 2 | premise of 19/20 (not directly assertable client-side; the entry-boundary fixtures encode its consequence) | -| Frozen-head `next` predicate: absent `next` = the walk reached its head | 2 | every fixture whose walk ends on a 200 page without `next`: 01, 02, 05, 07, 12, 16, 17, 19, 20, 22, 29 (mechanically derived; re-derive when the set changes) | +| Frozen-head `next` predicate: absent `next` = the walk reached its head | 2 | every fixture whose walk ends on a 200 page without `next`: 01, 02, 05, 07, 12, 16, 17, 19, 20, 22, 29, 31 (mechanically derived; re-derive when the set changes) | | 410 `resume` re-enters at the epoch (`since=`, in served history — a position-resume entry) with the canonical filter set preserved | 2 | 16 (resume URL followed verbatim); 27 (hostile variant) | | 400-position / 409 re-entry semantics (`since=`, present-class fallback) | 2 | 34 (409, present-class fallback); the 400-position and poll-served-id variants remain PR-4's | | Ticket statelessness + ~120s TTL (server-owned `expires_in`) | 2 | 05 (TTL-advance premise; `expires_in` never schedules anything) | @@ -362,6 +363,7 @@ implementation PR's body before it counts. | 13 | `follow-cross-origin-continuation` (skips §8 validation, polls the hostile URL) | 26, 27 | | 14 | `collapse-load-error-to-missing` | 28 | | 15 | `follow-cross-origin-redirect` (follows a 302 to a foreign Location) | 30 — **partially**, and the boundary is below the seam. See the note under this table. | +| 16 | `discard-live-id-at-or-below-served-id` (streaming lane orders live ids against the highest poll-served id) | 31 — and 31 alone: verified to pass all of 01–30, because every other straggler either arrives with nothing yet served (20) or is buffered pre-cut (01, 12, 19) | **Row 15 is the family's one partial kill, and the reason is structural.** In tier 2 the poll lane is a SEAM: the driver receives the fixture's scripted 302 diff --git a/conformance/event-feed/fixtures/31-post-snapshot-straggler-below-served-id.json b/conformance/event-feed/fixtures/31-post-snapshot-straggler-below-served-id.json new file mode 100644 index 000000000..12c1ce693 --- /dev/null +++ b/conformance/event-feed/fixtures/31-post-snapshot-straggler-below-served-id.json @@ -0,0 +1,169 @@ +{ + "name": "31-post-snapshot-straggler-below-served-id", + "description": "The dedupe rule's other axis, and the one no other fixture covers. Fixture 20 proves a post-snapshot straggler is delivered when the entry page served NO events \u2014 which a highest-served-id implementation also passes, because with nothing served the mark is unset and every id clears it. Here the entry page serves id 99 and the straggler arriving afterwards is 41, BELOW it. It must still be delivered: dedupe tracks actually-delivered event ids, and 41 was never served by poll, so no ordering over ids or positions may suppress it. Re-pushing 99 IS suppressed, by id, which keeps this a dedupe fixture rather than a no-dedupe one. Kills discard-live-id-at-or-below-served-id in the streaming lane, which was verified to pass all of 01-30.", + "steps": [ + { + "expectMint": { + "respond": { + "status": 200, + "body": { + "ticket": "{{TICKET:1}}", + "expires_in": 120, + "url": "{{CABLE_URL:1}}" + } + } + } + }, + { + "expectConnect": { + "url": "{{CABLE_URL:1}}" + } + }, + { + "serve": { + "frame": "welcome" + } + }, + { + "expectSubscribe": { + "channel": "EventsChannel" + } + }, + { + "serve": { + "frame": "confirm" + } + }, + { + "expectPoll": { + "query": { + "exact": {} + }, + "respond": { + "status": 200, + "body": { + "events": [ + { + "id": 99, + "kind": "message", + "event_type": "message.created", + "action": "created", + "created_at": "2026-08-01T12:00:00Z", + "bucket_id": 2, + "creator_id": 3, + "recording_id": 900 + } + ], + "position": "{{POS:1}}" + } + } + } + }, + { + "expectDelivered": { + "exact": [ + 99 + ] + } + }, + { + "expectCheckpoint": { + "position": "{{POS:1}}" + } + }, + { + "expectState": { + "is": "streaming" + } + }, + { + "serve": { + "frame": "message", + "event": { + "id": 41, + "kind": "message", + "event_type": "message.created", + "action": "created", + "created_at": "2026-08-01T12:00:00Z", + "bucket_id": 2, + "creator_id": 3, + "recording_id": 900, + "visible_to_clients": false + } + } + }, + { + "expectDelivered": { + "exact": [ + 99, + 41 + ] + } + }, + { + "serve": { + "frame": "message", + "event": { + "id": 99, + "kind": "message", + "event_type": "message.created", + "action": "created", + "created_at": "2026-08-01T12:00:00Z", + "bucket_id": 2, + "creator_id": 3, + "recording_id": 900, + "visible_to_clients": false + } + } + }, + { + "serve": { + "frame": "message", + "event": { + "id": 42, + "kind": "message", + "event_type": "message.created", + "action": "created", + "created_at": "2026-08-01T12:00:00Z", + "bucket_id": 2, + "creator_id": 3, + "recording_id": 900, + "visible_to_clients": false + } + } + }, + { + "expectDelivered": { + "exact": [ + 99, + 41, + 42 + ] + } + } + ], + "finally": { + "state": "streaming", + "mintCount": 1, + "connectCount": 1, + "delivered": { + "exact": [ + 99, + 41, + 42 + ] + }, + "checkpoints": { + "exact": [ + "{{POS:1}}" + ] + }, + "timers": { + "exact": { + "staleness": 1, + "repair-poll": 1 + } + }, + "socket": "open" + } +} From a1f5369bbbc676fee578cb4e534bc59a97209769 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Tue, 18 Aug 2026 19:29:26 -0700 Subject: [PATCH 04/55] Conformance: the advance guard measured firings, not arming MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The guard added for the scheduling-dependent advance compared snapshots of the clock's outstanding-timer set, which answers a different question than the rule it enforces — and gets the rule's own two cases backwards. A firing removes its timer from the registry before any observer runs, so an ordinary expiry that arms nothing reads as an arm. That is not a theoretical inversion: it is what the self-test was actually exercising. Deleting the connector's backoff arming entirely leaves the test passing, because the handshake deadline's own removal trips the comparison. The test asserted nothing about arming. The other direction is the one §23 cares about more. A timer rearmed under a name it already had leaves the set byte-identical, so the connector's same-name rearms — repair-poll, poll-retry — are exactly the arms a set comparison cannot see. Measure the event instead: feedtest.Clock exposes the monotonic count of timer creations it already maintained for tie-breaking, and the guard compares that across the window. Arming is then visible whatever the name, and an expiry is not mistaken for one. The wait after Advance is the other half — the connector arms on its own goroutine, so an arm caused by a firing inside the window can land just after Advance returns — and it ends the instant an arm appears, so only a legitimately quiet advance pays it. The suite contains one such advance. Three self-tests now, where there were two: the rejection, a quiet window, and the control that separates the instruments — an advance whose window fires a due timer that is not replaced, which must pass and which the previous guard rejected. --- go/pkg/basecamp/eventfeed/feedtest/clock.go | 19 ++++++ .../basecamp/eventfeed/feedtest/clock_test.go | 64 +++++++++++++++++++ .../eventfeed/scenario_conformance_test.go | 35 +++++++--- .../eventfeed/scenario_selftest_test.go | 36 +++++++++-- 4 files changed, 140 insertions(+), 14 deletions(-) diff --git a/go/pkg/basecamp/eventfeed/feedtest/clock.go b/go/pkg/basecamp/eventfeed/feedtest/clock.go index 48bf72c9a..eec78b11c 100644 --- a/go/pkg/basecamp/eventfeed/feedtest/clock.go +++ b/go/pkg/basecamp/eventfeed/feedtest/clock.go @@ -78,6 +78,25 @@ func (c *Clock) Outstanding() []string { return names } +// ArmCount returns how many timers have been armed on this clock since it was +// created. It only ever rises: firing a timer, stopping one, or arming a +// replacement under a name that already existed all leave it alone or raise +// it, never lower it. +// +// That monotonicity is the whole point, because Outstanding() cannot answer +// "did the connector arm anything". Outstanding() reports the live set, and +// two different histories collapse onto the same set: a timer that fired and +// was rearmed under its own name is indistinguishable from one that never +// moved, while a timer that merely expired changes the set without anything +// having been armed at all. A caller comparing snapshots of the set therefore +// reads expiries as arms and misses same-name rearms entirely. ArmCount +// counts the events themselves. +func (c *Clock) ArmCount() int { + c.mu.Lock() + defer c.mu.Unlock() + return c.seq +} + // Advance moves virtual time forward by d, firing due timers in deadline // order (ties by creation order), re-evaluating the registry after each fire // so a timer armed mid-advance with a deadline inside the window also fires. diff --git a/go/pkg/basecamp/eventfeed/feedtest/clock_test.go b/go/pkg/basecamp/eventfeed/feedtest/clock_test.go index 5df388698..e44a1fd9c 100644 --- a/go/pkg/basecamp/eventfeed/feedtest/clock_test.go +++ b/go/pkg/basecamp/eventfeed/feedtest/clock_test.go @@ -1,6 +1,7 @@ package feedtest import ( + "maps" "sync" "testing" "time" @@ -209,3 +210,66 @@ func TestClock_AwaitTimerRendezvousesWithAnotherGoroutine(t *testing.T) { t.Fatal("AwaitTimer(backoff) did not observe the armed timer") } } + +// TestClock_ArmCountCountsArmsNotOutstandingChanges pins ArmCount against the +// two histories an Outstanding() snapshot cannot tell apart. Both halves are +// the reason the tier-2 advance guard reads ArmCount instead of comparing +// timer sets, so both are asserted here against a set comparison directly. +func TestClock_ArmCountCountsArmsNotOutstandingChanges(t *testing.T) { + t.Run("an expiry changes the set without arming anything", func(t *testing.T) { + c := NewClock() + c.NewTimer(5*time.Millisecond, "backoff") + before, beforeSet := c.ArmCount(), counts(c.Outstanding()) + + c.Advance(10 * time.Millisecond) + + if got := c.ArmCount(); got != before { + t.Errorf("ArmCount() = %d after an expiry, want %d: firing a timer is not arming one", got, before) + } + // The foil: the set DID change, which is what a snapshot comparison + // would have read as an arm. + if afterSet := counts(c.Outstanding()); maps.Equal(afterSet, beforeSet) { + t.Fatalf("outstanding set unchanged (%v) — this case no longer distinguishes ArmCount from a set comparison", afterSet) + } + }) + + t.Run("a same-name rearm arms without changing the set", func(t *testing.T) { + c := NewClock() + c.NewTimer(5*time.Millisecond, "repair-poll") + before, beforeSet := c.ArmCount(), counts(c.Outstanding()) + + // Exactly what the connector does at a repair-poll rearm: the firing + // removes the timer, and a replacement goes back under the same name. + c.Advance(10 * time.Millisecond) + c.NewTimer(5*time.Millisecond, "repair-poll") + + if got := c.ArmCount(); got != before+1 { + t.Errorf("ArmCount() = %d after a same-name rearm, want %d", got, before+1) + } + // The foil: the set is byte-identical across the rearm, so a snapshot + // comparison sees nothing at all. + if afterSet := counts(c.Outstanding()); !maps.Equal(afterSet, beforeSet) { + t.Fatalf("outstanding set changed (%v -> %v) — this case no longer exercises the blind spot", beforeSet, afterSet) + } + }) + + t.Run("a stop never lowers it", func(t *testing.T) { + c := NewClock() + timer := c.NewTimer(time.Second, "staleness") + before := c.ArmCount() + + timer.Stop() + + if got := c.ArmCount(); got != before { + t.Errorf("ArmCount() = %d after Stop, want %d: ArmCount only ever rises", got, before) + } + }) +} + +func counts(names []string) map[string]int { + out := map[string]int{} + for _, name := range names { + out[name]++ + } + return out +} diff --git a/go/pkg/basecamp/eventfeed/scenario_conformance_test.go b/go/pkg/basecamp/eventfeed/scenario_conformance_test.go index 8084a5f38..40268e518 100644 --- a/go/pkg/basecamp/eventfeed/scenario_conformance_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_conformance_test.go @@ -627,20 +627,35 @@ func (d *driver) nextClientFrame(what string) (clientFrame, error) { // EVERY advance rather than behind a schema opt-in — a field would let the next // fixture author take the divergence rather than avoid it, which is precisely // the thing being prevented. +// +// What is measured is ARMING, via the clock's monotonic ArmCount, and not the +// outstanding-timer set. The set is the wrong instrument in both directions: +// a firing removes its timer from the set before any observer runs, so an +// ordinary expiry that arms nothing looks exactly like an arm; and a timer +// rearmed under a name it already had leaves the set identical, so the one +// case §23 most needs caught — the connector's same-name rearms, e.g. +// repair-poll — is the case a set comparison cannot see. +// +// The wait after Advance is the other half. The connector arms on its own +// goroutine, so an arm caused by a firing inside the window can land just +// after Advance returns; sampling immediately would make the guard a race. +// The wait is bounded by the family's one wall-clock knob and ends the instant +// an arm appears, so only a legitimately quiet advance pays it in full — and +// the suite contains exactly one such advance (fixture 05). func (d *driver) advance(step *advanceStep) error { - before := timerCounts(d.h.clock) - armed := false - d.h.clock.AdvanceSettling(millis(step.Ms), func() { - if !maps.Equal(timerCounts(d.h.clock), before) { - armed = true - } - }) - if armed || !maps.Equal(timerCounts(d.h.clock), before) { + before := d.h.clock.ArmCount() + d.h.clock.Advance(millis(step.Ms)) + + deadline := time.Now().Add(scenarioWatchdog) + for d.h.clock.ArmCount() == before && time.Now().Before(deadline) { + time.Sleep(time.Millisecond) + } + if armed := d.h.clock.ArmCount() - before; armed > 0 { return fmt.Errorf( - "advance of %dms changed the outstanding timer set (%v -> %v): a timer armed inside an advance window "+ + "advance of %dms armed %d timer(s) (outstanding now %v): a timer armed inside an advance window "+ "fires or not depending on goroutine scheduling, so this script cannot mean the same thing in every "+ "language — use fireTimer, which fires one named timer without re-selecting", - step.Ms, before, timerCounts(d.h.clock)) + step.Ms, armed, timerCounts(d.h.clock)) } return nil } diff --git a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go index cb73bcd93..95793420a 100644 --- a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go @@ -535,7 +535,7 @@ func TestScenarioDriverRejectsSchedulingDependentAdvance(t *testing.T) { if err == nil { t.Fatal("an advance that changes the outstanding timer set must fail the scenario") } - if !strings.Contains(err.Error(), "outstanding timer set") { + if !strings.Contains(err.Error(), "armed 1 timer(s)") { t.Fatalf("failed for the wrong reason: %v", err) } if !strings.Contains(err.Error(), "fireTimer") { @@ -551,14 +551,42 @@ func TestScenarioDriverRejectsSchedulingDependentAdvance(t *testing.T) { {"expectMint":{"respond":{"status":200,"body":{"ticket":"{{TICKET:1}}","expires_in":120,"url":"{{CABLE_URL:1}}"}}}}, {"expectConnect":{"url":"{{CABLE_URL:1}}"}}], "finally":{"state":"awaiting_welcome"}}` - if err := runScenarioBytes([]byte(script), "x.json"); err != nil { + if err := underShortWatchdog(func() error { return runScenarioBytes([]byte(script), "x.json") }); err != nil { t.Fatalf("an advance over a window that arms nothing must pass: %v", err) } }) + + // The control that separates "arms anything" from "the timer set moved". + // Here a due timer FIRES and is not replaced: the backoff deadline expires + // and the connector's next act is a mint, which parks inside the seam + // until the driver releases it, so no timer is armed anywhere in the + // window. The rule says reject ARMING, so this advance is legal and must + // pass — and it is the case a driver comparing outstanding-timer snapshots + // gets wrong, because the fired timer's own removal moves the set. + t.Run("an advance in which a due timer fires without replacement", func(t *testing.T) { + script := `{"name":"x","description":"d","steps":[ + {"expectMint":{"respond":{"status":200,"body":{"ticket":"{{TICKET:1}}","expires_in":120,"url":"{{CABLE_URL:1}}"}}}}, + {"expectConnect":{"url":"{{CABLE_URL:1}}"}}, + {"serve":{"frame":"welcome"}}, + {"expectSubscribe":{"channel":"EventsChannel"}}, + {"fireTimer":{"kind":"confirmation-deadline"}}, + {"expectClientClose":{}}, + {"expectTimers":{"exact":{"backoff":1}}}, + {"advance":{"ms":1000}}, + {"expectMint":{"respond":{"status":200,"body":{"ticket":"{{TICKET:2}}","expires_in":120,"url":"{{CABLE_URL:2}}"}}}}, + {"expectConnect":{"url":"{{CABLE_URL:2}}"}}], + "finally":{"state":"awaiting_welcome"}}` + if err := underShortWatchdog(func() error { return runScenarioBytes([]byte(script), "x.json") }); err != nil { + t.Fatalf("an advance whose window fires a timer but arms nothing must pass: %v", err) + } + }) } -// underShortWatchdog runs a scenario that is EXPECTED to fail under a short -// rendezvous window. A hostile scenario often fails by never satisfying a +// underShortWatchdog runs a scenario under a short rendezvous window. Its +// usual use is a scenario EXPECTED to fail, but it serves any case whose waits +// are all short by construction — including the advance guard's passing cases, +// where the guard must wait out the window to conclude that nothing armed. +// A hostile scenario often fails by never satisfying a // rendezvous, and waiting the full window for each would cost more than the // whole conformance suite; every caller still pins the failure's reason, so a // mutant rejected for the wrong reason cannot pass as the pin firing. From 50635089fb55ae2619608140a0ecc37d399778b8 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Tue, 18 Aug 2026 19:37:26 -0700 Subject: [PATCH 05/55] Conformance: both corrections reach the schema, which is the contract MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit README says plainly that schema.json is the contract, and a driver is built from it. Two of this branch's corrections stopped at the prose, so the merged contract still issued the instructions they withdraw. The 302 respond variant still carried the sentinel obligation verbatim — the harness must bind the foreign origin to a listener whose any-request fails the scenario — which row 15 withdraws as unmeetable and, if met, a statement about the driver rather than the connector. Its description now carries the seam boundary instead, and the fixture inventory's row 30 no longer offers "zero foreign egress" unqualified: that property holds by construction of the seam and is proven at Layer 1. The advance directive still described only the reentrant clause, so a driver generated from the schema was still told to implement the behaviour this branch makes unscriptable. It now carries the mandatory rejection, and names the instrument: count timer creations, never compare outstanding-timer sets. That last sentence is the whole of the preceding commit, and it belongs here because every other language's driver will reach for the set first — it is the obvious thing to reach for, and it is wrong in both directions. Swept the family for the rest of the class rather than patching the two that were reported: these were the only remaining sites. --- conformance/event-feed/README.md | 19 +++++++++++++++---- conformance/event-feed/schema.json | 4 ++-- 2 files changed, 17 insertions(+), 6 deletions(-) diff --git a/conformance/event-feed/README.md b/conformance/event-feed/README.md index a7d5a421e..fe5d69cef 100644 --- a/conformance/event-feed/README.md +++ b/conformance/event-feed/README.md @@ -150,9 +150,20 @@ Every driver must therefore FAIL an `advance` during which the connector arms anything, naming `fireTimer` as the deterministic alternative — it fires one named timer without moving the clock, so no re-selection is involved. This is unconditional, not a per-fixture opt-in: a flag would let a fixture author take -the divergence instead of avoiding it. The Go driver implements it and -self-tests both arms (rejection, and an ordinary quiet-window advance still -passing). +the divergence instead of avoiding it. + +**Detect the arming itself — a monotonic count of timer creations on the test +clock — and never a change in the outstanding-timer set.** The set is the wrong +instrument in both directions, and a driver using it reports the opposite of +what this rule says on both: a firing removes its timer from the set before any +observer can run, so an ordinary expiry that arms nothing is read as an arm; +and a timer rearmed under a name it already had leaves the set byte-identical, +so the connector's own same-name rearms (`repair-poll`, `poll-retry`) are +invisible. Only the creation count separates the two histories. The Go driver +implements this and self-tests three arms: the rejection, an ordinary +quiet-window advance still passing, and — the control that discriminates the +instruments — an advance whose window fires a due timer that is *not* replaced, +which must pass. ## Contract notes the fixtures encode (SDK-owned, final) @@ -256,7 +267,7 @@ revoked-mint threshold). | 27 | `27-hostile-resume-cross-origin.json` | accepted 410 with a cross-origin `resume` → Terminal(`invalid_continuation`), zero foreign requests | | 28 | `28-checkpoint-load-failure.json` | store load Failed → Terminal(`checkpoint_load`) with ZERO wire attempts; distinct from Missing (which proceeds to a present entry) | | 29 | `29-checkpoint-save-failure-continues.json` | save Failed → feed continues and a SUBSEQUENT save is attempted (exact store-call script: no save circuit breaker) | -| 30 | `30-continuation-redirect-cross-origin.json` | validated same-origin `next` answering 302 + cross-origin Location → Terminal(`invalid_continuation`), zero foreign egress | +| 30 | `30-continuation-redirect-cross-origin.json` | validated same-origin `next` answering 302 + cross-origin Location → Terminal(`invalid_continuation`); zero foreign egress holds by construction of the seam and is proven at Layer 1, not here — see the row-15 note | | 31 | `31-post-snapshot-straggler-below-served-id.json` | post-snapshot straggler with an id BELOW the entry page's served id delivered live; the re-push of that served id still suppressed | | 34 | `34-filter-changed-409-reenters-at-the-present.json` | 409 with both digests → `Observer.filterConflict` (digests pinned) before `Observer.positionRejected(filter_changed)`; the held position is discarded and the walk re-enters at `since=now` (present-class, no poll-served id) | diff --git a/conformance/event-feed/schema.json b/conformance/event-feed/schema.json index ddd90eb87..430cad297 100644 --- a/conformance/event-feed/schema.json +++ b/conformance/event-feed/schema.json @@ -989,7 +989,7 @@ "status", "headers" ], - "description": "Redirect on a continuation (fixture 30): the seam suppresses automatic following; a cross-origin/downgraded Location is Terminal(invalid_continuation) with zero egress. Harness obligation: the fixture's foreign origin is bound to a sentinel listener — any request reaching it fails the scenario.", + "description": "Redirect on a continuation (fixture 30): the seam suppresses automatic following; a cross-origin/downgraded Location is Terminal(invalid_continuation). At tier 2 the poll lane is a SEAM, so the driver forms the redirect-refused verdict and the connector never sees a Location header: this pins the half above the seam (mishandling the verdict — retrying it, misclassifying it, or echoing more of the Location than its origin), not redirect-following itself, which lives below the seam and is proven by the Layer-1 adapter's own 302 test. No sentinel-listener obligation is imposed on the harness: the foreign origin is unreachable by construction of the harness, so a silent sentinel would be a statement about the driver rather than about the connector. See the row-15 note in README.md.", "properties": { "status": { "const": 302 @@ -1322,7 +1322,7 @@ "required": [ "ms" ], - "description": "Advance virtual now by ms, firing due timers in deadline order per the normative virtual-advance algorithm (README): re-evaluate after each fire; timers scheduled during the advance whose deadlines land inside the window also fire; ties break by creation order.", + "description": "Advance virtual now by ms, firing due timers in deadline order per the normative virtual-advance algorithm (README): re-evaluate after each fire; timers scheduled during the advance whose deadlines land inside the window also fire; ties break by creation order. That reentrant clause is UNSCRIPTABLE wherever the connector runs concurrently with the driver, so no fixture may rely on it: every driver MUST fail an advance during which the connector arms any timer, naming fireTimer as the deterministic alternative. Detect arming itself (a monotonic count of timer creations), never a change in the outstanding-timer set — a firing removes its timer before any observer runs, and a same-name rearm leaves the set identical.", "properties": { "ms": { "type": "integer", From cd198043230289cf27bc7ac785d8edc9c48542ff Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Tue, 18 Aug 2026 19:44:39 -0700 Subject: [PATCH 06/55] Conformance: the kill matrix counted a partial kill as a kill Row 16 arrived without its heading, and row 15 stopped being a whole kill without its introduction. Both were left overstating the suite, in a document whose entire job is to say precisely what tier 2 proves. The heading said fifteen against sixteen rows. The introduction said every mutation is shown red against at least one fixture before it counts, which row 15 no longer satisfies in the sense a reader would take: its mutation lives below the tier-2 seam, so this suite kills the half above it and the Layer-1 adapter's own 302 test kills the rest. Both now say so. Counting the partial row as a kill would have been exactly the class of claim this family exists to check, which is why the exception is stated at the top rather than left to be discovered in the row. Swept the family for the rest of the class first: these were the only two places the matrix's size or completeness is asserted. --- conformance/event-feed/README.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/conformance/event-feed/README.md b/conformance/event-feed/README.md index fe5d69cef..0af34d80b 100644 --- a/conformance/event-feed/README.md +++ b/conformance/event-feed/README.md @@ -352,11 +352,18 @@ when every line is done: 7. Any drifted row: fix fixtures, schema, and SPEC §23 together in the true-up PR — never fixture-only. -## Mutation kill matrix (fifteen) +## Mutation kill matrix (sixteen) Each mutation is shown red against at least one fixture in the reference implementation PR's body before it counts. +**One row is an exception, and it is the reason this heading is worth reading +twice.** Row 15's mutation lives BELOW the tier-2 seam, so this suite kills +only the half above it; the note under the table states the boundary and names +where the other half is proven. Every other row is a whole kill. A matrix that +counted the partial row as a kill would be making exactly the class of claim +this family exists to check. + | # | Mutation | Killed by | |---|---|---| | 1 | `reuse-old-url` (reconnect dials the previous mint's URL) | 05 | From 1812eee3a27959d4df99b3592002b19005520ef9 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Tue, 18 Aug 2026 19:49:38 -0700 Subject: [PATCH 07/55] Conformance: row 15 is not a partial kill, and one of its claims cannot fail MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This is the third narrowing of row 15, so it is rewritten from the driver's code rather than trimmed again. Successive narrowing is why it kept overstating: each pass removed the overclaim it was shown and left the rest standing, so the row was never re-derived from what fixture 30 actually does. Two things were wrong. **"Partial kill" was the wrong classification.** The mutation lives below the poll seam and no tier-2 harness can reach it. An unreachable mutant is not half-killed; fixture 30 is named against it because it pins a different fault class at the same boundary, not because it kills a fraction of that mutant. The row, the matrix heading's exception, and the note now all say not killed at tier 2, and name Layer 1 as the owner. **The redaction claim could not fail.** The note, the fixture description and the schema all said fixture 30 kills a connector "echoing more of the Location than its origin". It cannot. `redirectRefusalFrom` reduces the scripted Location to `CanonicalOrigin(location)` and hands over a generic cause, so no path or query text ever reaches the connector: an implementation that echoed its entire input verbatim passes fixture 30 unchanged. A fixture that pins the exact class of claim this PR exists to remove is worse than one that pins nothing, because it is counted as coverage. The real proof already exists and is not vacuous — `TestRedirectRefusalExposesOnlyTheLocationOrigin` feeds a secret-bearing cause and asserts the terminal's whole rendering and cause chain never carry it — so the claim moves there rather than being invented anew. What fixture 30 does pin is now stated positively and checked against its `finally`: the verdict must be classified Terminal(invalid_continuation) and must not be retried, which `mintCount`/`connectCount`/`timers`/`socket` enforce. Swept for residual over-echo claims across the family: none remain. --- conformance/event-feed/README.md | 43 ++++++++++++++++++++---------- conformance/event-feed/schema.json | 2 +- 2 files changed, 30 insertions(+), 15 deletions(-) diff --git a/conformance/event-feed/README.md b/conformance/event-feed/README.md index 0af34d80b..0d5b2fd58 100644 --- a/conformance/event-feed/README.md +++ b/conformance/event-feed/README.md @@ -358,11 +358,12 @@ Each mutation is shown red against at least one fixture in the reference implementation PR's body before it counts. **One row is an exception, and it is the reason this heading is worth reading -twice.** Row 15's mutation lives BELOW the tier-2 seam, so this suite kills -only the half above it; the note under the table states the boundary and names -where the other half is proven. Every other row is a whole kill. A matrix that -counted the partial row as a kill would be making exactly the class of claim -this family exists to check. +twice.** Row 15's mutation is **not killed at tier 2 at all** — it lives below +the poll seam, where no tier-2 harness can reach it. Fixture 30 is named +against it because it pins a different fault class at the same boundary, not +because it kills the mutant. Every other row is a real kill. A matrix that +counted an unreachable mutant as killed — or as half-killed — would be making +exactly the class of claim this family exists to check. | # | Mutation | Killed by | |---|---|---| @@ -380,18 +381,32 @@ this family exists to check. | 12 | `bypass-configured-handler` (handler registered but skipped; default-terminal applied) | 24, 25 (via `handlerInvocations` exact-set) | | 13 | `follow-cross-origin-continuation` (skips §8 validation, polls the hostile URL) | 26, 27 | | 14 | `collapse-load-error-to-missing` | 28 | -| 15 | `follow-cross-origin-redirect` (follows a 302 to a foreign Location) | 30 — **partially**, and the boundary is below the seam. See the note under this table. | +| 15 | `follow-cross-origin-redirect` (follows a 302 to a foreign Location) | **not killed at tier 2** — below the poll seam; Layer 1 owns it. Fixture 30 pins a different fault class above the seam. See the note under this table. | | 16 | `discard-live-id-at-or-below-served-id` (streaming lane orders live ids against the highest poll-served id) | 31 — and 31 alone: verified to pass all of 01–30, because every other straggler either arrives with nothing yet served (20) or is buffered pre-cut (01, 12, 19) | -**Row 15 is the family's one partial kill, and the reason is structural.** In -tier 2 the poll lane is a SEAM: the driver receives the fixture's scripted 302 -and hands the connector an already-formed redirect-refused verdict. The +**Row 15 is not killed at tier 2, and the reason is structural.** In tier 2 the +poll lane is a SEAM. The driver receives the fixture's scripted 302, reduces +the `Location` to its origin with `CanonicalOrigin`, and hands the connector a +`PollRedirectRefused` verdict carrying that origin and a generic cause. The connector never sees a `Location` header and never decides whether to follow -one, so the `follow-cross-origin-redirect` mutation lives **below** the seam -and no tier-2 harness can reach it. What fixture 30 does kill is the half above -the seam: a connector that mishandles the verdict — retrying it, classifying it -as anything but Terminal(`invalid_continuation`), or echoing more of the -`Location` than its origin — diverges on `finally` and fails. +one, so `follow-cross-origin-redirect` is not merely hard to observe here — it +is **unreachable**, and an unreachable mutant is not a partial kill. + +What fixture 30 does pin is a different fault class, above the seam: given a +`PollRedirectRefused` verdict, a connector must classify it as +Terminal(`invalid_continuation`) and must not retry it. Its `finally` makes +both fail loudly — the reason is asserted exactly, and `mintCount: 1` / +`connectCount: 1` / `timers: {}` / `socket: closed` leave no room for a retry, +a reconnect, or a lingering timer. + +**Redaction is not among them**, and an earlier revision of this note said it +was. The driver performs the redaction itself, before the connector runs: no +path or query text from the `Location` ever reaches the connector, so a +connector that echoed its entire input verbatim would pass fixture 30 +unchanged. Claiming it here would have been a kill that cannot fail. That proof +belongs to `TestRedirectRefusalExposesOnlyTheLocationOrigin`, which feeds a +secret-bearing cause and asserts the terminal's whole rendering and cause chain +never carry it, and to the Layer-1 adapter. An earlier revision of this row claimed a harness obligation to "bind the foreign origin to a sentinel listener whose any-request fails the scenario". diff --git a/conformance/event-feed/schema.json b/conformance/event-feed/schema.json index 430cad297..90b1fd24b 100644 --- a/conformance/event-feed/schema.json +++ b/conformance/event-feed/schema.json @@ -989,7 +989,7 @@ "status", "headers" ], - "description": "Redirect on a continuation (fixture 30): the seam suppresses automatic following; a cross-origin/downgraded Location is Terminal(invalid_continuation). At tier 2 the poll lane is a SEAM, so the driver forms the redirect-refused verdict and the connector never sees a Location header: this pins the half above the seam (mishandling the verdict — retrying it, misclassifying it, or echoing more of the Location than its origin), not redirect-following itself, which lives below the seam and is proven by the Layer-1 adapter's own 302 test. No sentinel-listener obligation is imposed on the harness: the foreign origin is unreachable by construction of the harness, so a silent sentinel would be a statement about the driver rather than about the connector. See the row-15 note in README.md.", + "description": "Redirect on a continuation (fixture 30): the seam suppresses automatic following; a cross-origin/downgraded Location is Terminal(invalid_continuation). At tier 2 the poll lane is a SEAM, so the driver forms the redirect-refused verdict and the connector never sees a Location header: this pins the fault class above the seam (mishandling the verdict — retrying it or misclassifying it), not redirect-following itself, which lives below the seam, is unreachable at tier 2, and is proven by the Layer-1 adapter's own 302 test. Redaction is not pinned here: the driver reduces the Location to its origin before the connector runs, so no path or query text reaches it. No sentinel-listener obligation is imposed on the harness: the foreign origin is unreachable by construction of the harness, so a silent sentinel would be a statement about the driver rather than about the connector. See the row-15 note in README.md.", "properties": { "status": { "const": 302 From 950591d8ec36148f06f047b1cdd715b9ecf4b340 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Tue, 18 Aug 2026 19:56:27 -0700 Subject: [PATCH 08/55] Conformance: a proof that has not landed is an obligation, not a proof MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Row 15 cited "the Layer-1 adapter's own 302 test" in the present tense, as a proof the repository already contains. It does not. `doc.go` lists the Layer-1 adapters over the generated CreateStreamTicket and PollEvents operations among the pieces still to land, AGENTS.md calls them pending, and no adapter file exists. The row's own closing line said "Tracked for G1b" — the sentence tracking the work and the sentence claiming it was done sat four lines apart. That is the same defect as the redaction claim removed in the previous commit, which is what makes it worth saying out loud rather than just fixing: the rewrite there was checked against the driver's code and came out right about tier 2, while every claim it made about ANOTHER layer went unverified. Verifying a document against the layer it lives in cannot catch a false claim about a layer it does not. So all four cross-layer proof claims — the fixture inventory's row 30, the kill matrix's row 15, the redaction paragraph's tail, and the zero-egress paragraph — now state an assigned obligation with its pending status, and the one test that does exist is marked as existing. The distinction is load-bearing for a reader deciding what this suite has established. Swept for residual present-tense Layer-1 proof claims across the family: none remain. --- conformance/event-feed/README.md | 15 +++++++++------ conformance/event-feed/schema.json | 2 +- 2 files changed, 10 insertions(+), 7 deletions(-) diff --git a/conformance/event-feed/README.md b/conformance/event-feed/README.md index 0d5b2fd58..d465aa685 100644 --- a/conformance/event-feed/README.md +++ b/conformance/event-feed/README.md @@ -267,7 +267,7 @@ revoked-mint threshold). | 27 | `27-hostile-resume-cross-origin.json` | accepted 410 with a cross-origin `resume` → Terminal(`invalid_continuation`), zero foreign requests | | 28 | `28-checkpoint-load-failure.json` | store load Failed → Terminal(`checkpoint_load`) with ZERO wire attempts; distinct from Missing (which proceeds to a present entry) | | 29 | `29-checkpoint-save-failure-continues.json` | save Failed → feed continues and a SUBSEQUENT save is attempted (exact store-call script: no save circuit breaker) | -| 30 | `30-continuation-redirect-cross-origin.json` | validated same-origin `next` answering 302 + cross-origin Location → Terminal(`invalid_continuation`); zero foreign egress holds by construction of the seam and is proven at Layer 1, not here — see the row-15 note | +| 30 | `30-continuation-redirect-cross-origin.json` | validated same-origin `next` answering 302 + cross-origin Location → Terminal(`invalid_continuation`); zero foreign egress holds by construction of the seam here, and proving it against a real redirect is ASSIGNED to Layer 1, whose adapters are still pending (G1b) — see the row-15 note | | 31 | `31-post-snapshot-straggler-below-served-id.json` | post-snapshot straggler with an id BELOW the entry page's served id delivered live; the re-push of that served id still suppressed | | 34 | `34-filter-changed-409-reenters-at-the-present.json` | 409 with both digests → `Observer.filterConflict` (digests pinned) before `Observer.positionRejected(filter_changed)`; the held position is discarded and the walk re-enters at `since=now` (present-class, no poll-served id) | @@ -381,7 +381,7 @@ exactly the class of claim this family exists to check. | 12 | `bypass-configured-handler` (handler registered but skipped; default-terminal applied) | 24, 25 (via `handlerInvocations` exact-set) | | 13 | `follow-cross-origin-continuation` (skips §8 validation, polls the hostile URL) | 26, 27 | | 14 | `collapse-load-error-to-missing` | 28 | -| 15 | `follow-cross-origin-redirect` (follows a 302 to a foreign Location) | **not killed at tier 2** — below the poll seam; Layer 1 owns it. Fixture 30 pins a different fault class above the seam. See the note under this table. | +| 15 | `follow-cross-origin-redirect` (follows a 302 to a foreign Location) | **not killed at tier 2** — below the poll seam; assigned to Layer 1, whose adapters are still pending (G1b). Fixture 30 pins a different fault class above the seam. See the note under this table. | | 16 | `discard-live-id-at-or-below-served-id` (streaming lane orders live ids against the highest poll-served id) | 31 — and 31 alone: verified to pass all of 01–30, because every other straggler either arrives with nothing yet served (20) or is buffered pre-cut (01, 12, 19) | **Row 15 is not killed at tier 2, and the reason is structural.** In tier 2 the @@ -406,7 +406,8 @@ connector that echoed its entire input verbatim would pass fixture 30 unchanged. Claiming it here would have been a kill that cannot fail. That proof belongs to `TestRedirectRefusalExposesOnlyTheLocationOrigin`, which feeds a secret-bearing cause and asserts the terminal's whole rendering and cause chain -never carry it, and to the Layer-1 adapter. +never carry it — a test that exists today — and, for the real-adapter path, to +Layer 1 once its adapters land. An earlier revision of this row claimed a harness obligation to "bind the foreign origin to a sentinel listener whose any-request fails the scenario". @@ -414,9 +415,11 @@ That is withdrawn. No implementation met it, and meeting it would prove nothing: the foreign origin is unreachable **by construction of the harness**, because the harness is the seam, so a silent sentinel is a statement about the driver rather than about the connector. Zero egress to a foreign redirect -target is a Layer-1 property, and its proof is the Layer-1 seam adapter's own -302 test, where a real generated `PollEvents` call meets a real redirect. -Tracked for G1b. +target is a Layer-1 property, and proving it is ASSIGNED to the Layer-1 seam +adapter's own 302 test, where a real generated `PollEvents` call will meet a +real redirect. Those adapters have not landed — `go/pkg/basecamp/eventfeed/doc.go` +lists them among the pieces still to come — so this is a recorded obligation, +not a proof the repository contains today. Tracked for G1b. Auto-continue-past-unhandled-gap needs no separate mutation — fixture 23's exact-set `finally` is its direct test. Fixture 29's exact store-call script is the diff --git a/conformance/event-feed/schema.json b/conformance/event-feed/schema.json index 90b1fd24b..8241f8cb4 100644 --- a/conformance/event-feed/schema.json +++ b/conformance/event-feed/schema.json @@ -989,7 +989,7 @@ "status", "headers" ], - "description": "Redirect on a continuation (fixture 30): the seam suppresses automatic following; a cross-origin/downgraded Location is Terminal(invalid_continuation). At tier 2 the poll lane is a SEAM, so the driver forms the redirect-refused verdict and the connector never sees a Location header: this pins the fault class above the seam (mishandling the verdict — retrying it or misclassifying it), not redirect-following itself, which lives below the seam, is unreachable at tier 2, and is proven by the Layer-1 adapter's own 302 test. Redaction is not pinned here: the driver reduces the Location to its origin before the connector runs, so no path or query text reaches it. No sentinel-listener obligation is imposed on the harness: the foreign origin is unreachable by construction of the harness, so a silent sentinel would be a statement about the driver rather than about the connector. See the row-15 note in README.md.", + "description": "Redirect on a continuation (fixture 30): the seam suppresses automatic following; a cross-origin/downgraded Location is Terminal(invalid_continuation). At tier 2 the poll lane is a SEAM, so the driver forms the redirect-refused verdict and the connector never sees a Location header: this pins the fault class above the seam (mishandling the verdict — retrying it or misclassifying it), not redirect-following itself, which lives below the seam, is unreachable at tier 2, and is ASSIGNED to the Layer-1 adapter's 302 test — a recorded obligation, not an existing proof: those adapters are still pending (tracked for G1b). Redaction is not pinned here: the driver reduces the Location to its origin before the connector runs, so no path or query text reaches it. No sentinel-listener obligation is imposed on the harness: the foreign origin is unreachable by construction of the harness, so a silent sentinel would be a statement about the driver rather than about the connector. See the row-15 note in README.md.", "properties": { "status": { "const": 302 From de50ac2732590511a746d4592bdc297492e44c55 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Tue, 18 Aug 2026 20:03:04 -0700 Subject: [PATCH 09/55] Conformance: the zero-egress obligation was scoped to what a harness can see MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The withdrawn sentinel obligation had a third home, in different words. The placeholder section told every harness to "assert those hosts receive zero requests" for any literal foreign origin — which is exactly the obligation row 15 withdraws for fixture 30, phrased without the word sentinel or egress, which is why two term-based sweeps walked past it. The rule is right for the fixtures it was written for and wrong only where it was over-extended. In 26 and 27 the connector itself holds the hostile URL as a continuation target: it could poll it, no expect step serves it, and zero requests is a real structural property of the connector. In 30 the foreign origin reaches the connector only through a Location the driver has already reduced to an origin, so there is no egress for a harness to observe and asserting its absence describes the driver. So the obligation is now scoped to connector-visible continuation targets, with the seam-converted case named and pointed at the row-15 note, rather than the blanket "any literal foreign origin" that contradicted its own withdrawal four hundred lines later. Found by sweeping the concept instead of the vocabulary: the literal hostile origin plus every phrasing of a harness obligation, which enumerates five sites and shows the other four are correct as written. --- conformance/event-feed/README.md | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/conformance/event-feed/README.md b/conformance/event-feed/README.md index d465aa685..7f5d98a78 100644 --- a/conformance/event-feed/README.md +++ b/conformance/event-feed/README.md @@ -111,8 +111,16 @@ teeth): Literal origins (e.g. `https://attacker.example.com`) are intentional and must **not** be substituted — the hostile-continuation fixtures depend on them staying -foreign, and the harness must assert those hosts receive **zero** requests -(structurally guaranteed: no expect step ever serves them). +foreign, and **where the connector itself holds such a URL as a continuation +target** (fixtures 26 and 27) the harness must assert those hosts receive +**zero** requests (structurally guaranteed: no expect step ever serves them). + +That obligation is deliberately scoped to connector-visible targets. Where a +foreign origin reaches the connector only through a value the seam has already +converted — fixture 30's redirect `Location`, which the driver reduces to an +origin before the connector sees anything — there is no egress for a harness to +observe, and asserting its absence would be a statement about the driver rather +than about the connector. See the row-15 note under the mutation kill matrix. ## Count semantics: seam calls, never wire attempts From 524376da4b5f54c0de789c3cde828e056a9aa81c Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Tue, 18 Aug 2026 23:07:40 -0700 Subject: [PATCH 10/55] Conformance: the spec still said the two things this branch stopped doing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The branch changed the README and the schema and never touched SPEC §23, so each of them now contradicted it outright. Both are the spec's side of a call already made and already enforced elsewhere. **Reentrant scheduling.** §23 listed it among the semantics every language's test clock must pass before its tier-2 results count, while the schema's `$defs.advance` requires every driver to HARD-FAIL an advance during which the connector arms a timer. The distinction the spec was missing is that the clause is normative for the ALGORITHM and forbidden as a FIXTURE DEPENDENCY: a clock that ignored it would fire the wrong set, but whether a timer armed during the window lands inside it depends on when the connector's goroutine got scheduled, which no fixture can pin. The checklist keeps deadline order and the creation-order tie-break; the reentrant clause moves to its own paragraph with the driver obligation, including the detection rule — count timer CREATIONS, never compare the outstanding set, because a firing removes its timer before any observer runs and a same-name rearm leaves the set identical. **Fixture 30's zero egress.** §23 required "zero requests to the foreign origin" as tier-2 coverage. Tier 2 cannot deliver it, and the README's row-15 note already says why: the poll lane IS the seam, so the driver reduces the `Location` to its origin and hands the connector a refusal verdict. The connector never sees a `Location`, so the foreign origin is unreachable by construction of the harness — a harness asserting nothing reached it would be asserting something about itself. The requirement becomes what tier 2 can actually hold (terminal reason, no retry, no further poll), and the egress proof is named as the Layer-1 obligation it is, pending G1b. **The published memory ceiling was short by one frame.** It read (pump depth + LIVE_BUFFER_CAPACITY) × MAX_FRAME_BYTES. The pump is a single reader, so it can hold one frame it has already READ and is blocked handing off, over and above the queue's depth — which is precisely why the drain's protocol-fatal scan is budgeted at pumpDepth+1 rather than pumpDepth, and that comment has been in the tree explaining the same +1 since #760. It is one rather than unbounded for the same reason: one reader holds at most one frame outside the queue. The three code comments restating the formula are corrected with it. --- SPEC.md | 47 +++++++++++++++++++---- go/pkg/basecamp/eventfeed/buffer_test.go | 2 +- go/pkg/basecamp/eventfeed/catchup.go | 2 +- go/pkg/basecamp/eventfeed/catchup_test.go | 2 +- 4 files changed, 42 insertions(+), 11 deletions(-) diff --git a/SPEC.md b/SPEC.md index 081981bb6..865ec7857 100644 --- a/SPEC.md +++ b/SPEC.md @@ -3618,10 +3618,16 @@ Two dispatch clarifications, pinned: overflow signal is the only drop signal. Worst-case connector memory is therefore bounded multiplicatively — every queued or buffered item is itself bounded by `EVENT_FEED_MAX_FRAME_BYTES`, so the ceiling is - (pump depth + `EVENT_FEED_LIVE_BUFFER_CAPACITY`) × `EVENT_FEED_MAX_FRAME_BYTES` + (pump depth + 1 + `EVENT_FEED_LIVE_BUFFER_CAPACITY`) × `EVENT_FEED_MAX_FRAME_BYTES` (≈ 10 GiB at the defaults' extreme, reached only if every slot holds a maximum-size - frame) — even under a slow consumer. Implementations MAY additionally impose a total - byte cap on the live buffer; if they do, eviction routes through the same overflow + frame) — even under a slow consumer. The **+ 1** is the pump's own in-flight frame: the + pump is a single reader, so it may hold exactly one frame it has already READ and is + blocked handing off, over and above the queue's depth. It is one frame rather than an + unbounded number for the same reason — one reader can hold no more than one frame + outside the queue — and it is the same quantity the drain's protocol-fatal scan is + budgeted at (`pump depth + 1`), because "every frame the pump had already read" is one + more than "every frame sitting in the queue". Implementations MAY additionally impose a + total byte cap on the live buffer; if they do, eviction routes through the same overflow signal, never a silent drop. - The transport negotiates subprotocol `actioncable-v1-json`, sends no `Origin` header (non-browser clients), and passes the mint URL through untouched, query string included. @@ -3991,9 +3997,21 @@ logged (Security Invariants below). Required tier-2 coverage: a hostile cross-origin `next` mid-walk, a hostile 410 `resume` URL, and a validated same-origin `next` answering 302 with a cross-origin -`Location` each terminate with `invalid_continuation` and zero requests to the foreign -origin; store-failure coverage proves Failed(load) terminates with zero wire attempts and -Failed(save) continues with the observer signal and a subsequent save attempt. +`Location` each terminate with `invalid_continuation`, are not retried, and issue no +further poll; store-failure coverage proves Failed(load) terminates with zero wire +attempts and Failed(save) continues with the observer signal and a subsequent save +attempt. + +**Zero egress to the foreign origin is a Layer-1 obligation, not tier-2 coverage**, and +this paragraph used to require it here. Tier 2 cannot deliver it: the poll lane IS the +seam, so the driver reduces the `Location` to its origin and hands the connector a +refusal verdict. The connector never sees a `Location` and never decides whether to +follow one, which makes the foreign origin unreachable by construction of the harness — +a harness that asserted no request reached it would be asserting something about itself. +The obligation belongs to the Layer-1 seam adapter's own 302 test, where a real +generated `PollEvents` call meets a real redirect against an adapter with automatic +redirect-following disabled. `conformance/event-feed/README.md`'s row-15 note records it +as a pending obligation rather than a proof the repository contains. ### Clock, Timers, and Virtual Time `[conformance]` @@ -4048,8 +4066,21 @@ the advance whose deadlines land inside the window also fire; ties break by crea order.* A harness may additionally fire a named timer without advancing the clock, asserting its scheduled delay against a `{min, max}` envelope — that is how jitter is asserted without a cross-language RNG seam. Each language's test clock passes a shared -semantics checklist (deadline order, reentrant scheduling within an advance, creation-order -tie-break) before its tier-2 results count. +semantics checklist (deadline order, creation-order tie-break) before its tier-2 results +count. + +**The reentrant clause is normative for the algorithm and forbidden as a fixture +dependency.** It stays in the algorithm because a clock that ignored it would fire the +wrong set. But it is UNSCRIPTABLE wherever the connector runs concurrently with the +driver: whether a timer armed during the window lands inside it depends on when the +connector's goroutine, thread, or task got scheduled, which no fixture can pin. So **no +fixture may rely on it, and every driver MUST fail an advance during which the connector +arms any timer**, naming `fireTimer` as the deterministic alternative. Detect the ARMING +itself — a monotonic count of timer creations — never a change in the outstanding-timer +set: a firing removes its timer before any observer runs, and a same-name rearm leaves the +set identical, so a set comparison reports nothing in exactly the case that matters. This +is what `conformance/event-feed/schema.json`'s `$defs.advance` states, and the driver +obligation is enforced there. Teardown discipline: disposing a connection attempt — deadline lapse, staleness, socket death, terminal — cancels the frame pump, **cancels any in-flight seam call belonging to diff --git a/go/pkg/basecamp/eventfeed/buffer_test.go b/go/pkg/basecamp/eventfeed/buffer_test.go index 62fc0d179..7aaa250e5 100644 --- a/go/pkg/basecamp/eventfeed/buffer_test.go +++ b/go/pkg/basecamp/eventfeed/buffer_test.go @@ -7,7 +7,7 @@ import ( // TestLiveBufferAddClearsEvictedSlots pins the eviction half of the live // buffer's memory ceiling. SPEC.md §23 publishes the connector's worst case -// as (pump depth + EVENT_FEED_LIVE_BUFFER_CAPACITY) × EVENT_FEED_MAX_FRAME_BYTES; +// as (pump depth + 1 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × EVENT_FEED_MAX_FRAME_BYTES; // a reslice alone removes the evicted event LOGICALLY while the slice that // results still points into the same backing array, whose prefix keeps that // event's strings reachable until a later reallocation. Under sustained diff --git a/go/pkg/basecamp/eventfeed/catchup.go b/go/pkg/basecamp/eventfeed/catchup.go index 1d4565333..6133f488b 100644 --- a/go/pkg/basecamp/eventfeed/catchup.go +++ b/go/pkg/basecamp/eventfeed/catchup.go @@ -1117,7 +1117,7 @@ func (l *loop) probeFatal(at *attempt) (cycleOutcome, bool) { // // This is why the scan needs no queue and no share of pumpDepth: it retains // exactly what the single slot always retained. The connector's published -// memory bound — (pump depth + liveBufferCapacity) × MAX_FRAME_BYTES — is +// memory bound — (pump depth + 1 + liveBufferCapacity) × MAX_FRAME_BYTES — is // untouched, and so is the depth at which the pump blocks. func (l *loop) deferForDrain(d *deferredFrame) { if l.deferred == nil { diff --git a/go/pkg/basecamp/eventfeed/catchup_test.go b/go/pkg/basecamp/eventfeed/catchup_test.go index 789b40fee..bb7ba8fbf 100644 --- a/go/pkg/basecamp/eventfeed/catchup_test.go +++ b/go/pkg/basecamp/eventfeed/catchup_test.go @@ -1396,7 +1396,7 @@ func TestDrainScanAdmissionIsNotStranded(t *testing.T) { // TestDrainHoldsNoMoreThanTheLiveBufferCapacity: the live buffer's capacity // is a bound on events HELD AT ONCE — SPEC §23 sizes the connector's whole -// memory ceiling off it, "(pump depth + EVENT_FEED_LIVE_BUFFER_CAPACITY) × +// memory ceiling off it, "(pump depth + 1 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × // EVENT_FEED_MAX_FRAME_BYTES" — so a drain must not be able to hold a batch // outside the buffer while the buffer refills to capacity behind it. // From 9f8567d29744161f21d04e09a99298304edb1143 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Wed, 19 Aug 2026 12:55:45 -0700 Subject: [PATCH 11/55] Conformance: the corrected ceiling was still one frame short MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Correcting pump depth to pump depth + 1 named the pump's in-flight frame and stopped there. There are two frames the queue's depth does not count, and daybreak's finding is that they are held at the SAME TIME rather than being alternatives: the pump's held frame, and the deferred socket outcome parked in the single slot the in-flight-poll servicing and the drain's scan share. The slot is retained while the queue behind it refills, so the stable bound is pump depth + 2 + live capacity. The scan's budget stays at pump depth + 1, and the SPEC now says why that is not an inconsistency: the budget counts what the scan may DEQUEUE — the queue plus the pump's held frame — while the ceiling counts what may be RETAINED, and the deferral slot is retained without being dequeued by that scan. Both numbers are correct for their own question, which is exactly the confusion a bare "+1/+2" mismatch would otherwise invite. The advance guard also stops reading as a proof. It waits out the family's wall-clock watchdog for an arm and treats silence as "never armed", which is a heuristic and cannot be otherwise from outside — the connector arms on its own goroutine, and the only alternative is the rendezvous a scheduling-dependent fixture would need, which is the thing the guard refuses. The doc now names the bound and the failure direction: a missed arm lets such a fixture through, and it then fails non-deterministically rather than silently meaning different things in different languages. --- SPEC.md | 23 ++++++++++++------- go/pkg/basecamp/eventfeed/buffer_test.go | 2 +- go/pkg/basecamp/eventfeed/catchup.go | 7 ++++-- go/pkg/basecamp/eventfeed/catchup_test.go | 2 +- .../eventfeed/scenario_conformance_test.go | 12 ++++++++++ 5 files changed, 34 insertions(+), 12 deletions(-) diff --git a/SPEC.md b/SPEC.md index 865ec7857..cd050d2a6 100644 --- a/SPEC.md +++ b/SPEC.md @@ -3618,15 +3618,22 @@ Two dispatch clarifications, pinned: overflow signal is the only drop signal. Worst-case connector memory is therefore bounded multiplicatively — every queued or buffered item is itself bounded by `EVENT_FEED_MAX_FRAME_BYTES`, so the ceiling is - (pump depth + 1 + `EVENT_FEED_LIVE_BUFFER_CAPACITY`) × `EVENT_FEED_MAX_FRAME_BYTES` + (pump depth + 2 + `EVENT_FEED_LIVE_BUFFER_CAPACITY`) × `EVENT_FEED_MAX_FRAME_BYTES` (≈ 10 GiB at the defaults' extreme, reached only if every slot holds a maximum-size - frame) — even under a slow consumer. The **+ 1** is the pump's own in-flight frame: the - pump is a single reader, so it may hold exactly one frame it has already READ and is - blocked handing off, over and above the queue's depth. It is one frame rather than an - unbounded number for the same reason — one reader can hold no more than one frame - outside the queue — and it is the same quantity the drain's protocol-fatal scan is - budgeted at (`pump depth + 1`), because "every frame the pump had already read" is one - more than "every frame sitting in the queue". Implementations MAY additionally impose a + frame) — even under a slow consumer. The **+ 2** is two raw frames the queue's depth does + not count, and they are retained by different parties at the same time: + - the **pump's own in-flight frame** — the pump is a single reader, so it may hold exactly + one frame it has already READ and not yet handed off. One rather than an unbounded + number for that reason: one reader holds at most one frame outside the queue. + - the **deferred socket outcome** — the single slot the in-flight-poll servicing and the + drain's scan park one receive in. It is retained while the queue behind it refills, so it + is concurrent with a full queue and with the pump's held frame, not an alternative to + either. + + The drain's protocol-fatal scan is budgeted at `pump depth + 1` and not at this figure, + which is not an inconsistency: the budget counts what the scan may DEQUEUE — the queue plus + the pump's held frame — while the ceiling counts what may be RETAINED, and the deferral slot + is retained without being dequeued by that scan. Implementations MAY additionally impose a total byte cap on the live buffer; if they do, eviction routes through the same overflow signal, never a silent drop. - The transport negotiates subprotocol `actioncable-v1-json`, sends no `Origin` header diff --git a/go/pkg/basecamp/eventfeed/buffer_test.go b/go/pkg/basecamp/eventfeed/buffer_test.go index 7aaa250e5..3e4a1e619 100644 --- a/go/pkg/basecamp/eventfeed/buffer_test.go +++ b/go/pkg/basecamp/eventfeed/buffer_test.go @@ -7,7 +7,7 @@ import ( // TestLiveBufferAddClearsEvictedSlots pins the eviction half of the live // buffer's memory ceiling. SPEC.md §23 publishes the connector's worst case -// as (pump depth + 1 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × EVENT_FEED_MAX_FRAME_BYTES; +// as (pump depth + 2 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × EVENT_FEED_MAX_FRAME_BYTES; // a reslice alone removes the evicted event LOGICALLY while the slice that // results still points into the same backing array, whose prefix keeps that // event's strings reachable until a later reallocation. Under sustained diff --git a/go/pkg/basecamp/eventfeed/catchup.go b/go/pkg/basecamp/eventfeed/catchup.go index 6133f488b..4da771a5d 100644 --- a/go/pkg/basecamp/eventfeed/catchup.go +++ b/go/pkg/basecamp/eventfeed/catchup.go @@ -1117,8 +1117,11 @@ func (l *loop) probeFatal(at *attempt) (cycleOutcome, bool) { // // This is why the scan needs no queue and no share of pumpDepth: it retains // exactly what the single slot always retained. The connector's published -// memory bound — (pump depth + 1 + liveBufferCapacity) × MAX_FRAME_BYTES — is -// untouched, and so is the depth at which the pump blocks. +// memory bound — (pump depth + 2 + liveBufferCapacity) × MAX_FRAME_BYTES — is +// untouched, and so is the depth at which the pump blocks. The slot IS one of +// that formula's two raw-frame terms beyond the queue; the other is the frame +// the pump has read and not yet handed off. Both are retained WHILE the queue +// is full, which is why they are addends and not alternatives. func (l *loop) deferForDrain(d *deferredFrame) { if l.deferred == nil { l.deferred = d diff --git a/go/pkg/basecamp/eventfeed/catchup_test.go b/go/pkg/basecamp/eventfeed/catchup_test.go index bb7ba8fbf..2014a8ac3 100644 --- a/go/pkg/basecamp/eventfeed/catchup_test.go +++ b/go/pkg/basecamp/eventfeed/catchup_test.go @@ -1396,7 +1396,7 @@ func TestDrainScanAdmissionIsNotStranded(t *testing.T) { // TestDrainHoldsNoMoreThanTheLiveBufferCapacity: the live buffer's capacity // is a bound on events HELD AT ONCE — SPEC §23 sizes the connector's whole -// memory ceiling off it, "(pump depth + 1 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × +// memory ceiling off it, "(pump depth + 2 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × // EVENT_FEED_MAX_FRAME_BYTES" — so a drain must not be able to hold a batch // outside the buffer while the buffer refills to capacity behind it. // diff --git a/go/pkg/basecamp/eventfeed/scenario_conformance_test.go b/go/pkg/basecamp/eventfeed/scenario_conformance_test.go index 40268e518..3e01085f5 100644 --- a/go/pkg/basecamp/eventfeed/scenario_conformance_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_conformance_test.go @@ -642,6 +642,18 @@ func (d *driver) nextClientFrame(what string) (clientFrame, error) { // The wait is bounded by the family's one wall-clock knob and ends the instant // an arm appears, so only a legitimately quiet advance pays it in full — and // the suite contains exactly one such advance (fixture 05). +// +// That bound makes this a HEURISTIC, and it is worth saying so rather than +// letting the guard read as a proof. "No arm within the watchdog" is treated +// as "no arm", so an arm landing later is missed. It cannot be otherwise from +// outside: the connector arms on its own goroutine with no rendezvous the +// driver can take, and the only alternative — holding the advance until the +// connector says it is quiet — is the rendezvous a scheduling-dependent +// fixture would need in the first place, which is the thing being refused. +// The failure direction is the safe one: a missed arm lets a +// scheduling-dependent fixture through, where the fixture then fails +// non-deterministically instead of silently meaning different things in +// different languages. func (d *driver) advance(step *advanceStep) error { before := d.h.clock.ArmCount() d.h.clock.Advance(millis(step.Ms)) From 636338f2da0e740dd2669ae2c1e4bae067832ccc Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Wed, 19 Aug 2026 12:58:38 -0700 Subject: [PATCH 12/55] Conformance: a prose "Tracked in #N" was a promise nothing verified MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The known-defect gate verifies that every issue the bc3 route allowlist leans on is still OPEN, because #588 auto-closed while nine live 404s pointed at it and "is an Integer" was the entire test. This branch and its siblings now make the same class of claim in PROSE — the README says the arrival-strict gap is "Tracked in #792", and awaitSupersededPoll's doc leans on #758 for the overshoot — and nothing checked those at all. Same failure, same remedy, same script: the issue closes, the sentence keeps promising someone owns the gap, and no one is told. spec/tracking-issues.yml registers the promises and the existing gate verifies them alongside the allowlist's, reusing its fail-closed behavior and its "name the site, not just the number" reporting unchanged. A registry rather than a scan, and the file says why: `#\d+` over prose cannot separate a tracking promise from an as-of citation — "shipped in #12380", "the fix in #760" — and telling them apart needs the author, not a matcher. That is the same distinction AGENTS.md draws for bc3 revisions in prose, for the same reason. Two things the extension had to fix in the test rig rather than add to it. The self-test never set the registry path, so the checker would have read the repository's real one mid-test, making hermetic cases depend on live issue state and the network — precisely what the gh stub exists to remove. And the summary line restated its case count by hand, so it said "9 cases" while running twelve; it now counts what ran. Verified red against a genuinely closed issue (#784) before wiring, not only against the stub. --- scripts/check-known-defect-issues-open | 21 ++++++- scripts/test-check-known-defect-issues-open | 68 ++++++++++++++++++++- spec/tracking-issues.yml | 26 ++++++++ 3 files changed, 109 insertions(+), 6 deletions(-) create mode 100644 spec/tracking-issues.yml diff --git a/scripts/check-known-defect-issues-open b/scripts/check-known-defect-issues-open index 29f84fcff..36cc6a0f5 100755 --- a/scripts/check-known-defect-issues-open +++ b/scripts/check-known-defect-issues-open @@ -9,6 +9,13 @@ # bc3_routes_not_modeled[].tracking_issue # "an OPEN issue owns absorbing it" # +# spec/tracking-issues.yml discharges the same kind of claim made in PROSE — +# "Tracked in #N" in a README or a code comment. Same failure, same remedy: a +# closed issue tracks nothing, and a sentence saying otherwise is a lie nobody +# is told about. It is a registry rather than a scan of the prose because +# "#\d+" cannot tell a tracking promise from an as-of citation ("shipped in +# #12380"), and sorting those needs the author, not a matcher. +# # check-bc3-route-parity already requires those to be numeric. A number is not a # tracker: #588 auto-closed while nine live 404s still pointed at it, and that # gate stayed green the whole time because "is an Integer" was the entire test. @@ -56,6 +63,9 @@ unless File.exist?(allow_path) end allow = YAML.safe_load(File.read(allow_path)) || {} +tracking_path = ENV['TRACKING_ISSUES'] || File.join(ROOT, 'spec', 'tracking-issues.yml') +tracking = File.exist?(tracking_path) ? (YAML.safe_load(File.read(tracking_path)) || {}) : {} + # (issue number, human-readable site) pairs, so a failure names the entry that # has to change rather than just the number. refs = [] @@ -68,8 +78,13 @@ end refs << [e['tracking_issue'], "bc3_routes_not_modeled #{e['method']} #{e['path']}"] end +(tracking['prose_tracking_issues'] || []).each do |e| + next unless e['issue'].is_a?(Integer) + refs << [e['issue'], e['site'].to_s.empty? ? "prose_tracking_issues ##{e['issue']}" : e['site']] +end + if refs.empty? - puts ' ✓ bc3 route allowlist references no tracking issues — nothing to verify' + puts ' ✓ no tracking issues referenced — nothing to verify' exit 0 end @@ -98,8 +113,8 @@ refs.each do |number, site| next if state.nil? || state == 'open' failures << "#{site} points at ##{number}, which is #{state.upcase}. " \ 'A closed issue tracks nothing: reopen it, repoint the entry at the ' \ - 'issue that now owns the defect, or remove the entry because the ' \ - 'route is fixed.' + 'issue that now owns the defect, or remove the entry because what it ' \ + 'tracked is fixed — and delete the claim that leaned on it.' end unless failures.empty? diff --git a/scripts/test-check-known-defect-issues-open b/scripts/test-check-known-defect-issues-open index 58ffdcf9a..32d8e7de0 100755 --- a/scripts/test-check-known-defect-issues-open +++ b/scripts/test-check-known-defect-issues-open @@ -89,7 +89,7 @@ failures = [] # all, when gh is meant to be absent), so the real gh can never answer. The # interpreter is invoked by absolute path for the same reason. Returns # [combined_output, status, gh_log_lines]. -def run_checker(allowlist:, states: nil, gh_fail: false, repo: nil, github_repository: nil) +def run_checker(allowlist:, tracking: {}, states: nil, gh_fail: false, repo: nil, github_repository: nil) Dir.mktmpdir("known-defect-selftest") do |dir| path_dir = File.join(dir, "bin") Dir.mkdir(path_dir) @@ -104,9 +104,17 @@ def run_checker(allowlist:, states: nil, gh_fail: false, repo: nil, github_repos allow_path = File.join(dir, "bc3-route-allowlist.yml") File.write(allow_path, YAML.dump(allowlist)) + # Pointed at a temp file even when the case supplies nothing, so the + # checker can never read the repository's REAL registry mid-self-test — + # which would make these cases depend on live issue state and on the + # network, the two things the stub exists to remove. + tracking_path = File.join(dir, "tracking-issues.yml") + File.write(tracking_path, YAML.dump(tracking)) + env = { "PATH" => path_dir, "BC3_ROUTE_ALLOWLIST" => allow_path, + "TRACKING_ISSUES" => tracking_path, "GH_STUB_LOG" => gh_log, "GH_STUB_STATES" => states, "GH_STUB_FAIL" => (gh_fail ? "1" : nil), @@ -120,7 +128,13 @@ def run_checker(allowlist:, states: nil, gh_fail: false, repo: nil, github_repos end end +# CASES counts what actually ran. The summary used to restate the number by +# hand, and it was wrong the moment a case was added — the same stale-constant +# failure the gates in this directory exist to catch, in the gate's own tests. +CASES = [ 0 ] + def expect_pass(failures, label, out, status, fragment = nil) + CASES[0] += 1 if !status.success? puts " FAIL #{label}" failures << "#{label}: expected PASS but checker exited #{status.exitstatus}:\n#{out}" @@ -133,6 +147,7 @@ def expect_pass(failures, label, out, status, fragment = nil) end def expect_fail(failures, label, out, status, fragment) + CASES[0] += 1 if status.success? puts " FAIL #{label}" failures << "#{label}: expected FAILURE but checker passed:\n#{out}" @@ -161,7 +176,7 @@ puts "==> known-defect tracking-issue self-test (checker: #{CHECKER.sub("#{ROOT} out, status, calls = run_checker(allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }) expect_pass(failures, "1. empty allowlist is a no-op", out, status, - "references no tracking issues") + "no tracking issues referenced") unless calls.empty? puts " FAIL 1a. empty allowlist makes no API calls" failures << "1a: expected no gh calls, got #{calls.inspect}" @@ -281,8 +296,55 @@ expect_fail(failures, "9. missing gh fails closed", out, status, # --- Report ------------------------------------------------------------------------ +# --- N. Prose tracking claims are collected from the registry -------------------- +# +# "Tracked in #N" in a README is the same claim the allowlist makes in +# structured form, and it fails the same way: the issue closes, the sentence +# keeps promising someone owns the gap, and nothing says otherwise. The registry +# exists because prose cannot be scanned for this — "#\d+" cannot separate a +# tracking promise from an as-of citation — so the author registers the promise +# and the gate verifies it alongside the allowlist's. + +out, status, calls = run_checker( + allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, + tracking: { "prose_tracking_issues" => [ { "issue" => 303, "site" => "README.md — a gap" } ] }, + states: "303=open" +) +expect_pass(failures, "N. prose registry entries are collected", out, status, + "README.md — a gap") +unless calls.any? { |c| c.end_with?("/issues/303") } + puts " FAIL Na. prose entry is looked up" + failures << "Na: expected a lookup of #303, got #{calls.inspect}" +end + +# --- N+1. A closed prose tracking issue fails the gate -------------------------- +# +# The whole point. A registry that reported a closed issue as fine would be the +# #588 failure with extra steps. + +out, status, _ = run_checker( + allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, + tracking: { "prose_tracking_issues" => [ { "issue" => 404, "site" => "SPEC.md — a bound" } ] }, + states: "404=closed" +) +expect_fail(failures, "N+1. a CLOSED prose tracking issue fails", out, status, + "SPEC.md — a bound points at #404, which is CLOSED") + +# --- N+2. Allowlist and registry are collected together ------------------------- +# +# Neither source may mask the other: a green allowlist must not make an unread +# registry look verified, which is what a single-source read would do. + +out, status, _ = run_checker( + allowlist: { "sdk_routes_known_defective" => [ defective(101) ], "bc3_routes_not_modeled" => [] }, + tracking: { "prose_tracking_issues" => [ { "issue" => 303, "site" => "README.md — a gap" } ] }, + states: "101=open,303=closed" +) +expect_fail(failures, "N+2. a closed registry entry fails even with a green allowlist", out, status, + "README.md — a gap points at #303, which is CLOSED") + if failures.empty? - puts "==> known-defect tracking-issue self-test passed — 9 cases" + puts "==> known-defect tracking-issue self-test passed — #{CASES[0]} cases" exit 0 else warn "known-defect tracking-issue self-test FAILED:" diff --git a/spec/tracking-issues.yml b/spec/tracking-issues.yml new file mode 100644 index 000000000..4ad57c4b6 --- /dev/null +++ b/spec/tracking-issues.yml @@ -0,0 +1,26 @@ +# Prose claims that discharge an obligation by pointing at an issue number. +# +# A sentence like "Tracked in #792" is a promise that someone owns the gap it +# describes. If that issue closes without the gap closing, the sentence becomes +# a documentation lie and nothing notices — which is exactly what happened to +# the bc3 route allowlist when #588 auto-closed with nine live 404s pointing at +# it (see scripts/check-known-defect-issues-open). +# +# The route allowlist gets that check because its entries are structured data. +# Prose is not, so the claims are registered here instead: one entry per +# sentence that leans on an issue, verified OPEN by the same gate, failing +# closed the same way. +# +# This is deliberately a REGISTRY and not a scanner. A regex over prose for +# "#\d+" would sweep up every historical citation — "shipped in #12380", "the +# fix in #760" — which are as-of facts that must never be reopened, and would +# need an ever-growing exclusion list to tell the two apart. The registry makes +# the author say which kind they meant, once, where the reason is visible. +# +# Add an entry when you write a sentence promising an issue owns something. +# Remove it when the sentence goes, and the gate stops asking. +prose_tracking_issues: + - issue: 792 + site: "conformance/event-feed/README.md — the Go driver's arrival-strict gap" + - issue: 758 + site: "go/pkg/basecamp/eventfeed/catchup.go awaitSupersededPoll — the superseded-poll bound's overshoot" From ed94105e0662fcf47ab4a2fd64d516125921c728 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Wed, 19 Aug 2026 15:27:05 -0700 Subject: [PATCH 13/55] Conformance: the registry gate was fail-open, in a gate that exists to fail closed MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A missing registry file became `{}`, so the checker reported "nothing to verify" and exited 0 with every prose claim in the repository unverified. Delete the file, or typo the override, and the gate went quiet — the #588 failure with one extra step, in the script written to end it. daybreak reproduced exit 0; the file is now required, and its absence is a failure with the same wording as an unreachable API. Rows are validated rather than skipped, and that differs from the allowlist on purpose. There, `next unless Integer` is right: an entry with no issue number is making no claim. Here the row IS the claim, so a malformed one is a promise the author wrote and the gate silently dropped — worse than either a failure or an absent row. A row now needs a positive integer `issue` and a non-empty `site`, because a bare number tells the next reader nothing to go fix. Four self-test cases cover it: a missing registry, a row with no site, a row with no issue, and a non-list registry. The runner also had to gain the ability NOT to write the file, which it previously always did — so the missing-registry case is expressed without deleting the repository's real one. --- scripts/check-known-defect-issues-open | 41 ++++++++++++++++++-- scripts/test-check-known-defect-issues-open | 43 ++++++++++++++++++++- 2 files changed, 79 insertions(+), 5 deletions(-) diff --git a/scripts/check-known-defect-issues-open b/scripts/check-known-defect-issues-open index 36cc6a0f5..0ee5a4e4d 100755 --- a/scripts/check-known-defect-issues-open +++ b/scripts/check-known-defect-issues-open @@ -63,8 +63,19 @@ unless File.exist?(allow_path) end allow = YAML.safe_load(File.read(allow_path)) || {} +# REQUIRED, not optional. Treating an absent registry as an empty one is a +# fail-OPEN path in a gate whose entire purpose is failing closed: delete the +# file (or typo the override) and the checker reports "nothing to verify" and +# exits 0, with every prose claim in the repository unverified. That is the +# #588 failure with an extra step, which is what this script exists to end. tracking_path = ENV['TRACKING_ISSUES'] || File.join(ROOT, 'spec', 'tracking-issues.yml') -tracking = File.exist?(tracking_path) ? (YAML.safe_load(File.read(tracking_path)) || {}) : {} +unless File.exist?(tracking_path) + warn "ERROR: tracking registry not found at #{tracking_path}" + warn ' This check fails closed by design: a missing registry is indistinguishable' + warn ' from one whose claims are all unverified. Restore it, or empty its list.' + exit 1 +end +tracking = YAML.safe_load(File.read(tracking_path)) || {} # (issue number, human-readable site) pairs, so a failure names the entry that # has to change rather than just the number. @@ -78,9 +89,31 @@ end refs << [e['tracking_issue'], "bc3_routes_not_modeled #{e['method']} #{e['path']}"] end -(tracking['prose_tracking_issues'] || []).each do |e| - next unless e['issue'].is_a?(Integer) - refs << [e['issue'], e['site'].to_s.empty? ? "prose_tracking_issues ##{e['issue']}" : e['site']] +# Every row is VALIDATED rather than skipped. `next unless Integer` is how the +# allowlist entries are read, and there it is tolerable because an entry with no +# issue number is making no claim. Here the row IS the claim: a malformed one is +# a promise the author wrote and the gate silently dropped, which is worse than +# either a failure or an absent row. +rows = tracking['prose_tracking_issues'] +unless rows.nil? || rows.is_a?(Array) + warn "ERROR: prose_tracking_issues in #{tracking_path} must be a list, got #{rows.class}" + exit 1 +end +(rows || []).each_with_index do |e, i| + unless e.is_a?(Hash) + warn "ERROR: prose_tracking_issues[#{i}] in #{tracking_path} must be a mapping, got #{e.class}" + exit 1 + end + unless e['issue'].is_a?(Integer) && e['issue'].positive? + warn "ERROR: prose_tracking_issues[#{i}] in #{tracking_path} needs a positive integer `issue`, got #{e['issue'].inspect}" + exit 1 + end + if e['site'].to_s.strip.empty? + warn "ERROR: prose_tracking_issues[#{i}] (##{e['issue']}) in #{tracking_path} needs a `site` naming " \ + 'the sentence that leans on it — a bare number tells the next reader nothing to go fix.' + exit 1 + end + refs << [e['issue'], e['site']] end if refs.empty? diff --git a/scripts/test-check-known-defect-issues-open b/scripts/test-check-known-defect-issues-open index 32d8e7de0..502e636cf 100755 --- a/scripts/test-check-known-defect-issues-open +++ b/scripts/test-check-known-defect-issues-open @@ -109,7 +109,9 @@ def run_checker(allowlist:, tracking: {}, states: nil, gh_fail: false, repo: nil # which would make these cases depend on live issue state and on the # network, the two things the stub exists to remove. tracking_path = File.join(dir, "tracking-issues.yml") - File.write(tracking_path, YAML.dump(tracking)) + # nil means "do not create the file", which is how the missing-registry + # case is expressed without deleting the repository's real one. + File.write(tracking_path, YAML.dump(tracking)) unless tracking.nil? env = { "PATH" => path_dir, @@ -343,6 +345,45 @@ out, status, _ = run_checker( expect_fail(failures, "N+2. a closed registry entry fails even with a green allowlist", out, status, "README.md — a gap points at #303, which is CLOSED") +# --- N+3. A MISSING registry fails closed --------------------------------------- +# +# The fail-open path this gate cannot have. Treating an absent file as an empty +# one means deleting it (or typoing the override) reports "nothing to verify" +# and exits 0, with every prose claim unverified — the #588 failure with an +# extra step, in the script written to end it. + +out, status, _ = run_checker( + allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, + tracking: nil +) +expect_fail(failures, "N+3. a missing registry fails closed", out, status, + "tracking registry not found") + +# --- N+4. Malformed rows fail rather than being skipped -------------------------- +# +# The allowlist skips entries without an issue number, and there that is right: +# such an entry makes no claim. Here the row IS the claim, so a malformed one is +# a promise the author wrote and the gate silently dropped. + +out, status, _ = run_checker( + allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, + tracking: { "prose_tracking_issues" => [ { "issue" => 303 } ] } +) +expect_fail(failures, "N+4. a row with no site fails", out, status, "needs a `site`") + +out, status, _ = run_checker( + allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, + tracking: { "prose_tracking_issues" => [ { "site" => "README.md — a gap" } ] } +) +expect_fail(failures, "N+5. a row with no issue number fails", out, status, + "needs a positive integer `issue`") + +out, status, _ = run_checker( + allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, + tracking: { "prose_tracking_issues" => "not a list" } +) +expect_fail(failures, "N+6. a non-list registry fails", out, status, "must be a list") + if failures.empty? puts "==> known-defect tracking-issue self-test passed — #{CASES[0]} cases" exit 0 From 5b4d17537ea245ab157bb8f222f989dc212efca8 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Wed, 19 Aug 2026 22:46:56 -0700 Subject: [PATCH 14/55] Conformance: the arrival-strict gap closed, so its registry entry goes with it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit #792 is implemented and closed, and the README sentence that leaned on it is gone. The registry entry has to go in the same breath: it names a site that no longer says anything, and the gate would have failed on the closed issue at the next run — which is the gate working, not the gate complaining. That coupling is the point of registering prose claims rather than scanning for them. A promise and its tracker are removed together or the gate objects. --- spec/tracking-issues.yml | 2 -- 1 file changed, 2 deletions(-) diff --git a/spec/tracking-issues.yml b/spec/tracking-issues.yml index 4ad57c4b6..ae5a6f2bf 100644 --- a/spec/tracking-issues.yml +++ b/spec/tracking-issues.yml @@ -20,7 +20,5 @@ # Add an entry when you write a sentence promising an issue owns something. # Remove it when the sentence goes, and the gate stops asking. prose_tracking_issues: - - issue: 792 - site: "conformance/event-feed/README.md — the Go driver's arrival-strict gap" - issue: 758 site: "go/pkg/basecamp/eventfeed/catchup.go awaitSupersededPoll — the superseded-poll bound's overshoot" From f97625ad8cfe102273d04d3933a59790d003b4d2 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Wed, 19 Aug 2026 22:52:29 -0700 Subject: [PATCH 15/55] Conformance: the advance guard was a heuristic wearing a MUST MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit It sampled the clock's arm count, advanced, waited out the family's wall-clock watchdog for the count to move, and failed if it did — so "no arm within five seconds" meant "no arm", and a later arm was simply missed. No wait makes that sound: there is no instant at which "nothing further will be armed" becomes knowable from outside a connector that arms on its own goroutine. daybreak was right to call it out; the previous round only wrote the weakness into a comment. The question changes instead. "Did the advance cause an arm?" is racy. "Can this advance fire anything at all?" is one atomic read of the clock, taken under the same lock the advance selects under, before time moves. The inversion is sound because of what an advance does when it fires nothing. A test clock holds its lock while selecting due timers and releases it ONLY across a firing's aftermath — deliberately, so a woken recipient can arm inside the window. An advance with nothing due never releases that lock, never wakes anything, and so cannot be the cause of any arm. There is nothing left to detect, which is why rejecting the shape is complete where detecting the divergence could only ever be probable. It is stricter than the arming rule it replaces: a firing that replaces nothing is now rejected too, where before it was expressly legal. That case had been the control distinguishing arm-counting from set-comparison, and it becomes a rejection — the third self-test arm now pins that. A script that wanted such a firing writes `fireTimer`, which fires one named timer without moving the clock and says which timer it means, so the loss is expressiveness nobody was using and legibility gained. Fixture 05 is the suite's only advance and it qualifies: Streaming, with staleness and repair-poll configured to ~11 days against a 121-second window, existing to age a ticket past its TTL rather than to fire anything. Verified before writing the rule rather than after. `ArmCount` and its clock test go with the guard that was its only caller — it was built for this and nothing else, and a test double keeping an API for a deleted mechanism invites the next reader to use it. README, schema and §23 move together, since the obligation binds five drivers still to be written. --- .triage.md | 92 +++++++++++++++++++ SPEC.md | 20 ++-- conformance/event-feed/README.md | 44 +++++---- conformance/event-feed/schema.json | 2 +- go/pkg/basecamp/eventfeed/feedtest/clock.go | 43 +++++---- .../basecamp/eventfeed/feedtest/clock_test.go | 64 ------------- .../eventfeed/scenario_conformance_test.go | 87 ++++++------------ .../eventfeed/scenario_selftest_test.go | 59 +++++++----- 8 files changed, 222 insertions(+), 189 deletions(-) create mode 100644 .triage.md diff --git a/.triage.md b/.triage.md new file mode 100644 index 000000000..cc11e84dc --- /dev/null +++ b/.triage.md @@ -0,0 +1,92 @@ +# PR #705 — review triage (workstream A) + +## Ledger, verified + +`reviewThreads(first:100).totalCount = **41**`, 41 fetched, `hasNextPage=false`. +29 resolved from waves 1–3, **12 unresolved**, every one single-comment and awaiting a reply. + +Three surfaces swept: +- **Review threads**: 41 (12 open). +- **Review bodies**: 40 reviews. Copilot's *suppressed-comment* blocks carry findings that never + became threads; the round-3/4 ones were already answered in issue comments. +- **Issue comments**: 3, all mine. The last one (2026-08-12T06:50Z) deliberately **held two + findings open** — the suspendable bound and the swallowed deferred outcome — and put shape + (a) patch-in-place vs (b) one-event-loop to a human. This round answers it. + +**Discrepancy with the briefed list of 12**: the briefed list named 10 findings; the 12 open threads +are those 10 **plus** `loop.go:454` (evicted live-buffer slots not cleared before reslicing), which +the brief omitted. `catchup.go:355` is two threads (Codex + Copilot) reporting one finding. +So: **11 distinct findings across 12 threads.** + +## The stopping rule, applied + +Two re-reports across four waves on one mechanism. Per AGENTS.md that is a signal about the +instrument, not a queue. Reading the code, the four rounds do **not** orbit the deferral slot +(which §23 sanctions — transition 21's deferred consumption, pinned by fixtures 01/19). They orbit +**one borrowed bound**: `awaitSupersededPoll` waits on the *staleness* verdict, and that verdict is +suspendable by a party other than the waiter. + +The asymmetry nobody had named: every other staleness wait in the package (`awaitConfirmation`, +`socketCheck`, `stream`, the poll-retry wait) **drains the frame queue while it waits**, so the +suspension rule's premise — "a full queue proves the connector is being outrun" — holds. +`awaitSupersededPoll` is the **only** wait that deliberately stops draining. Its premise is false by +construction, so it is the only place the suspension can be granted forever. + +That makes the remedy an obviation, not a fifth patch: **stop borrowing the bound.** The wait gets a +fixed deadline read from the injected `Clock` at the instant of deferral, non-resetting and +unsuspendable, and the staleness firing is demoted to a wake-up. Shape (b) (one event loop) is +declined: it dissolves the same class at the cost of restructuring `catchup.go`'s core and re-proving +the delivery/save ordering nine fixtures pin, for a class the deadline change already closes. + +Constraint that ruled out the obvious alternative: SPEC §23 pins **exactly six kebab-case timer +kinds** and per-state exact timer sets, asserted by the cross-SDK fixtures. Arming a dedicated timer +for this wait would be a seventh kind — a SPEC change across six SDKs for a Go-local bug. A deadline +value plus the existing wakes needs no new kind. + +## Triage table + +| # | Finding | Disposition | Failure mode named | +|---|---|---|---| +| 1 | `catchup.go:355` ×2 — staleness bound re-armed forever by a blocked pump | **BLOCKS** | **Hang.** Stalled `PollSource` + peer filling the 256-slot queue ⇒ every firing suspended-and-re-armed ⇒ consumer goroutine held indefinitely. Round-1's fix does not hold. | +| 2 | `catchup.go:556` — drain budget misses the pump's blocked hand-off | **BLOCKS** | **Wrong verdict + wrong save.** 256 queued + 1 held in `handOff` = 257 frames read at drain start; budget 256 ⇒ a fatal `invalid_event_stream_command` escapes the carve-out, the held position saves and `caught_up` announces. | +| 3 | `catchup.go:158` — deferred socket outcome lost on the poll-error path | **BLOCKS** | **Lost protocol-fatal verdict.** `recoverPoll`'s terminal branches call `disposeAttempt`, which clears `l.deferred`; an observed `invalid_event_stream_command` is replaced by `authorization_failed`/`poll_failed`. | +| 4 | `cable.go:180` — full-envelope unmarshal before the type switch | **BLOCKS** | **Reconnect loop on a server extension.** SPEC §23: "parseable JSON whose `type` the connector doesn't recognize — update liveness and are otherwise ignored". `{"type":"future","identifier":1}` currently tears the socket down; a server that keeps sending it loops every client forever. | +| 5 | `transport.go:51` — out-of-range port accepted by policy | **BLOCKS** | **Unbounded re-mint.** `wss://h:99999/` passes, dial fails `DialTransient` ⇒ re-mint/backoff forever on a permanently unusable URL — the exact class the function's own comment commits to catching. | +| 6 | `loop.go:932` — `frameReject` not gated on `AwaitingConfirmation` | **BLOCKS** | **Wrong permanent terminal.** Transition 12 exists only from `AwaitingConfirmation`; a premature `reject_subscription` yields `subscription_rejected` with ZERO reconnects and zero subscribe attempts. | +| 7 | `loop.go:901` — subscribe write is synchronous and unbounded | **BLOCKS** | **Defeated deadline.** A `CableConn.WriteFrame` that blocks past the handshake/confirmation deadline cannot be interrupted: `at.ctx` is only cancelled by a teardown the write is blocking. Feed hangs until `Connector.Close`. | +| 8 | `loop.go:908` — unchecked `deadline.Stop()` on welcome | **BLOCKS** | **Lapsed deadline not honored.** Both the frame and the expired timer ready ⇒ the `select` picks either; if the frame wins, a fired handshake deadline is replaced by a fresh confirmation timer and a late welcome is accepted instead of taking transition 9. Fix 7 widens this window, so they ship together. | +| 9 | `loop.go:454` — evicted buffer slots not cleared before reslicing | **BLOCKS** | **2× the published memory ceiling.** §23 publishes `(pumpDepth + capacity) × MAX_FRAME_BYTES`; the backing array keeps dropped payloads alive until a realloc. One-liner, and `shift` already does exactly this three lines below. | +| 10 | `loop.go:1110` — `Observer.Disconnected` missing on terminal teardowns | **FOLLOW-UP** | Real: `Observer.Disconnected` is documented "fires when a socket is torn down", unconditional, and `disposeAttempt` leaves four terminal paths silent. But the remedy changes the **normative observer sequence** that `conformance/event-feed/` fixtures pin across six SDKs (`expectDisconnected*` in `conformance/event-feed/schema.json`), so it is a §23 contract decision, not a Go-local bug. Issue filed. | +| 11 | `websocket_transport.go:256` — native `time.After` close budget | **DECLINE** | The failure does not occur. §23 "Clock, Timers and Virtual Time" scopes the rule to delays *the connector itself* takes and exempts delays **inside a seam call**; `CableConn.Close` is a seam call, and `WebSocketTransport` is a seam *implementation*. No deterministic scenario can reach it: every virtual-clock run substitutes `feedtest.Transport`, because a real socket is not deterministic either. A consumer holding a real socket is already in real time. | + +## Verification contract for the nine fixes + +Each gets a regression test proven RED against un-fixed code, with `REAL_EXIT` written to a log file +and grepped back. Mutated files restored by `cp` + `diff -q`, never `git checkout --`. + +--- + +# Outcome (appended after implementation; the table above is the pre-edit triage) + +The initial triage held, with two corrections the code forced: + +1. **`loop.go:1110` (Observer.Disconnected)** — filed as #753, as planned. +2. **`websocket_transport.go:256`** — declined, as planned. A LATER, different + finding on the same file (the close-budget timeout not unblocking reads) was + real and became #762. +3. **Everything marked BLOCKS was fixed and red-proven**, except that the + `loop.go:908` proof needed a mutation rather than a revert (its test needs a + hook the un-fixed code lacks). + +Six further bot review rounds arrived during the session, which the pre-edit +triage could not anticipate. Net: **13 fixes, 7 issues filed, 2 declines**, and +3 threads left open at handover with the reasoning posted publicly. + +The stopping-rule call in the header — "the four rounds orbit one borrowed +bound, not the deferral slot" — was **too strong, and I corrected it publicly**. +Removing the borrowing closed that half; Codex and Copilot then independently +found a second hang in the same wait from a different cause (no staleness case +at all in `pollPage`), and the obvious fix for it broke a pinned invariant +(`TestWalkFailureBetweenPages/staleness_expiry`). That became #758, with the +failing output attached — the most useful thing produced this session, because +it saves the next person from rediscovering it by breaking the same test. diff --git a/SPEC.md b/SPEC.md index cd050d2a6..ab98dda98 100644 --- a/SPEC.md +++ b/SPEC.md @@ -4081,13 +4081,19 @@ dependency.** It stays in the algorithm because a clock that ignored it would fi wrong set. But it is UNSCRIPTABLE wherever the connector runs concurrently with the driver: whether a timer armed during the window lands inside it depends on when the connector's goroutine, thread, or task got scheduled, which no fixture can pin. So **no -fixture may rely on it, and every driver MUST fail an advance during which the connector -arms any timer**, naming `fireTimer` as the deterministic alternative. Detect the ARMING -itself — a monotonic count of timer creations — never a change in the outstanding-timer -set: a firing removes its timer before any observer runs, and a same-name rearm leaves the -set identical, so a set comparison reports nothing in exactly the case that matters. This -is what `conformance/event-feed/schema.json`'s `$defs.advance` states, and the driver -obligation is enforced there. +fixture may rely on it, and every driver MUST REJECT an `advance` whose window would fire +any timer**, naming `fireTimer` as the deterministic alternative. + +The test is what would FIRE, decided from the clock's state before time moves — not what +gets ARMED. Arming happens on the connector's schedule, so a driver can only look for it +by waiting and then assuming nothing further is coming, which is a heuristic wearing a +MUST and passes a late arm in silence. Firing is one atomic read under the same lock the +advance selects under. The inversion is sound because a test clock releases that lock only +across a firing's aftermath — so an advance that fires nothing never wakes anything and +cannot cause an arm, leaving nothing to detect. It is stricter than an arming rule (a +firing that replaces nothing is rejected too) and that is the trade: a script wanting that +firing writes `fireTimer` and names the timer. `conformance/event-feed/schema.json`'s +`$defs.advance` states it, and the driver obligation is enforced there. Teardown discipline: disposing a connection attempt — deadline lapse, staleness, socket death, terminal — cancels the frame pump, **cancels any in-flight seam call belonging to diff --git a/conformance/event-feed/README.md b/conformance/event-feed/README.md index 7f5d98a78..081bb0e41 100644 --- a/conformance/event-feed/README.md +++ b/conformance/event-feed/README.md @@ -154,24 +154,32 @@ requirement that a staleness window closing during a delivery is latched and observed later, and waiting for the follow-on ARMING requires knowing one is coming, which nothing can tell you. -Every driver must therefore FAIL an `advance` during which the connector arms -anything, naming `fireTimer` as the deterministic alternative — it fires one -named timer without moving the clock, so no re-selection is involved. This is -unconditional, not a per-fixture opt-in: a flag would let a fixture author take -the divergence instead of avoiding it. - -**Detect the arming itself — a monotonic count of timer creations on the test -clock — and never a change in the outstanding-timer set.** The set is the wrong -instrument in both directions, and a driver using it reports the opposite of -what this rule says on both: a firing removes its timer from the set before any -observer can run, so an ordinary expiry that arms nothing is read as an arm; -and a timer rearmed under a name it already had leaves the set byte-identical, -so the connector's own same-name rearms (`repair-poll`, `poll-retry`) are -invisible. Only the creation count separates the two histories. The Go driver -implements this and self-tests three arms: the rejection, an ordinary -quiet-window advance still passing, and — the control that discriminates the -instruments — an advance whose window fires a due timer that is *not* replaced, -which must pass. +**So an `advance` whose window would fire ANY timer is rejected**, and the +driver names `fireTimer` as the alternative — it fires one named timer without +moving the clock, so no re-selection is involved. This is unconditional, not a +per-fixture opt-in: a flag would let a fixture author take the divergence +instead of avoiding it. + +The rule asks what an advance would FIRE, not what it arms, and the difference +is the whole reason it is enforceable. Arming happens on the connector's +schedule, so a driver can only look for it by waiting and then guessing that +nothing more is coming — a heuristic wearing a MUST, which silently passes a +late arm. Firing is decided by the clock's own state before time moves: one +atomic read, under the same lock the advance selects under, answers it +completely. + +That inversion is sound because of what an advance does when it fires nothing. +A test clock holds its lock while selecting due timers and releases it only +across a firing's aftermath — deliberately, so a woken recipient can arm inside +the window. An advance with nothing due therefore never releases the lock, never +wakes anything, and cannot be the cause of any arm. There is nothing left to +detect. + +It is stricter than an arming rule, and deliberately: a firing that replaces +nothing is rejected too. A script that wants that firing writes `fireTimer` and +says which timer it means, which is more legible anyway. The Go driver +self-tests three arms — the rejection, an ordinary quiet-window advance still +passing, and a firing that arms nothing being rejected all the same. ## Contract notes the fixtures encode (SDK-owned, final) diff --git a/conformance/event-feed/schema.json b/conformance/event-feed/schema.json index 8241f8cb4..d8b93f298 100644 --- a/conformance/event-feed/schema.json +++ b/conformance/event-feed/schema.json @@ -1322,7 +1322,7 @@ "required": [ "ms" ], - "description": "Advance virtual now by ms, firing due timers in deadline order per the normative virtual-advance algorithm (README): re-evaluate after each fire; timers scheduled during the advance whose deadlines land inside the window also fire; ties break by creation order. That reentrant clause is UNSCRIPTABLE wherever the connector runs concurrently with the driver, so no fixture may rely on it: every driver MUST fail an advance during which the connector arms any timer, naming fireTimer as the deterministic alternative. Detect arming itself (a monotonic count of timer creations), never a change in the outstanding-timer set — a firing removes its timer before any observer runs, and a same-name rearm leaves the set identical.", + "description": "Advance virtual now by ms. The normative virtual-advance algorithm (README) fires due timers in deadline order, re-evaluating after each fire, with timers scheduled during the advance whose deadlines land inside the window also firing and ties breaking by creation order. That reentrant clause is UNSCRIPTABLE wherever the connector runs concurrently with the driver, so no fixture may rely on it: every driver MUST REJECT an advance whose window would fire ANY timer, naming fireTimer as the deterministic alternative. The test is what would FIRE, decided from the clock's state before time moves — not what gets armed, which happens on the connector's schedule and can only be sampled. An advance that fires nothing never wakes anything, so it cannot cause an arm; that is what makes the check complete rather than probabilistic. A firing that replaces nothing is rejected too.", "properties": { "ms": { "type": "integer", diff --git a/go/pkg/basecamp/eventfeed/feedtest/clock.go b/go/pkg/basecamp/eventfeed/feedtest/clock.go index eec78b11c..0d51cfe90 100644 --- a/go/pkg/basecamp/eventfeed/feedtest/clock.go +++ b/go/pkg/basecamp/eventfeed/feedtest/clock.go @@ -66,6 +66,30 @@ func (c *Clock) NewTimer(d time.Duration, name string) eventfeed.Timer { return t } +// DueWithin returns the names of live timers due within d of the current +// virtual time — exactly the set an Advance(d) would fire — in creation order. +// +// Read under the same lock advance selects under and NewTimer arms under, so +// the answer is atomic with respect to both. That is what lets a caller turn a +// racy question into a decidable one: an EMPTY result means the advance fires +// nothing, and advance never unlocks unless it fires something, so no +// recipient can be woken by it and no timer it could arm can land inside its +// window. A non-empty result means the script is asking for a firing whose +// aftermath races the re-selection, which is the thing no cross-language +// fixture can mean the same way twice. +func (c *Clock) DueWithin(d time.Duration) []string { + c.mu.Lock() + defer c.mu.Unlock() + target := c.now.Add(d) + var names []string + for _, t := range c.live { + if !t.deadline.After(target) { + names = append(names, t.name) + } + } + return names +} + // Outstanding returns the names of live (unfired, unstopped) timers, in // creation order. func (c *Clock) Outstanding() []string { @@ -78,25 +102,6 @@ func (c *Clock) Outstanding() []string { return names } -// ArmCount returns how many timers have been armed on this clock since it was -// created. It only ever rises: firing a timer, stopping one, or arming a -// replacement under a name that already existed all leave it alone or raise -// it, never lower it. -// -// That monotonicity is the whole point, because Outstanding() cannot answer -// "did the connector arm anything". Outstanding() reports the live set, and -// two different histories collapse onto the same set: a timer that fired and -// was rearmed under its own name is indistinguishable from one that never -// moved, while a timer that merely expired changes the set without anything -// having been armed at all. A caller comparing snapshots of the set therefore -// reads expiries as arms and misses same-name rearms entirely. ArmCount -// counts the events themselves. -func (c *Clock) ArmCount() int { - c.mu.Lock() - defer c.mu.Unlock() - return c.seq -} - // Advance moves virtual time forward by d, firing due timers in deadline // order (ties by creation order), re-evaluating the registry after each fire // so a timer armed mid-advance with a deadline inside the window also fires. diff --git a/go/pkg/basecamp/eventfeed/feedtest/clock_test.go b/go/pkg/basecamp/eventfeed/feedtest/clock_test.go index e44a1fd9c..5df388698 100644 --- a/go/pkg/basecamp/eventfeed/feedtest/clock_test.go +++ b/go/pkg/basecamp/eventfeed/feedtest/clock_test.go @@ -1,7 +1,6 @@ package feedtest import ( - "maps" "sync" "testing" "time" @@ -210,66 +209,3 @@ func TestClock_AwaitTimerRendezvousesWithAnotherGoroutine(t *testing.T) { t.Fatal("AwaitTimer(backoff) did not observe the armed timer") } } - -// TestClock_ArmCountCountsArmsNotOutstandingChanges pins ArmCount against the -// two histories an Outstanding() snapshot cannot tell apart. Both halves are -// the reason the tier-2 advance guard reads ArmCount instead of comparing -// timer sets, so both are asserted here against a set comparison directly. -func TestClock_ArmCountCountsArmsNotOutstandingChanges(t *testing.T) { - t.Run("an expiry changes the set without arming anything", func(t *testing.T) { - c := NewClock() - c.NewTimer(5*time.Millisecond, "backoff") - before, beforeSet := c.ArmCount(), counts(c.Outstanding()) - - c.Advance(10 * time.Millisecond) - - if got := c.ArmCount(); got != before { - t.Errorf("ArmCount() = %d after an expiry, want %d: firing a timer is not arming one", got, before) - } - // The foil: the set DID change, which is what a snapshot comparison - // would have read as an arm. - if afterSet := counts(c.Outstanding()); maps.Equal(afterSet, beforeSet) { - t.Fatalf("outstanding set unchanged (%v) — this case no longer distinguishes ArmCount from a set comparison", afterSet) - } - }) - - t.Run("a same-name rearm arms without changing the set", func(t *testing.T) { - c := NewClock() - c.NewTimer(5*time.Millisecond, "repair-poll") - before, beforeSet := c.ArmCount(), counts(c.Outstanding()) - - // Exactly what the connector does at a repair-poll rearm: the firing - // removes the timer, and a replacement goes back under the same name. - c.Advance(10 * time.Millisecond) - c.NewTimer(5*time.Millisecond, "repair-poll") - - if got := c.ArmCount(); got != before+1 { - t.Errorf("ArmCount() = %d after a same-name rearm, want %d", got, before+1) - } - // The foil: the set is byte-identical across the rearm, so a snapshot - // comparison sees nothing at all. - if afterSet := counts(c.Outstanding()); !maps.Equal(afterSet, beforeSet) { - t.Fatalf("outstanding set changed (%v -> %v) — this case no longer exercises the blind spot", beforeSet, afterSet) - } - }) - - t.Run("a stop never lowers it", func(t *testing.T) { - c := NewClock() - timer := c.NewTimer(time.Second, "staleness") - before := c.ArmCount() - - timer.Stop() - - if got := c.ArmCount(); got != before { - t.Errorf("ArmCount() = %d after Stop, want %d: ArmCount only ever rises", got, before) - } - }) -} - -func counts(names []string) map[string]int { - out := map[string]int{} - for _, name := range names { - out[name]++ - } - return out -} diff --git a/go/pkg/basecamp/eventfeed/scenario_conformance_test.go b/go/pkg/basecamp/eventfeed/scenario_conformance_test.go index 3e01085f5..1ce507533 100644 --- a/go/pkg/basecamp/eventfeed/scenario_conformance_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_conformance_test.go @@ -604,71 +604,44 @@ func (d *driver) nextClientFrame(what string) (clientFrame, error) { // // The algorithm says timers armed during a window whose deadlines land inside // it also fire. In a single-threaded test clock that is exact. In Go the -// connector arms on its own goroutine, so whether the re-selection sees the -// follow-on timer is a scheduling question — the same fixture could fire it in -// TypeScript and not in Go, which is the divergence, not a flake. +// connector arms on its own goroutine, so whether such a timer lands before +// the re-selection that would fire it is a scheduling outcome — the same +// script means two things, and a fixture cannot pin which. // -// It is not fixable by settling, and that is worth writing down because the -// obvious fixes both fail: +// # Rejecting the shape, rather than detecting the divergence // -// - Waiting for the firing to be CONSUMED hangs. The connector deliberately -// does not consume a staleness firing promptly — a window that closes while -// the consumer is inside a delivery is latched and observed later, by -// design (staleHolder). A clock that waited would deadlock against the very -// behavior §23 requires. -// - Waiting for the follow-on ARMING requires knowing one is coming. Nothing -// can distinguish "has not armed yet" from "will not arm", so any such wait -// is either a guess or the same hang. +// This used to detect: sample the clock's arm count, advance, wait out the +// family's wall-clock watchdog for the count to move, and fail if it did. That +// is a heuristic wearing a MUST, and it reads "no arm within five seconds" as +// "no arm" — an arm landing later is simply missed. No wait makes it sound, +// because there is no instant at which "nothing further will be armed" becomes +// knowable from outside. // -// So the divergence is made impossible to script instead of impossible to hit: -// an advance during which the connector arms anything is failed, loudly, with -// fireTimer named as the deterministic alternative (it fires one named timer -// without moving the clock, so no re-selection is involved). This is checked on -// EVERY advance rather than behind a schema opt-in — a field would let the next -// fixture author take the divergence rather than avoid it, which is precisely -// the thing being prevented. +// So the question changes from "did the advance cause an arm?", which is racy, +// to "can this advance fire anything at all?", which is decidable. feedtest's +// clock selects due timers under its own lock and unlocks ONLY across a +// firing's aftermath — deliberately, so a woken recipient can arm inside the +// window. An advance with nothing due therefore never unlocks, never wakes +// anything, and cannot be the cause of any arm. One atomic read of the clock +// settles it before time moves. // -// What is measured is ARMING, via the clock's monotonic ArmCount, and not the -// outstanding-timer set. The set is the wrong instrument in both directions: -// a firing removes its timer from the set before any observer runs, so an -// ordinary expiry that arms nothing looks exactly like an arm; and a timer -// rearmed under a name it already had leaves the set identical, so the one -// case §23 most needs caught — the connector's same-name rearms, e.g. -// repair-poll — is the case a set comparison cannot see. +// A script that wants a firing writes `fireTimer`, which fires one named timer +// without advancing the clock and so involves no re-selection at all. The +// suite's only `advance` (fixture 05) sits in Streaming with staleness and +// repair-poll configured to ~11 days against a 121-second window: it exists to +// age a ticket past its TTL, not to fire anything, and it qualifies. // -// The wait after Advance is the other half. The connector arms on its own -// goroutine, so an arm caused by a firing inside the window can land just -// after Advance returns; sampling immediately would make the guard a race. -// The wait is bounded by the family's one wall-clock knob and ends the instant -// an arm appears, so only a legitimately quiet advance pays it in full — and -// the suite contains exactly one such advance (fixture 05). -// -// That bound makes this a HEURISTIC, and it is worth saying so rather than -// letting the guard read as a proof. "No arm within the watchdog" is treated -// as "no arm", so an arm landing later is missed. It cannot be otherwise from -// outside: the connector arms on its own goroutine with no rendezvous the -// driver can take, and the only alternative — holding the advance until the -// connector says it is quiet — is the rendezvous a scheduling-dependent -// fixture would need in the first place, which is the thing being refused. -// The failure direction is the safe one: a missed arm lets a -// scheduling-dependent fixture through, where the fixture then fails -// non-deterministically instead of silently meaning different things in -// different languages. +// AdvanceSettling remains for a caller that genuinely wants a chained firing +// with an explicit rendezvous. It is deliberately not reachable from a fixture. func (d *driver) advance(step *advanceStep) error { - before := d.h.clock.ArmCount() - d.h.clock.Advance(millis(step.Ms)) - - deadline := time.Now().Add(scenarioWatchdog) - for d.h.clock.ArmCount() == before && time.Now().Before(deadline) { - time.Sleep(time.Millisecond) - } - if armed := d.h.clock.ArmCount() - before; armed > 0 { + if due := d.h.clock.DueWithin(millis(step.Ms)); len(due) > 0 { return fmt.Errorf( - "advance of %dms armed %d timer(s) (outstanding now %v): a timer armed inside an advance window "+ - "fires or not depending on goroutine scheduling, so this script cannot mean the same thing in every "+ - "language — use fireTimer, which fires one named timer without re-selecting", - step.Ms, armed, timerCounts(d.h.clock)) + "advance of %dms would fire %v: whether a timer armed by one of those firings lands inside the "+ + "same window depends on goroutine scheduling, so this script cannot mean the same thing in "+ + "every language — use fireTimer, which fires one named timer without re-selecting", + step.Ms, due) } + d.h.clock.Advance(millis(step.Ms)) return nil } diff --git a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go index 95793420a..00053a20d 100644 --- a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go @@ -516,16 +516,23 @@ func TestScenarioDriverRejectsUnmatchedActions(t *testing.T) { // result that differs between languages for the same fixture, which is worse // than a failure because nothing reports it. // -// The control matters as much as the mutant: an advance over a window in which -// the connector arms nothing is ordinary and must still pass, or the guard -// would be rejecting every advance and the suite's one real advance (fixture -// 05) would be failing for the wrong reason. +// The rule is about what an advance would FIRE, not about what it arms, and +// the third case below is where those differ: a firing that replaces nothing +// is still rejected. That is stricter than the arming rule this replaced, and +// deliberately so — the arming rule could only be enforced by sampling, and a +// sampled MUST is not one. +// +// The control matters as much as the mutants: an advance over a window with +// nothing due is ordinary and must still pass, or the guard would be rejecting +// every advance and the suite's one real advance (fixture 05) would be failing +// for the wrong reason. func TestScenarioDriverRejectsSchedulingDependentAdvance(t *testing.T) { t.Run("an advance during which the connector arms a timer", func(t *testing.T) { - // Advancing past the handshake deadline makes the connector tear the - // attempt down and arm `backoff` — a timer armed INSIDE the window, - // which is exactly the reentrant clause the algorithm cannot resolve - // identically across languages when the recipient is another goroutine. + // Advancing past the handshake deadline fires it, and the teardown it + // causes arms `backoff` inside the same window — the reentrant clause + // the algorithm cannot resolve identically across languages when the + // recipient is another goroutine. The guard never has to observe that + // arming: the firing alone is enough to reject the script. script := `{"name":"x","description":"d","steps":[ {"expectMint":{"respond":{"status":200,"body":{"ticket":"{{TICKET:1}}","expires_in":120,"url":"{{CABLE_URL:1}}"}}}}, {"expectConnect":{"url":"{{CABLE_URL:1}}"}}, @@ -535,7 +542,7 @@ func TestScenarioDriverRejectsSchedulingDependentAdvance(t *testing.T) { if err == nil { t.Fatal("an advance that changes the outstanding timer set must fail the scenario") } - if !strings.Contains(err.Error(), "armed 1 timer(s)") { + if !strings.Contains(err.Error(), "would fire") { t.Fatalf("failed for the wrong reason: %v", err) } if !strings.Contains(err.Error(), "fireTimer") { @@ -544,7 +551,7 @@ func TestScenarioDriverRejectsSchedulingDependentAdvance(t *testing.T) { }) t.Run("an advance over a quiet window is ordinary", func(t *testing.T) { - // No connection yet, so nothing is armed and nothing can be: the + // No connection yet, so nothing is armed and nothing is due: the // guard must not reject an advance merely for existing. script := `{"name":"x","description":"d","steps":[ {"advance":{"ms":1000}}, @@ -556,14 +563,15 @@ func TestScenarioDriverRejectsSchedulingDependentAdvance(t *testing.T) { } }) - // The control that separates "arms anything" from "the timer set moved". - // Here a due timer FIRES and is not replaced: the backoff deadline expires - // and the connector's next act is a mint, which parks inside the seam - // until the driver releases it, so no timer is armed anywhere in the - // window. The rule says reject ARMING, so this advance is legal and must - // pass — and it is the case a driver comparing outstanding-timer snapshots - // gets wrong, because the fired timer's own removal moves the set. - t.Run("an advance in which a due timer fires without replacement", func(t *testing.T) { + // A firing that replaces nothing is STILL rejected, and this is the case + // that shows the rule changed rather than merely being reimplemented. + // Here the backoff deadline expires and the connector's next act is a mint, + // which parks inside the seam until the driver releases it, so nothing is + // armed anywhere in the window. Under the arming rule this was legal. It is + // not any more, because "did anything get armed?" can only be answered by + // waiting and hoping, while "is anything due?" is one atomic read — and the + // script that wanted this has `fireTimer`, which says which timer it means. + t.Run("an advance in which a due timer fires without replacement is still rejected", func(t *testing.T) { script := `{"name":"x","description":"d","steps":[ {"expectMint":{"respond":{"status":200,"body":{"ticket":"{{TICKET:1}}","expires_in":120,"url":"{{CABLE_URL:1}}"}}}}, {"expectConnect":{"url":"{{CABLE_URL:1}}"}}, @@ -576,17 +584,22 @@ func TestScenarioDriverRejectsSchedulingDependentAdvance(t *testing.T) { {"expectMint":{"respond":{"status":200,"body":{"ticket":"{{TICKET:2}}","expires_in":120,"url":"{{CABLE_URL:2}}"}}}}, {"expectConnect":{"url":"{{CABLE_URL:2}}"}}], "finally":{"state":"awaiting_welcome"}}` - if err := underShortWatchdog(func() error { return runScenarioBytes([]byte(script), "x.json") }); err != nil { - t.Fatalf("an advance whose window fires a timer but arms nothing must pass: %v", err) + err := underShortWatchdog(func() error { return runScenarioBytes([]byte(script), "x.json") }) + if err == nil { + t.Fatal("an advance whose window fires a timer must fail, even when it replaces nothing") + } + if !strings.Contains(err.Error(), "would fire") { + t.Fatalf("failed for the wrong reason: %v", err) + } + if !strings.Contains(err.Error(), "fireTimer") { + t.Errorf("the rejection must name the deterministic alternative: %v", err) } }) } // underShortWatchdog runs a scenario under a short rendezvous window. Its // usual use is a scenario EXPECTED to fail, but it serves any case whose waits -// are all short by construction — including the advance guard's passing cases, -// where the guard must wait out the window to conclude that nothing armed. -// A hostile scenario often fails by never satisfying a +// are all short by construction. A hostile scenario often fails by never satisfying a // rendezvous, and waiting the full window for each would cost more than the // whole conformance suite; every caller still pins the failure's reason, so a // mutant rejected for the wrong reason cannot pass as the pin firing. From b2b1110a193441b09103b0b2b1cfee42a4c05d78 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Wed, 19 Aug 2026 22:56:55 -0700 Subject: [PATCH 16/55] Conformance: both tracked promises came due, so the registry empties MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit #758 and #792 are implemented, and the two sentences that leaned on them are gone with them — `awaitSupersededPoll`'s overshoot paragraph replaced by the published bound, and the README's arrival-strict gap note removed. Their entries go in the same breath, which is the coupling the registry exists to enforce: a promise and its tracker are removed together, or the gate objects the moment the issue closes. The file stays, with an empty list. It is a habit rather than a one-off — the next sentence that promises an issue owns something registers here, and the first self-test case already pins that an empty registry is a legitimate no-op rather than a missing one. --- spec/tracking-issues.yml | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/spec/tracking-issues.yml b/spec/tracking-issues.yml index ae5a6f2bf..437bd5787 100644 --- a/spec/tracking-issues.yml +++ b/spec/tracking-issues.yml @@ -19,6 +19,4 @@ # # Add an entry when you write a sentence promising an issue owns something. # Remove it when the sentence goes, and the gate stops asking. -prose_tracking_issues: - - issue: 758 - site: "go/pkg/basecamp/eventfeed/catchup.go awaitSupersededPoll — the superseded-poll bound's overshoot" +prose_tracking_issues: [] From 0e6ebf2c5bf9c541c04cd957d10b38f75b9956e5 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Fri, 21 Aug 2026 23:18:30 -0700 Subject: [PATCH 17/55] Take five review findings on the tier-2 corrections, and decline one MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Triage of the final Copilot round on this branch — two open threads and four suppressed findings, five taken, one declined. * Registry gate: the KEY is required, not just the file. A registry whose `prose_tracking_issues` key was misspelled or dropped read as nil and passed as empty — the same fail-open path as a deleted file, one typo later, in a gate that exists to fail closed. Red-proven: the new self-test case (N+7) was run against the unfixed checker first and failed with the checker exiting 0 on "no tracking issues referenced"; after the fix, all 17 cases pass. * Registry seeded with the repository's live prose promises. The file shipped empty while `Makefile` (#589), SPEC §6 and the TypeScript Retry-After test (#775), and the Ruby/Python todolists guards (#578) all carry present-tense "tracked in #N" sentences — so the gate exited before its first lookup. Sweeping for the class also found the failure the gate was built to catch, already live: four sentences saying "tracked in #576" with #576 CLOSED (its MergeSafe guards shipped). Those four are rewritten as as-of facts; the checker now verifies all five registered claims and reports #589, #775 and #578 open. * feedtest.Clock gains AdvanceIfQuiet: DueWithin and Advance as one critical section, and the driver's advance directive uses it. With two lock acquisitions, a timer armed between the check and the movement turned an accepted advance into a firing one — the divergence the MUST exists to reject — and SPEC §23 already described the decision as one atomic read. Mutation-checked: with AdvanceIfQuiet split back into the two-section shape, both eventfeed packages stay green (REAL_EXIT=0) — no existing test can reach the race, deterministically or otherwise (the suite's only advance, fixture 05, holds nothing but ~11-day timers), so the guarantee rests on the method's single-critical-section construction and its semantics self-tests rather than on a manufactured stress test. What stays undecidable is stated in the method's doc: whether a concurrent arm lands before or after the critical section is still the arming goroutine's schedule; the restored invariant is that an ACCEPTED advance provably fires nothing. * Family README's clock checklist aligned with SPEC §23: "reentrant scheduling within an advance" leaves the tier-2 gate, since the advance-rejection rule makes it unreachable from any fixture. The clause stays normative for the algorithm, and the README now says both halves in one place instead of contradicting SPEC. * The memory ceiling is restated as an enumeration by holder, which closes the count: hand-off queue ≤ pump depth, live buffer ≤ capacity, deferral slot ≤ 1, and one in-hand frame per frame-touching goroutine — the pump's read-not-yet-handed-off frame AND the protocol-fatal scan's dequeued-not-yet-disposed frame, whose dequeue is the very receive that lets a blocked pump refill the queue behind it. The published bound becomes (pump depth + 3 + capacity) × MAX_FRAME_BYTES, mirrored in the Go comments and tests; the scan's pump depth + 1 dequeue budget is a different figure and is untouched. * .triage.md, a point-in-time review ledger for PR #705, was committed by accident and is removed. Declined: the suppressed SPEC:3231 finding (zero-egress withdrawal said to over-reach fixtures 26/27) — the withdrawal is scoped correctly in every operative artifact: the preceding sentence retains "issue no further poll" for the hostile next/resume cases, README rows 26/27 keep their structural zero-request claims, and mutation row 13 stays killed by 26+27; what the paragraph withdraws is only the wire-level sentinel obligation, which tier 2 cannot deliver for any of the three cases. Gates: tracking-issues self-test 17/17; event-feed-fixtures-check green (LC_ALL=C.UTF-8); doc-constants-check green under LC_ALL=C; both eventfeed Go packages green, including under -race. --- .triage.md | 92 ------------------- SPEC.md | 27 ++++-- conformance/event-feed/README.md | 6 +- go/pkg/basecamp/eventfeed/buffer_test.go | 2 +- go/pkg/basecamp/eventfeed/catchup.go | 10 +- go/pkg/basecamp/eventfeed/catchup_test.go | 2 +- go/pkg/basecamp/eventfeed/feedtest/clock.go | 38 +++++++- .../basecamp/eventfeed/feedtest/clock_test.go | 58 ++++++++++++ .../eventfeed/scenario_conformance_test.go | 9 +- python/src/basecamp/services/todolists.py | 6 +- .../tests/services/test_todolists_service.py | 3 +- .../basecamp/services/todolists_extensions.rb | 7 +- .../services/todolists_service_test.rb | 3 +- scripts/check-known-defect-issues-open | 12 ++- scripts/test-check-known-defect-issues-open | 19 +++- spec/tracking-issues.yml | 12 ++- 16 files changed, 182 insertions(+), 124 deletions(-) delete mode 100644 .triage.md diff --git a/.triage.md b/.triage.md deleted file mode 100644 index cc11e84dc..000000000 --- a/.triage.md +++ /dev/null @@ -1,92 +0,0 @@ -# PR #705 — review triage (workstream A) - -## Ledger, verified - -`reviewThreads(first:100).totalCount = **41**`, 41 fetched, `hasNextPage=false`. -29 resolved from waves 1–3, **12 unresolved**, every one single-comment and awaiting a reply. - -Three surfaces swept: -- **Review threads**: 41 (12 open). -- **Review bodies**: 40 reviews. Copilot's *suppressed-comment* blocks carry findings that never - became threads; the round-3/4 ones were already answered in issue comments. -- **Issue comments**: 3, all mine. The last one (2026-08-12T06:50Z) deliberately **held two - findings open** — the suspendable bound and the swallowed deferred outcome — and put shape - (a) patch-in-place vs (b) one-event-loop to a human. This round answers it. - -**Discrepancy with the briefed list of 12**: the briefed list named 10 findings; the 12 open threads -are those 10 **plus** `loop.go:454` (evicted live-buffer slots not cleared before reslicing), which -the brief omitted. `catchup.go:355` is two threads (Codex + Copilot) reporting one finding. -So: **11 distinct findings across 12 threads.** - -## The stopping rule, applied - -Two re-reports across four waves on one mechanism. Per AGENTS.md that is a signal about the -instrument, not a queue. Reading the code, the four rounds do **not** orbit the deferral slot -(which §23 sanctions — transition 21's deferred consumption, pinned by fixtures 01/19). They orbit -**one borrowed bound**: `awaitSupersededPoll` waits on the *staleness* verdict, and that verdict is -suspendable by a party other than the waiter. - -The asymmetry nobody had named: every other staleness wait in the package (`awaitConfirmation`, -`socketCheck`, `stream`, the poll-retry wait) **drains the frame queue while it waits**, so the -suspension rule's premise — "a full queue proves the connector is being outrun" — holds. -`awaitSupersededPoll` is the **only** wait that deliberately stops draining. Its premise is false by -construction, so it is the only place the suspension can be granted forever. - -That makes the remedy an obviation, not a fifth patch: **stop borrowing the bound.** The wait gets a -fixed deadline read from the injected `Clock` at the instant of deferral, non-resetting and -unsuspendable, and the staleness firing is demoted to a wake-up. Shape (b) (one event loop) is -declined: it dissolves the same class at the cost of restructuring `catchup.go`'s core and re-proving -the delivery/save ordering nine fixtures pin, for a class the deadline change already closes. - -Constraint that ruled out the obvious alternative: SPEC §23 pins **exactly six kebab-case timer -kinds** and per-state exact timer sets, asserted by the cross-SDK fixtures. Arming a dedicated timer -for this wait would be a seventh kind — a SPEC change across six SDKs for a Go-local bug. A deadline -value plus the existing wakes needs no new kind. - -## Triage table - -| # | Finding | Disposition | Failure mode named | -|---|---|---|---| -| 1 | `catchup.go:355` ×2 — staleness bound re-armed forever by a blocked pump | **BLOCKS** | **Hang.** Stalled `PollSource` + peer filling the 256-slot queue ⇒ every firing suspended-and-re-armed ⇒ consumer goroutine held indefinitely. Round-1's fix does not hold. | -| 2 | `catchup.go:556` — drain budget misses the pump's blocked hand-off | **BLOCKS** | **Wrong verdict + wrong save.** 256 queued + 1 held in `handOff` = 257 frames read at drain start; budget 256 ⇒ a fatal `invalid_event_stream_command` escapes the carve-out, the held position saves and `caught_up` announces. | -| 3 | `catchup.go:158` — deferred socket outcome lost on the poll-error path | **BLOCKS** | **Lost protocol-fatal verdict.** `recoverPoll`'s terminal branches call `disposeAttempt`, which clears `l.deferred`; an observed `invalid_event_stream_command` is replaced by `authorization_failed`/`poll_failed`. | -| 4 | `cable.go:180` — full-envelope unmarshal before the type switch | **BLOCKS** | **Reconnect loop on a server extension.** SPEC §23: "parseable JSON whose `type` the connector doesn't recognize — update liveness and are otherwise ignored". `{"type":"future","identifier":1}` currently tears the socket down; a server that keeps sending it loops every client forever. | -| 5 | `transport.go:51` — out-of-range port accepted by policy | **BLOCKS** | **Unbounded re-mint.** `wss://h:99999/` passes, dial fails `DialTransient` ⇒ re-mint/backoff forever on a permanently unusable URL — the exact class the function's own comment commits to catching. | -| 6 | `loop.go:932` — `frameReject` not gated on `AwaitingConfirmation` | **BLOCKS** | **Wrong permanent terminal.** Transition 12 exists only from `AwaitingConfirmation`; a premature `reject_subscription` yields `subscription_rejected` with ZERO reconnects and zero subscribe attempts. | -| 7 | `loop.go:901` — subscribe write is synchronous and unbounded | **BLOCKS** | **Defeated deadline.** A `CableConn.WriteFrame` that blocks past the handshake/confirmation deadline cannot be interrupted: `at.ctx` is only cancelled by a teardown the write is blocking. Feed hangs until `Connector.Close`. | -| 8 | `loop.go:908` — unchecked `deadline.Stop()` on welcome | **BLOCKS** | **Lapsed deadline not honored.** Both the frame and the expired timer ready ⇒ the `select` picks either; if the frame wins, a fired handshake deadline is replaced by a fresh confirmation timer and a late welcome is accepted instead of taking transition 9. Fix 7 widens this window, so they ship together. | -| 9 | `loop.go:454` — evicted buffer slots not cleared before reslicing | **BLOCKS** | **2× the published memory ceiling.** §23 publishes `(pumpDepth + capacity) × MAX_FRAME_BYTES`; the backing array keeps dropped payloads alive until a realloc. One-liner, and `shift` already does exactly this three lines below. | -| 10 | `loop.go:1110` — `Observer.Disconnected` missing on terminal teardowns | **FOLLOW-UP** | Real: `Observer.Disconnected` is documented "fires when a socket is torn down", unconditional, and `disposeAttempt` leaves four terminal paths silent. But the remedy changes the **normative observer sequence** that `conformance/event-feed/` fixtures pin across six SDKs (`expectDisconnected*` in `conformance/event-feed/schema.json`), so it is a §23 contract decision, not a Go-local bug. Issue filed. | -| 11 | `websocket_transport.go:256` — native `time.After` close budget | **DECLINE** | The failure does not occur. §23 "Clock, Timers and Virtual Time" scopes the rule to delays *the connector itself* takes and exempts delays **inside a seam call**; `CableConn.Close` is a seam call, and `WebSocketTransport` is a seam *implementation*. No deterministic scenario can reach it: every virtual-clock run substitutes `feedtest.Transport`, because a real socket is not deterministic either. A consumer holding a real socket is already in real time. | - -## Verification contract for the nine fixes - -Each gets a regression test proven RED against un-fixed code, with `REAL_EXIT` written to a log file -and grepped back. Mutated files restored by `cp` + `diff -q`, never `git checkout --`. - ---- - -# Outcome (appended after implementation; the table above is the pre-edit triage) - -The initial triage held, with two corrections the code forced: - -1. **`loop.go:1110` (Observer.Disconnected)** — filed as #753, as planned. -2. **`websocket_transport.go:256`** — declined, as planned. A LATER, different - finding on the same file (the close-budget timeout not unblocking reads) was - real and became #762. -3. **Everything marked BLOCKS was fixed and red-proven**, except that the - `loop.go:908` proof needed a mutation rather than a revert (its test needs a - hook the un-fixed code lacks). - -Six further bot review rounds arrived during the session, which the pre-edit -triage could not anticipate. Net: **13 fixes, 7 issues filed, 2 declines**, and -3 threads left open at handover with the reasoning posted publicly. - -The stopping-rule call in the header — "the four rounds orbit one borrowed -bound, not the deferral slot" — was **too strong, and I corrected it publicly**. -Removing the borrowing closed that half; Codex and Copilot then independently -found a second hang in the same wait from a different cause (no staleness case -at all in `pollPage`), and the obvious fix for it broke a pinned invariant -(`TestWalkFailureBetweenPages/staleness_expiry`). That became #758, with the -failing output attached — the most useful thing produced this session, because -it saves the next person from rediscovering it by breaking the same test. diff --git a/SPEC.md b/SPEC.md index ab98dda98..a58d17375 100644 --- a/SPEC.md +++ b/SPEC.md @@ -3616,19 +3616,32 @@ Two dispatch clarifications, pinned: back-pressure propagates to the socket and TCP — rather than dropping: the state-machine-owned live buffer is the only place a frame can ever be dropped, and its overflow signal is the only drop signal. Worst-case connector memory is therefore - bounded multiplicatively — every queued or buffered item is itself bounded by - `EVENT_FEED_MAX_FRAME_BYTES`, so the ceiling is - (pump depth + 2 + `EVENT_FEED_LIVE_BUFFER_CAPACITY`) × `EVENT_FEED_MAX_FRAME_BYTES` + bounded multiplicatively — every retained item is itself bounded by + `EVENT_FEED_MAX_FRAME_BYTES`, and retention is an enumeration by HOLDER, which is what + closes the count: a frame lives in the hand-off queue (≤ pump depth), in the live + buffer (≤ `EVENT_FEED_LIVE_BUFFER_CAPACITY`), in the single deferral slot (≤ 1), or in + the hands of one of the exactly two goroutines that touch frames — one in-hand frame + each. The ceiling is + (pump depth + 3 + `EVENT_FEED_LIVE_BUFFER_CAPACITY`) × `EVENT_FEED_MAX_FRAME_BYTES` (≈ 10 GiB at the defaults' extreme, reached only if every slot holds a maximum-size - frame) — even under a slow consumer. The **+ 2** is two raw frames the queue's depth does - not count, and they are retained by different parties at the same time: + frame) — even under a slow consumer. The **+ 3** is three raw frames the queue's depth + does not count — the deferral slot plus one in-hand frame per frame-touching party — + and they are retained by different parties at the same time: - the **pump's own in-flight frame** — the pump is a single reader, so it may hold exactly one frame it has already READ and not yet handed off. One rather than an unbounded number for that reason: one reader holds at most one frame outside the queue. + - the **state machine's in-hand frame** — the protocol-fatal scan's dequeue is the very + receive that lets a blocked pump refill the queue, so while the scan still holds that + frame — examining, admitting, or parking it — the queue is full again and the pump may + already hold its next read. A single consumer, so one frame, for the pump's own reason. - the **deferred socket outcome** — the single slot the in-flight-poll servicing and the drain's scan park one receive in. It is retained while the queue behind it refills, so it - is concurrent with a full queue and with the pump's held frame, not an alternative to - either. + is concurrent with a full queue and with both in-hand frames, not an alternative to + any of them. + + The enumeration cannot grow by a further party being noticed: every frame is in one of + the three counted structures or in the hands of the pump or the state machine, and each + of those holders is already counted. The drain's protocol-fatal scan is budgeted at `pump depth + 1` and not at this figure, which is not an inconsistency: the budget counts what the scan may DEQUEUE — the queue plus diff --git a/conformance/event-feed/README.md b/conformance/event-feed/README.md index 081bb0e41..0c995fa2e 100644 --- a/conformance/event-feed/README.md +++ b/conformance/event-feed/README.md @@ -141,8 +141,10 @@ asserting its scheduled delay against a `{min, max}` envelope — that is how ji is asserted without a cross-language RNG seam (Go additionally pins the full-jitter formula exactly in tier 3; a degenerate always-0 RNG is caught only there — a documented divergence). Each language's test clock passes the shared semantics -checklist (deadline order, reentrant scheduling within an advance, creation-order -tie-break) before its tier-2 results count. +checklist (deadline order, creation-order tie-break) before its tier-2 results +count; the reentrant clause stays normative for the algorithm — a clock that +ignored it would fire the wrong set — but no tier-2 fixture can reach it (next +paragraph), so it is not part of that gate. **The reentrant clause is unscriptable where the connector runs concurrently, so no fixture may rely on it.** In a single-threaded test clock, "a timer armed diff --git a/go/pkg/basecamp/eventfeed/buffer_test.go b/go/pkg/basecamp/eventfeed/buffer_test.go index 3e4a1e619..0ddd9fe1d 100644 --- a/go/pkg/basecamp/eventfeed/buffer_test.go +++ b/go/pkg/basecamp/eventfeed/buffer_test.go @@ -7,7 +7,7 @@ import ( // TestLiveBufferAddClearsEvictedSlots pins the eviction half of the live // buffer's memory ceiling. SPEC.md §23 publishes the connector's worst case -// as (pump depth + 2 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × EVENT_FEED_MAX_FRAME_BYTES; +// as (pump depth + 3 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × EVENT_FEED_MAX_FRAME_BYTES; // a reslice alone removes the evicted event LOGICALLY while the slice that // results still points into the same backing array, whose prefix keeps that // event's strings reachable until a later reallocation. Under sustained diff --git a/go/pkg/basecamp/eventfeed/catchup.go b/go/pkg/basecamp/eventfeed/catchup.go index 4da771a5d..46f8d465c 100644 --- a/go/pkg/basecamp/eventfeed/catchup.go +++ b/go/pkg/basecamp/eventfeed/catchup.go @@ -1117,11 +1117,13 @@ func (l *loop) probeFatal(at *attempt) (cycleOutcome, bool) { // // This is why the scan needs no queue and no share of pumpDepth: it retains // exactly what the single slot always retained. The connector's published -// memory bound — (pump depth + 2 + liveBufferCapacity) × MAX_FRAME_BYTES — is +// memory bound — (pump depth + 3 + liveBufferCapacity) × MAX_FRAME_BYTES — is // untouched, and so is the depth at which the pump blocks. The slot IS one of -// that formula's two raw-frame terms beyond the queue; the other is the frame -// the pump has read and not yet handed off. Both are retained WHILE the queue -// is full, which is why they are addends and not alternatives. +// that formula's three raw-frame terms beyond the queue; the others are the +// frame the pump has read and not yet handed off, and the frame the scan +// itself has dequeued and not yet disposed of — the very receive that lets a +// blocked pump refill the queue behind it. All three are retained WHILE the +// queue is full, which is why they are addends and not alternatives. func (l *loop) deferForDrain(d *deferredFrame) { if l.deferred == nil { l.deferred = d diff --git a/go/pkg/basecamp/eventfeed/catchup_test.go b/go/pkg/basecamp/eventfeed/catchup_test.go index 2014a8ac3..5fe1b1df8 100644 --- a/go/pkg/basecamp/eventfeed/catchup_test.go +++ b/go/pkg/basecamp/eventfeed/catchup_test.go @@ -1396,7 +1396,7 @@ func TestDrainScanAdmissionIsNotStranded(t *testing.T) { // TestDrainHoldsNoMoreThanTheLiveBufferCapacity: the live buffer's capacity // is a bound on events HELD AT ONCE — SPEC §23 sizes the connector's whole -// memory ceiling off it, "(pump depth + 2 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × +// memory ceiling off it, "(pump depth + 3 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × // EVENT_FEED_MAX_FRAME_BYTES" — so a drain must not be able to hold a batch // outside the buffer while the buffer refills to capacity behind it. // diff --git a/go/pkg/basecamp/eventfeed/feedtest/clock.go b/go/pkg/basecamp/eventfeed/feedtest/clock.go index 0d51cfe90..2794a62e7 100644 --- a/go/pkg/basecamp/eventfeed/feedtest/clock.go +++ b/go/pkg/basecamp/eventfeed/feedtest/clock.go @@ -80,7 +80,12 @@ func (c *Clock) NewTimer(d time.Duration, name string) eventfeed.Timer { func (c *Clock) DueWithin(d time.Duration) []string { c.mu.Lock() defer c.mu.Unlock() - target := c.now.Add(d) + return c.dueWithinLocked(c.now.Add(d)) +} + +// dueWithinLocked returns the names of live timers due at or before target, +// in creation order. The caller holds c.mu. +func (c *Clock) dueWithinLocked(target time.Time) []string { var names []string for _, t := range c.live { if !t.deadline.After(target) { @@ -90,6 +95,37 @@ func (c *Clock) DueWithin(d time.Duration) []string { return names } +// AdvanceIfQuiet advances virtual time by d only if the window would fire +// nothing; otherwise it reports the due set and leaves the clock untouched. +// It is DueWithin and Advance as ONE critical section, for the driver MUST +// in SPEC §23: an advance whose window would fire any timer is rejected. +// Deciding that with two separate lock acquisitions leaves a gap — a timer +// armed (or stopped) between the check and the movement changes what the +// accepted directive does, so an advance the guard accepted could fire. +// Under one hold of the clock's locks, an accepted advance provably fires +// nothing. +// +// What stays undecidable, stated honestly: whether a CONCURRENT arm lands +// before or after this critical section is still the arming goroutine's +// schedule — no clock operation can order another goroutine's lock +// acquisition. The invariant restored here is the decidable one: whichever +// side the arm lands, an ACCEPTED advance fired nothing, and an arm that +// lost the race is due at its own deadline, unfired and unharmed. +func (c *Clock) AdvanceIfQuiet(d time.Duration) ([]string, bool) { + c.advancing.Lock() + defer c.advancing.Unlock() + c.mu.Lock() + target := c.now.Add(d) + if due := c.dueWithinLocked(target); len(due) > 0 { + c.mu.Unlock() + return due, false + } + c.now = target + c.mu.Unlock() + c.cond.Broadcast() + return nil, true +} + // Outstanding returns the names of live (unfired, unstopped) timers, in // creation order. func (c *Clock) Outstanding() []string { diff --git a/go/pkg/basecamp/eventfeed/feedtest/clock_test.go b/go/pkg/basecamp/eventfeed/feedtest/clock_test.go index 5df388698..4695df3ff 100644 --- a/go/pkg/basecamp/eventfeed/feedtest/clock_test.go +++ b/go/pkg/basecamp/eventfeed/feedtest/clock_test.go @@ -121,6 +121,64 @@ func TestClock_AdvanceFiresATimerArmedByAFiringsRecipient(t *testing.T) { } } +// The reject half of the one-critical-section contract: a window that would +// fire is refused with the due set named, and the clock is untouched — time +// has not moved and the refused timer is still outstanding and firable. +func TestClock_AdvanceIfQuietRejectsAFiringWindowUntouched(t *testing.T) { + c := NewClock() + base := c.Now() + tm := c.NewTimer(5*time.Millisecond, "backoff") + c.NewTimer(30*time.Millisecond, "repair-poll") + + due, ok := c.AdvanceIfQuiet(10 * time.Millisecond) + + if ok { + t.Fatal("AdvanceIfQuiet accepted a window that would fire backoff") + } + if len(due) != 1 || due[0] != "backoff" { + t.Errorf("due = %v, want [backoff]", due) + } + if got := c.Now(); !got.Equal(base) { + t.Errorf("Now() = %v, want unmoved %v", got, base) + } + select { + case at := <-tm.C(): + t.Errorf("rejected advance fired the timer at %v", at) + default: + } + if got := c.Outstanding(); len(got) != 2 { + t.Errorf("Outstanding() = %v, want both timers still live", got) + } + // The refused timer is unharmed: it still fires on its own terms. + if delay, found := c.FireTimer("backoff"); !found || delay != 5*time.Millisecond { + t.Errorf("FireTimer(backoff) = (%v, %v), want (5ms, true)", delay, found) + } +} + +// The accept half: a genuinely quiet window moves time by exactly d and +// leaves undue timers outstanding. A deadline landing exactly ON the window +// edge counts as due — the same !After(target) boundary Advance fires at. +func TestClock_AdvanceIfQuietMovesTimeOnAQuietWindow(t *testing.T) { + c := NewClock() + base := c.Now() + c.NewTimer(30*time.Millisecond, "repair-poll") + + if due, ok := c.AdvanceIfQuiet(10 * time.Millisecond); !ok { + t.Fatalf("quiet window rejected: due = %v", due) + } + if got := c.Now(); !got.Equal(base.Add(10 * time.Millisecond)) { + t.Errorf("Now() = %v, want %v", got, base.Add(10*time.Millisecond)) + } + if got := c.Outstanding(); len(got) != 1 || got[0] != "repair-poll" { + t.Errorf("Outstanding() = %v, want [repair-poll]", got) + } + + // Exactly-on-the-edge is a firing window, not a quiet one. + if due, ok := c.AdvanceIfQuiet(20 * time.Millisecond); ok || len(due) != 1 || due[0] != "repair-poll" { + t.Errorf("edge deadline: AdvanceIfQuiet = (%v, %v), want ([repair-poll], false)", due, ok) + } +} + func TestClock_StopRemovesTimerAndSuppressesFiring(t *testing.T) { c := NewClock() timer := c.NewTimer(5*time.Millisecond, "staleness") diff --git a/go/pkg/basecamp/eventfeed/scenario_conformance_test.go b/go/pkg/basecamp/eventfeed/scenario_conformance_test.go index 1ce507533..0968caefe 100644 --- a/go/pkg/basecamp/eventfeed/scenario_conformance_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_conformance_test.go @@ -622,8 +622,10 @@ func (d *driver) nextClientFrame(what string) (clientFrame, error) { // clock selects due timers under its own lock and unlocks ONLY across a // firing's aftermath — deliberately, so a woken recipient can arm inside the // window. An advance with nothing due therefore never unlocks, never wakes -// anything, and cannot be the cause of any arm. One atomic read of the clock -// settles it before time moves. +// anything, and cannot be the cause of any arm. AdvanceIfQuiet decides the +// due set and moves time under one hold of the clock's locks — a check and a +// movement in two acquisitions would leave a gap where a concurrently armed +// timer turns an accepted advance into a firing one. // // A script that wants a firing writes `fireTimer`, which fires one named timer // without advancing the clock and so involves no re-selection at all. The @@ -634,14 +636,13 @@ func (d *driver) nextClientFrame(what string) (clientFrame, error) { // AdvanceSettling remains for a caller that genuinely wants a chained firing // with an explicit rendezvous. It is deliberately not reachable from a fixture. func (d *driver) advance(step *advanceStep) error { - if due := d.h.clock.DueWithin(millis(step.Ms)); len(due) > 0 { + if due, ok := d.h.clock.AdvanceIfQuiet(millis(step.Ms)); !ok { return fmt.Errorf( "advance of %dms would fire %v: whether a timer armed by one of those firings lands inside the "+ "same window depends on goroutine scheduling, so this script cannot mean the same thing in "+ "every language — use fireTimer, which fires one named timer without re-selecting", step.Ms, due) } - d.h.clock.Advance(millis(step.Ms)) return nil } diff --git a/python/src/basecamp/services/todolists.py b/python/src/basecamp/services/todolists.py index 715012155..f2ca9904c 100644 --- a/python/src/basecamp/services/todolists.py +++ b/python/src/basecamp/services/todolists.py @@ -140,9 +140,9 @@ def _writable_string(body: dict[str, Any], key: str, *, non_empty: bool = False) makes the check explicit work here rather than something the layer below already did, and #544 did not change it: flattening the declared shape changes what the API returns, not what Python validates — ``get`` still - hands back the parsed JSON as ``dict[str, Any]``. The same shape is live in - the shipped Todos and Cards composites; that is tracked separately in #576, - and giving Python a decoder at all in #578. + hands back the parsed JSON as ``dict[str, Any]``. The same shape in the + shipped Todos and Cards composites is guarded by the ``_merge_safe`` checks + #576 closed with; giving Python a decoder at all is tracked in #578. """ if key not in body: raise ApiError( diff --git a/python/tests/services/test_todolists_service.py b/python/tests/services/test_todolists_service.py index 64957b416..2bac84c2b 100644 --- a/python/tests/services/test_todolists_service.py +++ b/python/tests/services/test_todolists_service.py @@ -278,7 +278,8 @@ class TestMalformedWritableFields: is full-replace, that value is then written back over the real one — the composite erases the field it exists to preserve, on a call that never mentioned it. Truthy non-strings are just as wrong: they reach the wire - verbatim. The shipped Todos/Cards analogue is tracked in #576. + verbatim. The shipped Todos/Cards analogue takes the same refusal from + the ``_merge_safe`` guards #576 closed with. """ @pytest.mark.parametrize("malformed", [False, 0, [], {}, 42, True, ["x"], {"a": 1}]) diff --git a/ruby/lib/basecamp/services/todolists_extensions.rb b/ruby/lib/basecamp/services/todolists_extensions.rb index a0324c062..32f55b2d1 100644 --- a/ruby/lib/basecamp/services/todolists_extensions.rb +++ b/ruby/lib/basecamp/services/todolists_extensions.rb @@ -180,9 +180,10 @@ def require_hash(body) # Ruby has no typed decoder between the GET and this read, unlike the Go, # Swift and Kotlin composites where a wrong-typed field fails at decode, # and flattening the shape did not add one: the generated method still - # returns http_get(...).json verbatim. The same shape is live in - # the shipped Todos composite; tracked in #576, with the generated - # validating layer that would retire this guard tracked in #578. + # returns http_get(...).json verbatim. The same shape in the + # shipped Todos composite is guarded by the MergeSafe checks #576 closed + # with; the generated validating layer that would retire this guard is + # tracked in #578. def writable_string(body, key, non_empty: false) raise_missing_field(key) unless body.key?(key) diff --git a/ruby/test/basecamp/services/todolists_service_test.rb b/ruby/test/basecamp/services/todolists_service_test.rb index 02d62be7d..ad47141e9 100644 --- a/ruby/test/basecamp/services/todolists_service_test.rb +++ b/ruby/test/basecamp/services/todolists_service_test.rb @@ -159,7 +159,8 @@ def test_replace_sends_an_explicit_empty_description # straight through. This endpoint is full-replace, so either outcome is # written back over the real value — the composite erases or corrupts the # field it exists to preserve, on a call that never mentioned it. The shipped - # Todos analogue is tracked in #576. + # Todos analogue takes the same refusal from the MergeSafe guards #576 + # closed with. [ false, 0, [], {}, 42, true, [ "x" ], { "a" => 1 } ].each do |malformed| define_method("test_update_refuses_a_malformed_description_#{malformed.inspect}") do stub_todolist_get_and_put(todolist: full_todolist.merge("description" => malformed)) diff --git a/scripts/check-known-defect-issues-open b/scripts/check-known-defect-issues-open index 0ee5a4e4d..7d3a2beb0 100755 --- a/scripts/check-known-defect-issues-open +++ b/scripts/check-known-defect-issues-open @@ -94,12 +94,20 @@ end # issue number is making no claim. Here the row IS the claim: a malformed one is # a promise the author wrote and the gate silently dropped, which is worse than # either a failure or an absent row. +# The KEY is required, not just the file. A registry whose key is misspelled +# or dropped in an edit reads as nil and used to pass as empty — the same +# fail-open path as a deleted file, one typo later. Only the explicit empty +# list says "no promises" on purpose. +unless tracking.key?('prose_tracking_issues') + warn "ERROR: #{tracking_path} must define prose_tracking_issues (use [] when there are none)." + exit 1 +end rows = tracking['prose_tracking_issues'] -unless rows.nil? || rows.is_a?(Array) +unless rows.is_a?(Array) warn "ERROR: prose_tracking_issues in #{tracking_path} must be a list, got #{rows.class}" exit 1 end -(rows || []).each_with_index do |e, i| +rows.each_with_index do |e, i| unless e.is_a?(Hash) warn "ERROR: prose_tracking_issues[#{i}] in #{tracking_path} must be a mapping, got #{e.class}" exit 1 diff --git a/scripts/test-check-known-defect-issues-open b/scripts/test-check-known-defect-issues-open index 502e636cf..627f45a4e 100755 --- a/scripts/test-check-known-defect-issues-open +++ b/scripts/test-check-known-defect-issues-open @@ -89,7 +89,10 @@ failures = [] # all, when gh is meant to be absent), so the real gh can never answer. The # interpreter is invoked by absolute path for the same reason. Returns # [combined_output, status, gh_log_lines]. -def run_checker(allowlist:, tracking: {}, states: nil, gh_fail: false, repo: nil, github_repository: nil) +# tracking defaults to the explicit empty registry — the one sanctioned no-op — +# because a file WITHOUT the key is an error (case N+7): most cases here are +# about the allowlist and just need the registry to say "no promises" on purpose. +def run_checker(allowlist:, tracking: { "prose_tracking_issues" => [] }, states: nil, gh_fail: false, repo: nil, github_repository: nil) Dir.mktmpdir("known-defect-selftest") do |dir| path_dir = File.join(dir, "bin") Dir.mkdir(path_dir) @@ -384,6 +387,20 @@ out, status, _ = run_checker( ) expect_fail(failures, "N+6. a non-list registry fails", out, status, "must be a list") +# --- N+7. A registry file without the key fails ----------------------------------- +# +# The missing-FILE case (N+3) fails closed; a file whose key is misspelled or +# dropped in an edit is the same accident one typo later, and it used to pass +# as an empty registry. Only the explicit empty list says "no promises" on +# purpose — a keyless file is indistinguishable from one the gate never read. + +out, status, _ = run_checker( + allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, + tracking: {} +) +expect_fail(failures, "N+7. a registry without the prose_tracking_issues key fails", out, status, + "must define prose_tracking_issues") + if failures.empty? puts "==> known-defect tracking-issue self-test passed — #{CASES[0]} cases" exit 0 diff --git a/spec/tracking-issues.yml b/spec/tracking-issues.yml index 437bd5787..0a5438379 100644 --- a/spec/tracking-issues.yml +++ b/spec/tracking-issues.yml @@ -19,4 +19,14 @@ # # Add an entry when you write a sentence promising an issue owns something. # Remove it when the sentence goes, and the gate stops asking. -prose_tracking_issues: [] +prose_tracking_issues: + - issue: 589 + site: "Makefile — the vendored bc3-route table's freshness gate needs BC3_REPO_PATH, so it is not in CI" + - issue: 775 + site: "SPEC.md §6 Retry-After — which statuses honour the header is divergent across the six SDKs" + - issue: 775 + site: "typescript/tests/retry-after.test.ts — pins TypeScript's side of the Retry-After status divergence" + - issue: 578 + site: "ruby/lib/basecamp/services/todolists_extensions.rb — the generated validating layer that would retire writable_string" + - issue: 578 + site: "python/src/basecamp/services/todolists.py — making the flat todolist read structurally safe" From d8beb4dd935386b8ccc9089c9bdb92dddba167a8 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Sat, 22 Aug 2026 01:41:10 -0700 Subject: [PATCH 18/55] Back the tracking registry with a detector for its canonical grammar MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot's second round on the registry named three more unregistered sites, which per AGENTS.md is the signal to reassess the instrument rather than enumerate again: hand-enumeration cannot prove coverage, and each round was proving it. Both halves ship here — the three sites are registered, and the checker gains the discovery sweep that ends the class for the canonical form. * The sweep scans tracked text files for the one canonical grammar — "tracked [separately] in #N" — and fails when a mention's [file, issue] pair is not registered, so that form can never again be written without being registered. The match tolerates a newline plus a comment leader mid-phrase, which is not hypothetical: MIGRATING.md's own mention wraps as "tracked / separately in #775", and a line-based grep reads exactly that promise as absence (it is how the earlier enumeration missed it). Files come from `git ls-files` — tracked text only, .gitignore respected — with a directory-walk fallback for a sweep root that is not a repository; binaries are skipped by NUL sniff; excluded are the registry itself and the gate's self-test (both quote the grammar as data) and spec/api-gaps/ (as-of citations, never promises). The live sweep costs ~1s. * Stated plainly, in the script header and here: non-canonical phrasings — "structural safety for this SDK is #578" — are out of the detector's reach BY DESIGN. An open vocabulary cannot be enumerated, and a regex that tried would sweep up the as-of citations that must never be flagged. The registry remains the judgment surface for those; the sweep guarantees only that the canonical form cannot dodge it. * Registry entries now carry `file`, because the sweep keys coverage on [file, issue] — the same issue registered for a different file must not cover a mention. The field is validated like `issue` and `site`. * The three sites from the review are registered: MIGRATING.md (#775, the wrapped mention above), typescript/src/services/ todolists-extensions.ts (#578), and python/tests/services/ test_todolist_groups_service.py (#578) — the latter two non-canonical, registered by hand exactly as the boundary above prescribes. Red-proven before the fix: the new self-test cases were run against the pre-sweep checker and exactly the three that assert the new behavior failed — an unregistered canonical mention (N+8), a line-wrapped one (N+11), and a row without `file` (N+12) — each with "expected FAILURE but checker passed". After the fix all 22 cases pass, and unregistering the MIGRATING.md entry against the real tree makes the live sweep name MIGRATING.md:176 precisely. The re-raised sentinel comment on README:432 needed no change: it is round one's thread re-anchored to the new head (created 2026-08-18, original commit 039aad65b), already fixed in be5a1c7e0 and resolved — schema.json:906 at head states the withdrawal it asks for. Gates: tracking-issues self-test 22/22; the live checker sweeps the tree clean and verifies all eight registered claims (#589, #775, #578 open); doc-constants-check green under LC_ALL=C. --- scripts/check-known-defect-issues-open | 85 ++++++++++++++++++ scripts/test-check-known-defect-issues-open | 97 ++++++++++++++++++++- spec/tracking-issues.yml | 22 +++++ 3 files changed, 200 insertions(+), 4 deletions(-) diff --git a/scripts/check-known-defect-issues-open b/scripts/check-known-defect-issues-open index 7d3a2beb0..fc2c397ae 100755 --- a/scripts/check-known-defect-issues-open +++ b/scripts/check-known-defect-issues-open @@ -16,6 +16,16 @@ # "#\d+" cannot tell a tracking promise from an as-of citation ("shipped in # #12380"), and sorting those needs the author, not a matcher. # +# The registry is the judgment surface, and one DETECTOR backs it: the checker +# sweeps tracked text files for the canonical grammar "tracked [separately] in +# #N" (tolerating a comment-prefixed line wrap mid-phrase) and fails when a +# mention's [file, issue] pair is not registered — so the canonical form can +# never be written without being registered. Non-canonical phrasings ("structural +# safety ... is #N") are OUT OF THE DETECTOR'S REACH BY DESIGN: an open +# vocabulary cannot be enumerated, and a regex that tried would sweep up the +# as-of citations that must never be flagged. Those still go through the +# registry by hand. PROSE_SWEEP_ROOT overrides the swept tree (self-test). +# # check-bc3-route-parity already requires those to be numeric. A number is not a # tracker: #588 auto-closed while nine live 404s still pointed at it, and that # gate stayed green the whole time because "is an Integer" was the entire test. @@ -121,9 +131,84 @@ rows.each_with_index do |e, i| 'the sentence that leans on it — a bare number tells the next reader nothing to go fix.' exit 1 end + if e['file'].to_s.strip.empty? + warn "ERROR: prose_tracking_issues[#{i}] (##{e['issue']}) in #{tracking_path} needs a `file` naming " \ + 'the path the sentence lives in — the discovery sweep keys coverage on [file, issue].' + exit 1 + end refs << [e['issue'], e['site']] end +# --- Discovery sweep -------------------------------------------------------------- +# +# Hand-enumeration cannot prove the registry covers the tree, so the one +# canonical grammar is swept for: "tracked [separately] in #N". A mention whose +# [file, issue] pair is not registered fails the gate. The GAP between the +# phrase's words tolerates a newline plus a comment leader, because the phrase +# wraps in real prose and a line-based match would read a wrapped promise as +# absence. Excluded: the registry itself and this gate's self-test (both quote +# the grammar as data — the header's example, the test's corpus literals) and +# spec/api-gaps/ (historical as-of citations, never promises). +# +# Files come from `git ls-files` — tracked text only, .gitignore respected — +# with a plain directory walk as the fallback for a sweep root that is not a +# repository (the self-test corpus). Binary files are skipped by NUL sniff. + +SWEEP_GAP = /(?:[ \t]|\r?\n[ \t]*(?:#(?!\d)|\/\/|\*+|--)?[ \t]*)+/ +SWEEP_MENTION = /\btracked(?:#{SWEEP_GAP}separately)?#{SWEEP_GAP}in#{SWEEP_GAP}#(\d+)/i + +def sweep_files(root) + listing = begin + out = IO.popen([ 'git', '-C', root, 'ls-files', '-z' ], err: File::NULL, &:read) + $?.success? ? out.split("\0") : nil + rescue Errno::ENOENT + nil + end + return listing unless listing.nil? || listing.empty? + + require 'find' + files = [] + Find.find(root) do |p| + if File.directory?(p) + Find.prune if File.basename(p) == '.git' + next + end + files << p.delete_prefix("#{root}#{File::SEPARATOR}") + end + files +end + +sweep_root = ENV['PROSE_SWEEP_ROOT'] || ROOT +registered_pairs = rows.map { |e| [ e['file'], e['issue'] ] } +violations = [] +sweep_files(sweep_root).each do |rel| + next if rel == 'spec/tracking-issues.yml' || + rel == 'scripts/test-check-known-defect-issues-open' || + rel.start_with?('spec/api-gaps/') + + path = File.join(sweep_root, rel) + next unless File.file?(path) + + content = File.binread(path) + next if content.index("\0".b) + + content = content.force_encoding(Encoding::UTF_8).scrub + content.to_enum(:scan, SWEEP_MENTION).each do + m = Regexp.last_match + next if registered_pairs.include?([ rel, m[1].to_i ]) + + line = content[0...m.begin(0)].count("\n") + 1 + violations << "#{rel}:#{line} — a canonical \"tracked in ##{m[1]}\" claim, and [#{rel}, ##{m[1]}] is not registered" + end +end +unless violations.empty? + warn 'ERROR: canonical tracking claims exist outside the registry:' + violations.each { |v| warn " - #{v}" } + warn ' Register each in spec/tracking-issues.yml (issue, file, site) — or, if the' + warn ' promise is already discharged, rewrite the sentence as an as-of fact.' + exit 1 +end + if refs.empty? puts ' ✓ no tracking issues referenced — nothing to verify' exit 0 diff --git a/scripts/test-check-known-defect-issues-open b/scripts/test-check-known-defect-issues-open index 627f45a4e..50bb65b6e 100755 --- a/scripts/test-check-known-defect-issues-open +++ b/scripts/test-check-known-defect-issues-open @@ -47,6 +47,7 @@ require "yaml" require "tmpdir" +require "fileutils" require "open3" require "rbconfig" @@ -92,7 +93,13 @@ failures = [] # tracking defaults to the explicit empty registry — the one sanctioned no-op — # because a file WITHOUT the key is an error (case N+7): most cases here are # about the allowlist and just need the registry to say "no promises" on purpose. -def run_checker(allowlist:, tracking: { "prose_tracking_issues" => [] }, states: nil, gh_fail: false, repo: nil, github_repository: nil) +# +# sweep is the discovery corpus: filename => content, written under a tmp dir +# that PROSE_SWEEP_ROOT points the checker's canonical-grammar sweep at. Every +# case gets a corpus — empty by default — so the sweep can never read the REAL +# repository against a synthetic registry, which would fail every case here on +# facts about the tree rather than about the checker. +def run_checker(allowlist:, tracking: { "prose_tracking_issues" => [] }, sweep: {}, states: nil, gh_fail: false, repo: nil, github_repository: nil) Dir.mktmpdir("known-defect-selftest") do |dir| path_dir = File.join(dir, "bin") Dir.mkdir(path_dir) @@ -116,10 +123,19 @@ def run_checker(allowlist:, tracking: { "prose_tracking_issues" => [] }, states: # case is expressed without deleting the repository's real one. File.write(tracking_path, YAML.dump(tracking)) unless tracking.nil? + sweep_root = File.join(dir, "corpus") + Dir.mkdir(sweep_root) + sweep.each do |rel, content| + full = File.join(sweep_root, rel) + FileUtils.mkdir_p(File.dirname(full)) + File.write(full, content) + end + env = { "PATH" => path_dir, "BC3_ROUTE_ALLOWLIST" => allow_path, "TRACKING_ISSUES" => tracking_path, + "PROSE_SWEEP_ROOT" => sweep_root, "GH_STUB_LOG" => gh_log, "GH_STUB_STATES" => states, "GH_STUB_FAIL" => (gh_fail ? "1" : nil), @@ -312,7 +328,7 @@ expect_fail(failures, "9. missing gh fails closed", out, status, out, status, calls = run_checker( allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, - tracking: { "prose_tracking_issues" => [ { "issue" => 303, "site" => "README.md — a gap" } ] }, + tracking: { "prose_tracking_issues" => [ { "issue" => 303, "file" => "README.md", "site" => "README.md — a gap" } ] }, states: "303=open" ) expect_pass(failures, "N. prose registry entries are collected", out, status, @@ -329,7 +345,7 @@ end out, status, _ = run_checker( allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, - tracking: { "prose_tracking_issues" => [ { "issue" => 404, "site" => "SPEC.md — a bound" } ] }, + tracking: { "prose_tracking_issues" => [ { "issue" => 404, "file" => "SPEC.md", "site" => "SPEC.md — a bound" } ] }, states: "404=closed" ) expect_fail(failures, "N+1. a CLOSED prose tracking issue fails", out, status, @@ -342,7 +358,7 @@ expect_fail(failures, "N+1. a CLOSED prose tracking issue fails", out, status, out, status, _ = run_checker( allowlist: { "sdk_routes_known_defective" => [ defective(101) ], "bc3_routes_not_modeled" => [] }, - tracking: { "prose_tracking_issues" => [ { "issue" => 303, "site" => "README.md — a gap" } ] }, + tracking: { "prose_tracking_issues" => [ { "issue" => 303, "file" => "README.md", "site" => "README.md — a gap" } ] }, states: "101=open,303=closed" ) expect_fail(failures, "N+2. a closed registry entry fails even with a green allowlist", out, status, @@ -401,6 +417,79 @@ out, status, _ = run_checker( expect_fail(failures, "N+7. a registry without the prose_tracking_issues key fails", out, status, "must define prose_tracking_issues") +# --- N+8. An unregistered canonical mention fails the sweep ---------------------- +# +# The instrument case. Hand-enumeration cannot prove coverage, so the checker +# sweeps tracked text for the one canonical grammar — "tracked [separately] in +# #N" — and an occurrence whose [file, issue] pair is absent from the registry +# fails the gate. The canonical form can never again be written without being +# registered. + +out, status, _ = run_checker( + allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, + sweep: { "README.md" => "This gap is tracked in #909 until the decoder lands.\n" } +) +expect_fail(failures, "N+8. an unregistered canonical mention fails the sweep", out, status, + "not registered") + +# --- N+9. A registered canonical mention passes ---------------------------------- +# +# Coverage is keyed on [file, issue]: the same issue registered for a DIFFERENT +# file must not cover this one, so the entry names the corpus file exactly. + +out, status, calls = run_checker( + allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, + tracking: { "prose_tracking_issues" => [ + { "issue" => 303, "file" => "docs/note.md", "site" => "docs/note.md — a gap" } + ] }, + sweep: { "docs/note.md" => "The gap is tracked in #303.\n" }, + states: "303=open" +) +expect_pass(failures, "N+9. a registered canonical mention passes the sweep", out, status) +unless calls.any? { |c| c.end_with?("/issues/303") } + puts " FAIL N+9a. the registered mention is still looked up" + failures << "N+9a: expected a lookup of #303, got #{calls.inspect}" +end + +# --- N+10. A non-canonical phrasing does not trip the sweep ---------------------- +# +# By design, not by gap: "structural safety ... is #N" is an open vocabulary no +# regex can enumerate without also sweeping up as-of citations. The registry +# stays the judgment surface for those; the sweep guarantees only the canonical +# form. + +out, status, _ = run_checker( + allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, + sweep: { "README.md" => "Structural safety for this SDK is #909.\n" } +) +expect_pass(failures, "N+10. a non-canonical phrasing does not trip the sweep", out, status, + "no tracking issues referenced") + +# --- N+11. The canonical grammar is caught across a comment line wrap ------------ +# +# The phrase wraps in real prose — MIGRATING.md's own mention is "tracked\n +# separately in #775" — and a line-based grep reads that as absence. The sweep +# must see through a newline plus a comment leader mid-phrase. + +out, status, _ = run_checker( + allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, + sweep: { "lib/service.rb" => "# the six SDKs disagree and this is tracked\n# separately in #909.\n" } +) +expect_fail(failures, "N+11. a line-wrapped canonical mention is still caught", out, status, + "not registered") + +# --- N+12. A row with no file fails ---------------------------------------------- +# +# The sweep keys coverage on [file, issue], so an entry without a file could +# never cover the mention it registers — it would pass validation and leave the +# sweep red with no way to say which entry is wrong. + +out, status, _ = run_checker( + allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, + tracking: { "prose_tracking_issues" => [ { "issue" => 303, "site" => "README.md — a gap" } ] } +) +expect_fail(failures, "N+12. a row with no file fails", out, status, "needs a `file`") + if failures.empty? puts "==> known-defect tracking-issue self-test passed — #{CASES[0]} cases" exit 0 diff --git a/spec/tracking-issues.yml b/spec/tracking-issues.yml index 0a5438379..ea844f6b0 100644 --- a/spec/tracking-issues.yml +++ b/spec/tracking-issues.yml @@ -17,16 +17,38 @@ # need an ever-growing exclusion list to tell the two apart. The registry makes # the author say which kind they meant, once, where the reason is visible. # +# Every entry names the `file` its sentence lives in, because one detector +# backs this registry: the checker sweeps tracked text for the canonical +# grammar "tracked [separately] in #N" and fails on any mention whose +# [file, issue] pair is not registered here — the canonical form cannot be +# written without being registered. Non-canonical phrasings are out of the +# detector's reach by design (an open vocabulary cannot be enumerated); they +# still go through this registry by hand, on the honor the header describes. +# # Add an entry when you write a sentence promising an issue owns something. # Remove it when the sentence goes, and the gate stops asking. prose_tracking_issues: - issue: 589 + file: "Makefile" site: "Makefile — the vendored bc3-route table's freshness gate needs BC3_REPO_PATH, so it is not in CI" - issue: 775 + file: "SPEC.md" site: "SPEC.md §6 Retry-After — which statuses honour the header is divergent across the six SDKs" - issue: 775 + file: "typescript/tests/retry-after.test.ts" site: "typescript/tests/retry-after.test.ts — pins TypeScript's side of the Retry-After status divergence" + - issue: 775 + file: "MIGRATING.md" + site: "MIGRATING.md — the Retry-After migration note repeats the status-divergence promise" - issue: 578 + file: "ruby/lib/basecamp/services/todolists_extensions.rb" site: "ruby/lib/basecamp/services/todolists_extensions.rb — the generated validating layer that would retire writable_string" - issue: 578 + file: "python/src/basecamp/services/todolists.py" site: "python/src/basecamp/services/todolists.py — making the flat todolist read structurally safe" + - issue: 578 + file: "typescript/src/services/todolists-extensions.ts" + site: "typescript/src/services/todolists-extensions.ts — structural safety for this SDK" + - issue: 578 + file: "python/tests/services/test_todolist_groups_service.py" + site: "python/tests/services/test_todolist_groups_service.py — giving Python a structural decoder" From 7f07755b09b2d0c647df5e3c45b42dddfb01613b Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Sat, 22 Aug 2026 02:16:18 -0700 Subject: [PATCH 19/55] Close the third round on the registry: leader by class, ceiling by lane MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot's third round: one detector gap, one SPEC overclaim, four more unregistered sites. All three taken. * The sweep's comment leader is matched by CLASS, not by count. `//` consumed exactly two slashes, so a Swift/Kotlin doc comment's third left `/ separately in #N` unmatchable and read the wrapped promise as absence — the same failure shape the wrap tolerance exists to close, one mark deeper. Now `#+`, two-or-more slashes, `*+`, and `--+` (with `#+` still refusing a following digit, so an issue reference is never eaten as a leader). Red-proven: the new Swift `///` corpus case was run against the previous detector first and failed with "expected FAILURE but checker passed"; all 23 cases pass after the widening. * The retention formula is scoped to what it counts: the CABLE lane. It was published as "worst-case connector memory", but every counted item is a socket frame or a buffered live event, while `PollSource.Poll` returns one page decoded whole and the walk retains that page until its rows are delivered — `PollPage.Events` carries no cap in the seam contract, §23, or the pagination sections (page size is the server's decision, which `EVENT_FEED_MAX_FRAME_BYTES` does not govern). The honest statement ships instead of an invented cap: the formula is the cable lane's bound; the poll lane holds at most one live page per walk by shape (sequential polls; a superseded attempt's in-flight poll may briefly hold another before its result is discarded), unbounded in size by this contract on purpose. The three Go comment mirrors and both test comments now say "cable lane" too. * The four named #578 sites are registered (typescript/tests todolists.test.ts, ruby merge_safe.rb, python _merge_safe.py, typescript merge-safe.ts) — and, three rounds being the signal that the set had to be closed rather than chased, the whole tree was enumerated for the three issues the registry knows: git grep -nI -e '#578' -e '#589' -e '#775' -- \ ':!spec/api-gaps' ':!spec/tracking-issues.yml' \ ':!scripts/test-check-known-defect-issues-open' 17 mention lines; after registering the three additional #775 sites it surfaced (kotlin Pagination.kt and PaginationTest.kt — "#775 carries the six-SDK table" — and typescript errors.ts), the residual is ZERO unregistered [file, issue] pairs. A fourth round on these three issues has nothing left to find; the registry holds 15 entries and the gate verifies every one. Gates: tracking-issues self-test 23/23; the live checker sweeps clean and verifies all 15 registered claims (#589, #775, #578 open); doc-constants-check green under LC_ALL=C; both eventfeed Go packages green. --- SPEC.md | 13 ++++++++++++- go/pkg/basecamp/eventfeed/buffer_test.go | 2 +- go/pkg/basecamp/eventfeed/catchup.go | 6 +++--- go/pkg/basecamp/eventfeed/catchup_test.go | 2 +- scripts/check-known-defect-issues-open | 7 ++++++- scripts/test-check-known-defect-issues-open | 13 +++++++++++++ spec/tracking-issues.yml | 21 +++++++++++++++++++++ 7 files changed, 57 insertions(+), 7 deletions(-) diff --git a/SPEC.md b/SPEC.md index a58d17375..17a2cd73f 100644 --- a/SPEC.md +++ b/SPEC.md @@ -3615,7 +3615,7 @@ Two dispatch clarifications, pinned: (implementation-chosen; the Go reference uses 256). At capacity the pump **blocks** — back-pressure propagates to the socket and TCP — rather than dropping: the state-machine-owned live buffer is the only place a frame can ever be dropped, and its - overflow signal is the only drop signal. Worst-case connector memory is therefore + overflow signal is the only drop signal. Worst-case cable-lane retention is therefore bounded multiplicatively — every retained item is itself bounded by `EVENT_FEED_MAX_FRAME_BYTES`, and retention is an enumeration by HOLDER, which is what closes the count: a frame lives in the hand-off queue (≤ pump depth), in the live @@ -3643,6 +3643,17 @@ Two dispatch clarifications, pinned: the three counted structures or in the hands of the pump or the state machine, and each of those holders is already counted. + The formula is the cable lane's retention, and only that — every counted item is a + raw socket frame or a buffered live event. The poll lane sits outside it on purpose: + `PollSource.Poll` returns one page decoded whole, and the walk retains that page + until its rows are delivered. What bounds it is shape, not size: pages are fetched + sequentially, so a walk holds at most one live page (a superseded attempt's in-flight + poll may briefly hold another before its result is discarded), but the page's SIZE is + the server's pagination decision — `EVENT_FEED_MAX_FRAME_BYTES` governs socket + frames and says nothing about an HTTP body the generated layer decodes. A + total-connector memory bound would need a poll-page cap this contract deliberately + does not impose. + The drain's protocol-fatal scan is budgeted at `pump depth + 1` and not at this figure, which is not an inconsistency: the budget counts what the scan may DEQUEUE — the queue plus the pump's held frame — while the ceiling counts what may be RETAINED, and the deferral slot diff --git a/go/pkg/basecamp/eventfeed/buffer_test.go b/go/pkg/basecamp/eventfeed/buffer_test.go index 0ddd9fe1d..988ea1c5a 100644 --- a/go/pkg/basecamp/eventfeed/buffer_test.go +++ b/go/pkg/basecamp/eventfeed/buffer_test.go @@ -6,7 +6,7 @@ import ( ) // TestLiveBufferAddClearsEvictedSlots pins the eviction half of the live -// buffer's memory ceiling. SPEC.md §23 publishes the connector's worst case +// buffer's memory ceiling. SPEC.md §23 publishes the cable lane's worst case // as (pump depth + 3 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × EVENT_FEED_MAX_FRAME_BYTES; // a reslice alone removes the evicted event LOGICALLY while the slice that // results still points into the same backing array, whose prefix keeps that diff --git a/go/pkg/basecamp/eventfeed/catchup.go b/go/pkg/basecamp/eventfeed/catchup.go index 46f8d465c..d875ea9f8 100644 --- a/go/pkg/basecamp/eventfeed/catchup.go +++ b/go/pkg/basecamp/eventfeed/catchup.go @@ -916,7 +916,7 @@ func (l *loop) drain(at *attempt) (cycleOutcome, bool) { // one. Both halves matter. Scanning first is what keeps the // protocol-fatal carve-out ahead of every delivery; dequeuing singly is // what keeps the drain inside the live buffer's capacity, which is a - // bound on events held AT ONCE (SPEC.md §23 sizes the connector's whole + // bound on events held AT ONCE (SPEC.md §23 sizes the cable lane's whole // memory ceiling off it). Taking the buffer's whole contents into a batch // instead let the buffer read as empty while `capacity` events were still // pending in that batch, so the scan could admit another full capacity @@ -1116,8 +1116,8 @@ func (l *loop) probeFatal(at *attempt) (cycleOutcome, bool) { // is actually reported. // // This is why the scan needs no queue and no share of pumpDepth: it retains -// exactly what the single slot always retained. The connector's published -// memory bound — (pump depth + 3 + liveBufferCapacity) × MAX_FRAME_BYTES — is +// exactly what the single slot always retained. The published cable-lane +// retention bound — (pump depth + 3 + liveBufferCapacity) × MAX_FRAME_BYTES — is // untouched, and so is the depth at which the pump blocks. The slot IS one of // that formula's three raw-frame terms beyond the queue; the others are the // frame the pump has read and not yet handed off, and the frame the scan diff --git a/go/pkg/basecamp/eventfeed/catchup_test.go b/go/pkg/basecamp/eventfeed/catchup_test.go index 5fe1b1df8..ef8dd0794 100644 --- a/go/pkg/basecamp/eventfeed/catchup_test.go +++ b/go/pkg/basecamp/eventfeed/catchup_test.go @@ -1395,7 +1395,7 @@ func TestDrainScanAdmissionIsNotStranded(t *testing.T) { } // TestDrainHoldsNoMoreThanTheLiveBufferCapacity: the live buffer's capacity -// is a bound on events HELD AT ONCE — SPEC §23 sizes the connector's whole +// is a bound on events HELD AT ONCE — SPEC §23 sizes the cable lane's whole // memory ceiling off it, "(pump depth + 3 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × // EVENT_FEED_MAX_FRAME_BYTES" — so a drain must not be able to hold a batch // outside the buffer while the buffer refills to capacity behind it. diff --git a/scripts/check-known-defect-issues-open b/scripts/check-known-defect-issues-open index fc2c397ae..304951171 100755 --- a/scripts/check-known-defect-issues-open +++ b/scripts/check-known-defect-issues-open @@ -154,7 +154,12 @@ end # with a plain directory walk as the fallback for a sweep root that is not a # repository (the self-test corpus). Binary files are skipped by NUL sniff. -SWEEP_GAP = /(?:[ \t]|\r?\n[ \t]*(?:#(?!\d)|\/\/|\*+|--)?[ \t]*)+/ +# The comment leader is matched by CLASS, not by count: Swift/Kotlin doc +# comments are `///`, box comments stack `*`, and a leader that consumed an +# exact number of marks would leave the surplus mark in front of the next word +# and read a wrapped promise as absence. `#+` refuses a following digit so an +# issue reference is never eaten as a leader. +SWEEP_GAP = /(?:[ \t]|\r?\n[ \t]*(?:#+(?!\d)|\/{2,}|\*+|-{2,})?[ \t]*)+/ SWEEP_MENTION = /\btracked(?:#{SWEEP_GAP}separately)?#{SWEEP_GAP}in#{SWEEP_GAP}#(\d+)/i def sweep_files(root) diff --git a/scripts/test-check-known-defect-issues-open b/scripts/test-check-known-defect-issues-open index 50bb65b6e..a0c70d490 100755 --- a/scripts/test-check-known-defect-issues-open +++ b/scripts/test-check-known-defect-issues-open @@ -478,6 +478,19 @@ out, status, _ = run_checker( expect_fail(failures, "N+11. a line-wrapped canonical mention is still caught", out, status, "not registered") +# --- N+11a. The wrap is caught behind a Swift doc-comment leader too ------------- +# +# `///` is three slashes; a leader that consumes exactly two leaves the third +# in front of "separately" and reads the promise as absence. The leader must +# take comment marks by class, not by count. + +out, status, _ = run_checker( + allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, + sweep: { "Sources/Feed.swift" => "/// the six SDKs disagree and this is tracked\n/// separately in #909.\n" } +) +expect_fail(failures, "N+11a. a Swift doc-comment (///) wrapped mention is still caught", out, status, + "not registered") + # --- N+12. A row with no file fails ---------------------------------------------- # # The sweep keys coverage on [file, issue], so an entry without a file could diff --git a/spec/tracking-issues.yml b/spec/tracking-issues.yml index ea844f6b0..07b883654 100644 --- a/spec/tracking-issues.yml +++ b/spec/tracking-issues.yml @@ -52,3 +52,24 @@ prose_tracking_issues: - issue: 578 file: "python/tests/services/test_todolist_groups_service.py" site: "python/tests/services/test_todolist_groups_service.py — giving Python a structural decoder" + - issue: 578 + file: "typescript/tests/services/todolists.test.ts" + site: "typescript/tests/services/todolists.test.ts — structural safety for this SDK" + - issue: 578 + file: "ruby/lib/basecamp/services/merge_safe.rb" + site: "ruby/lib/basecamp/services/merge_safe.rb — the generated validating layer as the guards' intended end state" + - issue: 578 + file: "python/src/basecamp/services/_merge_safe.py" + site: "python/src/basecamp/services/_merge_safe.py — the generated validating layer as the guards' intended end state" + - issue: 578 + file: "typescript/src/services/merge-safe.ts" + site: "typescript/src/services/merge-safe.ts — the generated validating layer as the guards' intended end state" + - issue: 775 + file: "kotlin/sdk/src/commonMain/kotlin/com/basecamp/sdk/Pagination.kt" + site: "kotlin/sdk/src/commonMain/kotlin/com/basecamp/sdk/Pagination.kt — #775 carries the six-SDK Retry-After table" + - issue: 775 + file: "kotlin/sdk/src/commonTest/kotlin/com/basecamp/sdk/PaginationTest.kt" + site: "kotlin/sdk/src/commonTest/kotlin/com/basecamp/sdk/PaginationTest.kt — #775 carries the six-SDK Retry-After table" + - issue: 775 + file: "typescript/src/errors.ts" + site: "typescript/src/errors.ts — the strictness gate is recorded against the #775 divergence" From afd32f49eedeb206e863e805c55c35bb43c86729 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Sat, 22 Aug 2026 02:43:50 -0700 Subject: [PATCH 20/55] Absorb the merged base's tracking promises into the registry MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Merging origin/event-feed-go-connector brought SPEC §6's Retry-After decision (#793) and its follow-on text, and the discovery sweep caught the first consequence on its first run: SPEC.md:640 now says "Python and Ruby still truncate, tracked in #799" — a canonical claim with no registry entry. That is the gate working across a merge, not a defect. Reconciled against the merged tree: * Seven new entries for the promises the merge introduced: #799 (SPEC.md, MIGRATING.md, go/pkg/basecamp/client.go, client_retry_after_test.go — the rounding/over-range halves of the divergence), #798 (SPEC.md, MIGRATING.md — the generated parser stays Atoi until its template change lands), and one #775 the audit surfaced in client.go:704 (status-set widening is #775's). * Every pre-merge entry stays valid: each registered file still carries live sentences for its issue (SPEC.md's #775 sentences moved but did not go; MIGRATING.md:252 likewise). Audit on the merged tree, per the round-3 protocol extended to the two new issues: git grep -nI -e '#578' -e '#589' -e '#775' -e '#798' -e '#799' -- \ ':!spec/api-gaps' ':!spec/tracking-issues.yml' \ ':!scripts/test-check-known-defect-issues-open' 33 mention lines, residual ZERO unregistered [file, issue] pairs. Gates on the merged tree: both eventfeed Go packages green, plain and -race (the era-scoped occupancy scan and the AdvanceIfQuiet advance merged without conflict — disjoint hunks in the driver); tracking self-test 23/23; the live checker sweeps clean and verifies all 22 entries (#578, #589, #775, #798, #799 all open); doc-constants-check green under LC_ALL=C. --- spec/tracking-issues.yml | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/spec/tracking-issues.yml b/spec/tracking-issues.yml index 07b883654..9abb6baf6 100644 --- a/spec/tracking-issues.yml +++ b/spec/tracking-issues.yml @@ -73,3 +73,24 @@ prose_tracking_issues: - issue: 775 file: "typescript/src/errors.ts" site: "typescript/src/errors.ts — the strictness gate is recorded against the #775 divergence" + - issue: 799 + file: "SPEC.md" + site: "SPEC.md §6 — Python and Ruby still truncate the delay; #799 tracks the rounding convergence" + - issue: 799 + file: "MIGRATING.md" + site: "MIGRATING.md — #799 tracks the cross-SDK convergence on Retry-After rounding" + - issue: 775 + file: "go/pkg/basecamp/client.go" + site: "go/pkg/basecamp/client.go — widening the set of statuses that carry a parsed Retry-After is #775's" + - issue: 799 + file: "go/pkg/basecamp/client.go" + site: "go/pkg/basecamp/client.go — the rounding and over-range halves of the divergence are #799's" + - issue: 799 + file: "go/pkg/basecamp/client_retry_after_test.go" + site: "go/pkg/basecamp/client_retry_after_test.go — pins Go's side pending #799's convergence" + - issue: 798 + file: "SPEC.md" + site: "SPEC.md §6 — the generated Go parser stays Atoi until #798 lands the template change" + - issue: 798 + file: "MIGRATING.md" + site: "MIGRATING.md — the generated copy of the parse is untouched pending #798" From e74b487cc1d555cf6ece6596a659ef6f648526e0 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Sat, 22 Aug 2026 12:21:08 -0700 Subject: [PATCH 21/55] Say what DueWithin reads, type the registry's strings, split the egress claim MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot's fourth round: four findings, all taken. * DueWithin's doc claimed "exactly the set an Advance(d) would fire", and Advance fires more than that: it re-evaluates after each fire, so a timer armed reentrantly by a firing's recipient fires inside the window without ever appearing in DueWithin's answer — TestClock_AdvanceFiresATimerArmedByAFiringsRecipient demonstrates it. The doc now names what is actually read: the INITIAL due set, a floor when non-empty and exact when empty (no first firing means no reentrant arm), which is the asymmetry the advance guard stands on. AdvanceIfQuiet's return doc carried the same overclaim one step removed ("the due set") and now says "the initially due set" with the same reasoning. * The registry's `site` and `file` were validated through to_s, which coerces [] and {} and 123 into non-empty text — a row with no human-readable location reached the issue check, and a non-String `file` could never match the sweep's [file, issue] pair: a promise registered into a shape nothing reads. Both now require a non-blank String, with the offending value named in the error. Red-proven: the two new self-test cases were run against the previous checker first and both failed with "expected FAILURE but checker passed"; all 25 cases pass after the fix. * §23's zero-egress paragraph led with "Zero egress to the foreign origin is a Layer-1 obligation, not tier-2 coverage" — a subject that swept all three hostile-URL cases while its justification described only the redirect. Three review rounds read it the same wrong way, which makes the sentence the defect regardless of the defense: the family README (placeholder rule, rows 26/27) affirmatively requires tier-2 zero-request coverage for connector-visible hostile targets, and tier 2 does own it — a connector that follows one hands the URL to the poll seam, which the driver observes and fails. The paragraph now splits at the seam: fixtures 26/27's structural zero-request coverage is tier 2's, and only the redirect Location — which the driver reduces to an origin before the connector sees anything — is the Layer-1 obligation, recorded as pending in the row-15 note. This narrows the earlier decline of the same finding: the operative artifacts were never in conflict, but a sentence that needs this much defending three rounds running earns the rewrite. Gates: tracking-issues self-test 25/25; the live checker sweeps clean and verifies all 22 entries (five issues open); doc-constants-check green under LC_ALL=C; both eventfeed Go packages green; gofmt clean. --- SPEC.md | 25 ++++++++++++--------- go/pkg/basecamp/eventfeed/feedtest/clock.go | 13 +++++++++-- scripts/check-known-defect-issues-open | 15 ++++++++----- scripts/test-check-known-defect-issues-open | 21 +++++++++++++++++ 4 files changed, 56 insertions(+), 18 deletions(-) diff --git a/SPEC.md b/SPEC.md index 17a2cd73f..05b3fa008 100644 --- a/SPEC.md +++ b/SPEC.md @@ -4033,16 +4033,21 @@ further poll; store-failure coverage proves Failed(load) terminates with zero wi attempts and Failed(save) continues with the observer signal and a subsequent save attempt. -**Zero egress to the foreign origin is a Layer-1 obligation, not tier-2 coverage**, and -this paragraph used to require it here. Tier 2 cannot deliver it: the poll lane IS the -seam, so the driver reduces the `Location` to its origin and hands the connector a -refusal verdict. The connector never sees a `Location` and never decides whether to -follow one, which makes the foreign origin unreachable by construction of the harness — -a harness that asserted no request reached it would be asserting something about itself. -The obligation belongs to the Layer-1 seam adapter's own 302 test, where a real -generated `PollEvents` call meets a real redirect against an adapter with automatic -redirect-following disabled. `conformance/event-feed/README.md`'s row-15 note records it -as a pending obligation rather than a proof the repository contains. +**Zero egress to the foreign origin splits at the seam.** For the hostile `next` and +`resume` cases the target is connector-visible and tier 2 owns the coverage: a +connector that follows one hands the URL to the poll seam, which the driver observes +and fails — fixtures 26/27 assert zero requests to those hosts, structurally (no step +ever serves them, and the harness's servers own only their own origins). For the +redirect the obligation is Layer-1's, and this paragraph used to require it at tier 2: +the poll lane IS the seam, so the driver reduces the `Location` to its origin and +hands the connector a refusal verdict. The connector never sees a `Location` and never +decides whether to follow one, which makes the foreign origin unreachable by +construction of the harness — a harness that asserted no request reached it would be +asserting something about itself. That obligation belongs to the Layer-1 seam +adapter's own 302 test, where a real generated `PollEvents` call meets a real redirect +against an adapter with automatic redirect-following disabled. +`conformance/event-feed/README.md`'s row-15 note records it as a pending obligation +rather than a proof the repository contains. ### Clock, Timers, and Virtual Time `[conformance]` diff --git a/go/pkg/basecamp/eventfeed/feedtest/clock.go b/go/pkg/basecamp/eventfeed/feedtest/clock.go index 2794a62e7..3be277e5d 100644 --- a/go/pkg/basecamp/eventfeed/feedtest/clock.go +++ b/go/pkg/basecamp/eventfeed/feedtest/clock.go @@ -67,7 +67,13 @@ func (c *Clock) NewTimer(d time.Duration, name string) eventfeed.Timer { } // DueWithin returns the names of live timers due within d of the current -// virtual time — exactly the set an Advance(d) would fire — in creation order. +// virtual time — the INITIAL set an Advance(d) would fire, read from the +// clock's present state — in creation order. It is not a complete firing +// prediction: Advance re-evaluates after each fire, so a timer armed +// reentrantly by a firing's recipient can fire inside the same window without +// ever appearing here (TestClock_AdvanceFiresATimerArmedByAFiringsRecipient +// shows one). The asymmetry is what makes the empty answer exact and the +// non-empty answer a floor. // // Read under the same lock advance selects under and NewTimer arms under, so // the answer is atomic with respect to both. That is what lets a caller turn a @@ -96,7 +102,10 @@ func (c *Clock) dueWithinLocked(target time.Time) []string { } // AdvanceIfQuiet advances virtual time by d only if the window would fire -// nothing; otherwise it reports the due set and leaves the clock untouched. +// nothing; otherwise it reports the INITIALLY due set — the same floor +// DueWithin reads, sufficient here because rejection needs only "non-empty", +// and an empty initial set means nothing fires at all — and leaves the clock +// untouched. // It is DueWithin and Advance as ONE critical section, for the driver MUST // in SPEC §23: an advance whose window would fire any timer is rejected. // Deciding that with two separate lock acquisitions leaves a gap — a timer diff --git a/scripts/check-known-defect-issues-open b/scripts/check-known-defect-issues-open index 304951171..e244fa819 100755 --- a/scripts/check-known-defect-issues-open +++ b/scripts/check-known-defect-issues-open @@ -126,14 +126,17 @@ rows.each_with_index do |e, i| warn "ERROR: prose_tracking_issues[#{i}] in #{tracking_path} needs a positive integer `issue`, got #{e['issue'].inspect}" exit 1 end - if e['site'].to_s.strip.empty? - warn "ERROR: prose_tracking_issues[#{i}] (##{e['issue']}) in #{tracking_path} needs a `site` naming " \ - 'the sentence that leans on it — a bare number tells the next reader nothing to go fix.' + # String-typed, not merely stringifiable: to_s coerces [] and {} and 123 + # into non-empty text, which would register a promise into a shape nothing + # reads — the sweep's [file, issue] pair can never match a non-String file. + unless e['site'].is_a?(String) && !e['site'].strip.empty? + warn "ERROR: prose_tracking_issues[#{i}] (##{e['issue']}) in #{tracking_path} needs a `site` — a non-blank " \ + 'string naming the sentence that leans on it — got ' + e['site'].inspect exit 1 end - if e['file'].to_s.strip.empty? - warn "ERROR: prose_tracking_issues[#{i}] (##{e['issue']}) in #{tracking_path} needs a `file` naming " \ - 'the path the sentence lives in — the discovery sweep keys coverage on [file, issue].' + unless e['file'].is_a?(String) && !e['file'].strip.empty? + warn "ERROR: prose_tracking_issues[#{i}] (##{e['issue']}) in #{tracking_path} needs a `file` — a non-blank " \ + 'string naming the path the sentence lives in (the sweep keys coverage on [file, issue]) — got ' + e['file'].inspect exit 1 end refs << [e['issue'], e['site']] diff --git a/scripts/test-check-known-defect-issues-open b/scripts/test-check-known-defect-issues-open index a0c70d490..c7a02c64e 100755 --- a/scripts/test-check-known-defect-issues-open +++ b/scripts/test-check-known-defect-issues-open @@ -403,6 +403,27 @@ out, status, _ = run_checker( ) expect_fail(failures, "N+6. a non-list registry fails", out, status, "must be a list") +# --- N+6a/b. site and file must be Strings, not merely stringifiable ------------- +# +# `to_s` coerces [] and {} and 123 into non-empty text, so a malformed row +# reached the issue check with no human-readable location. The row IS the +# claim: a value the sweep's [file, issue] pair can never match is a promise +# registered into a shape nothing reads. + +out, status, _ = run_checker( + allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, + tracking: { "prose_tracking_issues" => [ { "issue" => 303, "site" => 123, "file" => "README.md" } ] }, + states: "303=open" +) +expect_fail(failures, "N+6a. a non-string site fails", out, status, "needs a `site`") + +out, status, _ = run_checker( + allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, + tracking: { "prose_tracking_issues" => [ { "issue" => 303, "site" => "README.md — a gap", "file" => [ "README.md" ] } ] }, + states: "303=open" +) +expect_fail(failures, "N+6b. a non-string file fails", out, status, "needs a `file`") + # --- N+7. A registry file without the key fails ----------------------------------- # # The missing-FILE case (N+3) fails closed; a file whose key is misspelled or From 2367dab4a1da4083041e14a41a0ee7a1f9aaed2b Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Sat, 22 Aug 2026 12:41:25 -0700 Subject: [PATCH 22/55] Weigh the decode copy in the ceiling, and retire the no-op rationale MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot's fifth round: two findings, both taken. * The by-holder ceiling missed the codec's copy. parseFrame unmarshals a message frame's payload into a json.RawMessage — a copy — while the original bytes are still in hand, so the state machine's in-hand term briefly costs two frame-sized allocations, not one, for a near-limit message. Verified at head: all four parseFrame call sites are state-machine side (catchup.go, loop.go dispatch), so the pump's term is untouched and the copy is not a fourth party — it is a WEIGHT on a holder the enumeration already counts, which is why the closure argument survives: each holder is now counted at its worst-case weight. The ceiling becomes (pump depth + 4 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × MAX_FRAME_BYTES, the +4 being the deferral slot, the pump's in-hand frame, and the state machine's in-hand frame at its decode-time weight of two — the copy transient and bounded by one extra frame. The catchup.go mirror and both test comments carry the same figure. * The gate's "no-op when the allowlist references no issues, which is today's state" rationale predated the seeded registry: every run now makes live lookups regardless of the allowlist. The script header, the Makefile's test-target comment, and the workflow step comment now say what is true — a green live run proves only that today's referenced issues are open, and the offline self-test is what covers the failure branches (closed issues, malformed rows, the fail-closed paths) a green run never exercises. Gates: both eventfeed Go packages green, gofmt clean; tracking-issues self-test 25/25; the live checker sweeps clean and verifies all 22 entries (five issues open); doc-constants-check green under LC_ALL=C. --- .github/workflows/test.yml | 5 +++-- Makefile | 7 +++--- SPEC.md | 27 ++++++++++++++--------- go/pkg/basecamp/eventfeed/buffer_test.go | 2 +- go/pkg/basecamp/eventfeed/catchup.go | 14 +++++++----- go/pkg/basecamp/eventfeed/catchup_test.go | 2 +- scripts/check-known-defect-issues-open | 6 ++++- 7 files changed, 39 insertions(+), 24 deletions(-) diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index f6845714a..8c0855385 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -134,8 +134,9 @@ jobs: with: ruby-version: "3.3" - # Offline and first: the gate below is a no-op whenever the allowlist - # references no issues, so this is what proves it rejects anything. + # Offline and first: a green run of the live gate below proves only that + # today's referenced issues are open, so this is what proves it REJECTS + # anything — closed issues, malformed rows, the fail-closed paths. - name: Self-test the known-defect gate run: make test-check-known-defect-issues-open diff --git a/Makefile b/Makefile index 4e0371166..fec2599a9 100644 --- a/Makefile +++ b/Makefile @@ -156,9 +156,10 @@ check-known-defect-issues-open: @echo "==> Checking known-defect tracking issues are open..." @./scripts/check-known-defect-issues-open -# Drive that gate from outside. Its live run is a no-op today — the allowlist -# references no issues — so without this NOTHING exercises the closed-issue -# rejection, the fail-closed path, or the second reference shape. Offline: PATH +# Drive that gate from outside. Its live run verifies whatever the allowlist +# and registry currently reference, and a green run proves only that those +# issues are open — so without this NOTHING exercises the closed-issue +# rejection, the fail-closed paths, or the second reference shape. Offline: PATH # is stripped to a stub `gh` answering from a canned table, because a self-test # that asked GitHub would assert against whatever is true this morning. # diff --git a/SPEC.md b/SPEC.md index 05b3fa008..f045a7ba0 100644 --- a/SPEC.md +++ b/SPEC.md @@ -3620,28 +3620,35 @@ Two dispatch clarifications, pinned: `EVENT_FEED_MAX_FRAME_BYTES`, and retention is an enumeration by HOLDER, which is what closes the count: a frame lives in the hand-off queue (≤ pump depth), in the live buffer (≤ `EVENT_FEED_LIVE_BUFFER_CAPACITY`), in the single deferral slot (≤ 1), or in - the hands of one of the exactly two goroutines that touch frames — one in-hand frame - each. The ceiling is - (pump depth + 3 + `EVENT_FEED_LIVE_BUFFER_CAPACITY`) × `EVENT_FEED_MAX_FRAME_BYTES` + the hands of one of the exactly two goroutines that touch frames. The pump's hand + holds one frame; the state machine's weighs up to TWO frame-sized allocations while a + message frame is decoded, because `parseFrame` unmarshals the payload into a + `json.RawMessage` — a copy — while the original bytes are still in hand. The ceiling + is + (pump depth + 4 + `EVENT_FEED_LIVE_BUFFER_CAPACITY`) × `EVENT_FEED_MAX_FRAME_BYTES` (≈ 10 GiB at the defaults' extreme, reached only if every slot holds a maximum-size - frame) — even under a slow consumer. The **+ 3** is three raw frames the queue's depth - does not count — the deferral slot plus one in-hand frame per frame-touching party — - and they are retained by different parties at the same time: + frame) — even under a slow consumer. The **+ 4** is four frame-sized allocations the + queue's depth does not count — the deferral slot, the pump's in-hand frame, and the + state machine's in-hand frame at its decode-time weight of two — retained by + different parties at the same time: - the **pump's own in-flight frame** — the pump is a single reader, so it may hold exactly one frame it has already READ and not yet handed off. One rather than an unbounded number for that reason: one reader holds at most one frame outside the queue. - the **state machine's in-hand frame** — the protocol-fatal scan's dequeue is the very receive that lets a blocked pump refill the queue, so while the scan still holds that frame — examining, admitting, or parking it — the queue is full again and the pump may - already hold its next read. A single consumer, so one frame, for the pump's own reason. + already hold its next read. A single consumer, so one frame in hand, for the pump's + own reason — weighted at two because decoding it can briefly double it: the + `json.RawMessage` copy is transient and bounded by one extra frame, and it lives in + this holder's hands, not in a fourth party's. - the **deferred socket outcome** — the single slot the in-flight-poll servicing and the drain's scan park one receive in. It is retained while the queue behind it refills, so it is concurrent with a full queue and with both in-hand frames, not an alternative to any of them. - The enumeration cannot grow by a further party being noticed: every frame is in one of - the three counted structures or in the hands of the pump or the state machine, and each - of those holders is already counted. + The enumeration cannot grow by a further party being noticed: every frame-sized + allocation is in one of the three counted structures or in the hands of the pump or + the state machine, and each holder is counted at its worst-case weight. The formula is the cable lane's retention, and only that — every counted item is a raw socket frame or a buffered live event. The poll lane sits outside it on purpose: diff --git a/go/pkg/basecamp/eventfeed/buffer_test.go b/go/pkg/basecamp/eventfeed/buffer_test.go index 988ea1c5a..e99a42420 100644 --- a/go/pkg/basecamp/eventfeed/buffer_test.go +++ b/go/pkg/basecamp/eventfeed/buffer_test.go @@ -7,7 +7,7 @@ import ( // TestLiveBufferAddClearsEvictedSlots pins the eviction half of the live // buffer's memory ceiling. SPEC.md §23 publishes the cable lane's worst case -// as (pump depth + 3 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × EVENT_FEED_MAX_FRAME_BYTES; +// as (pump depth + 4 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × EVENT_FEED_MAX_FRAME_BYTES; // a reslice alone removes the evicted event LOGICALLY while the slice that // results still points into the same backing array, whose prefix keeps that // event's strings reachable until a later reallocation. Under sustained diff --git a/go/pkg/basecamp/eventfeed/catchup.go b/go/pkg/basecamp/eventfeed/catchup.go index d875ea9f8..8d787e319 100644 --- a/go/pkg/basecamp/eventfeed/catchup.go +++ b/go/pkg/basecamp/eventfeed/catchup.go @@ -1117,13 +1117,15 @@ func (l *loop) probeFatal(at *attempt) (cycleOutcome, bool) { // // This is why the scan needs no queue and no share of pumpDepth: it retains // exactly what the single slot always retained. The published cable-lane -// retention bound — (pump depth + 3 + liveBufferCapacity) × MAX_FRAME_BYTES — is +// retention bound — (pump depth + 4 + liveBufferCapacity) × MAX_FRAME_BYTES — is // untouched, and so is the depth at which the pump blocks. The slot IS one of -// that formula's three raw-frame terms beyond the queue; the others are the -// frame the pump has read and not yet handed off, and the frame the scan -// itself has dequeued and not yet disposed of — the very receive that lets a -// blocked pump refill the queue behind it. All three are retained WHILE the -// queue is full, which is why they are addends and not alternatives. +// that formula's frame-sized terms beyond the queue; the others are the frame +// the pump has read and not yet handed off, and the frame the scan itself has +// dequeued and not yet disposed of — the very receive that lets a blocked +// pump refill the queue behind it, counted at TWO, its decode-time weight, +// because parseFrame's json.RawMessage payload copy briefly coexists with the +// original bytes. All are retained WHILE the queue is full, which is why they +// are addends and not alternatives. func (l *loop) deferForDrain(d *deferredFrame) { if l.deferred == nil { l.deferred = d diff --git a/go/pkg/basecamp/eventfeed/catchup_test.go b/go/pkg/basecamp/eventfeed/catchup_test.go index ef8dd0794..478c62b0d 100644 --- a/go/pkg/basecamp/eventfeed/catchup_test.go +++ b/go/pkg/basecamp/eventfeed/catchup_test.go @@ -1396,7 +1396,7 @@ func TestDrainScanAdmissionIsNotStranded(t *testing.T) { // TestDrainHoldsNoMoreThanTheLiveBufferCapacity: the live buffer's capacity // is a bound on events HELD AT ONCE — SPEC §23 sizes the cable lane's whole -// memory ceiling off it, "(pump depth + 3 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × +// memory ceiling off it, "(pump depth + 4 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × // EVENT_FEED_MAX_FRAME_BYTES" — so a drain must not be able to hold a batch // outside the buffer while the buffer refills to capacity behind it. // diff --git a/scripts/check-known-defect-issues-open b/scripts/check-known-defect-issues-open index e244fa819..8ff059578 100755 --- a/scripts/check-known-defect-issues-open +++ b/scripts/check-known-defect-issues-open @@ -43,7 +43,11 @@ # fails rather than passing quietly, because an unverifiable "we know" claim is # indistinguishable from an untracked defect. # -# No-op when the allowlist references no issues at all, which is today's state. +# Every run verifies the registry's claims live (and the allowlist's, when it +# references issues). A green live run proves only that today's referenced +# issues are open; the offline self-test is what covers the failure branches — +# closed issues, malformed rows, the fail-closed paths — that a green run +# never exercises. # # Usage: ./check-known-defect-issues-open # BC3_ROUTE_ALLOWLIST=... KNOWN_DEFECT_ISSUE_REPO=... \ From 42ff0598a4969beb528c9cebcc752c7b7d505ad9 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Sat, 22 Aug 2026 13:07:03 -0700 Subject: [PATCH 23/55] Close the ceiling at the decode chain's length, and diagnose bad YAML shapes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot's sixth round. The ceiling finding reached its third increment (+3, +4, now +5), which per AGENTS.md is the signal to reassess the instrument rather than bump the number again — so the number is now DERIVED instead of discovered. * The whole decode chain was read end to end: a message frame exists in the state machine's hands as exactly THREE frame-scale representations — the wire bytes (still in scope at every decodeMessageEvent call site: catchup.go dispatch arms, loop.go), parseFrame's json.RawMessage payload copy, and the decoded Event's strings — each step's output allocated while its input is still in hand, and an adversarial near-limit payload can put frame-scale text in any of them. The state machine's weight IS that representation count: a fourth term requires CODING a fourth representation into the chain, not noticing one, which is what ends the increment series. The ceiling becomes (pump depth + 5 + capacity) × MAX_FRAME_BYTES, with the closure statement in SPEC and the catchup.go mirror, and both test comments at the same figure. * Verified for the buffer term, and stated: a buffered Event retains only the chain's LAST representation — Go's decoder never aliases its input buffer, so the Event's strings are copies, and Event carries no raw-bytes field — so capacity × MAX stands and admission sheds the first two representations. * A registry or allowlist file whose YAML top level is not a mapping ([] or a bare string) crashed the checker at .key? / Array#[String] with a backtrace — nonzero, but a backtrace is not a diagnosis. Both loads now validate the top-level type with the gate's own error. Red-proven: the two new self-test cases were run against the previous checker first; the registry case failed with the literal "checker:115: undefined method 'key?' for an instance of Array (NoMethodError)" and no diagnostic. All 27 cases pass after the fix. Gates: tracking-issues self-test 27/27; the sweep half of the live checker is clean on this tree, and the five issue lookups failed CLOSED twice on GitHub's shared-quota 403 — the same 22 entries verified open on the last quota-healthy run, and the next green run covers this tree; doc-constants-check green under LC_ALL=C; both eventfeed Go packages green; gofmt clean. --- SPEC.md | 31 +++++++++++++-------- go/pkg/basecamp/eventfeed/buffer_test.go | 2 +- go/pkg/basecamp/eventfeed/catchup.go | 11 ++++---- go/pkg/basecamp/eventfeed/catchup_test.go | 2 +- scripts/check-known-defect-issues-open | 10 +++++++ scripts/test-check-known-defect-issues-open | 18 ++++++++++++ 6 files changed, 56 insertions(+), 18 deletions(-) diff --git a/SPEC.md b/SPEC.md index f045a7ba0..2a1dc40d1 100644 --- a/SPEC.md +++ b/SPEC.md @@ -3621,15 +3621,17 @@ Two dispatch clarifications, pinned: closes the count: a frame lives in the hand-off queue (≤ pump depth), in the live buffer (≤ `EVENT_FEED_LIVE_BUFFER_CAPACITY`), in the single deferral slot (≤ 1), or in the hands of one of the exactly two goroutines that touch frames. The pump's hand - holds one frame; the state machine's weighs up to TWO frame-sized allocations while a - message frame is decoded, because `parseFrame` unmarshals the payload into a - `json.RawMessage` — a copy — while the original bytes are still in hand. The ceiling - is - (pump depth + 4 + `EVENT_FEED_LIVE_BUFFER_CAPACITY`) × `EVENT_FEED_MAX_FRAME_BYTES` + holds one frame. The state machine's holds up to THREE frame-sized allocations at + once, and three is not a discovered constant but the decode chain's REPRESENTATION + COUNT: a message frame exists as the wire bytes, as `parseFrame`'s `json.RawMessage` + payload copy, and as the decoded `Event`'s strings — each step's output allocated + while its input is still in hand, and an adversarial near-limit payload can put + frame-scale text in any of the three. The ceiling is + (pump depth + 5 + `EVENT_FEED_LIVE_BUFFER_CAPACITY`) × `EVENT_FEED_MAX_FRAME_BYTES` (≈ 10 GiB at the defaults' extreme, reached only if every slot holds a maximum-size - frame) — even under a slow consumer. The **+ 4** is four frame-sized allocations the + frame) — even under a slow consumer. The **+ 5** is five frame-sized allocations the queue's depth does not count — the deferral slot, the pump's in-hand frame, and the - state machine's in-hand frame at its decode-time weight of two — retained by + state machine's in-hand frame at its decode-chain weight of three — retained by different parties at the same time: - the **pump's own in-flight frame** — the pump is a single reader, so it may hold exactly one frame it has already READ and not yet handed off. One rather than an unbounded @@ -3638,9 +3640,11 @@ Two dispatch clarifications, pinned: receive that lets a blocked pump refill the queue, so while the scan still holds that frame — examining, admitting, or parking it — the queue is full again and the pump may already hold its next read. A single consumer, so one frame in hand, for the pump's - own reason — weighted at two because decoding it can briefly double it: the - `json.RawMessage` copy is transient and bounded by one extra frame, and it lives in - this holder's hands, not in a fourth party's. + own reason — weighted at the decode chain's length of three: wire bytes, + `RawMessage` copy, decoded `Event`, each transient past its step and every one in + this holder's hands, never a further party's. A fourth term here requires CODING a + fourth representation into the chain, not noticing one — which is what closes the + count. - the **deferred socket outcome** — the single slot the in-flight-poll servicing and the drain's scan park one receive in. It is retained while the queue behind it refills, so it is concurrent with a full queue and with both in-hand frames, not an alternative to @@ -3648,7 +3652,12 @@ Two dispatch clarifications, pinned: The enumeration cannot grow by a further party being noticed: every frame-sized allocation is in one of the three counted structures or in the hands of the pump or - the state machine, and each holder is counted at its worst-case weight. + the state machine, each holder counted at its worst-case weight — the pump at one + (it never parses), the state machine at the decode chain's three. The live buffer's + own weight stays one per slot: a buffered `Event` retains only the chain's LAST + representation — its strings are copies, since Go's decoder never aliases its input + buffer, and `Event` carries no raw-bytes field — so nothing of the first two + representations survives admission. The formula is the cable lane's retention, and only that — every counted item is a raw socket frame or a buffered live event. The poll lane sits outside it on purpose: diff --git a/go/pkg/basecamp/eventfeed/buffer_test.go b/go/pkg/basecamp/eventfeed/buffer_test.go index e99a42420..b549e7577 100644 --- a/go/pkg/basecamp/eventfeed/buffer_test.go +++ b/go/pkg/basecamp/eventfeed/buffer_test.go @@ -7,7 +7,7 @@ import ( // TestLiveBufferAddClearsEvictedSlots pins the eviction half of the live // buffer's memory ceiling. SPEC.md §23 publishes the cable lane's worst case -// as (pump depth + 4 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × EVENT_FEED_MAX_FRAME_BYTES; +// as (pump depth + 5 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × EVENT_FEED_MAX_FRAME_BYTES; // a reslice alone removes the evicted event LOGICALLY while the slice that // results still points into the same backing array, whose prefix keeps that // event's strings reachable until a later reallocation. Under sustained diff --git a/go/pkg/basecamp/eventfeed/catchup.go b/go/pkg/basecamp/eventfeed/catchup.go index 8d787e319..23a0f15e9 100644 --- a/go/pkg/basecamp/eventfeed/catchup.go +++ b/go/pkg/basecamp/eventfeed/catchup.go @@ -1117,15 +1117,16 @@ func (l *loop) probeFatal(at *attempt) (cycleOutcome, bool) { // // This is why the scan needs no queue and no share of pumpDepth: it retains // exactly what the single slot always retained. The published cable-lane -// retention bound — (pump depth + 4 + liveBufferCapacity) × MAX_FRAME_BYTES — is +// retention bound — (pump depth + 5 + liveBufferCapacity) × MAX_FRAME_BYTES — is // untouched, and so is the depth at which the pump blocks. The slot IS one of // that formula's frame-sized terms beyond the queue; the others are the frame // the pump has read and not yet handed off, and the frame the scan itself has // dequeued and not yet disposed of — the very receive that lets a blocked -// pump refill the queue behind it, counted at TWO, its decode-time weight, -// because parseFrame's json.RawMessage payload copy briefly coexists with the -// original bytes. All are retained WHILE the queue is full, which is why they -// are addends and not alternatives. +// pump refill the queue behind it, counted at THREE, the decode chain's +// representation count: wire bytes, parseFrame's json.RawMessage payload +// copy, and the decoded Event's strings, each step's output allocated while +// its input is still in hand. All are retained WHILE the queue is full, which +// is why they are addends and not alternatives. func (l *loop) deferForDrain(d *deferredFrame) { if l.deferred == nil { l.deferred = d diff --git a/go/pkg/basecamp/eventfeed/catchup_test.go b/go/pkg/basecamp/eventfeed/catchup_test.go index 478c62b0d..997179986 100644 --- a/go/pkg/basecamp/eventfeed/catchup_test.go +++ b/go/pkg/basecamp/eventfeed/catchup_test.go @@ -1396,7 +1396,7 @@ func TestDrainScanAdmissionIsNotStranded(t *testing.T) { // TestDrainHoldsNoMoreThanTheLiveBufferCapacity: the live buffer's capacity // is a bound on events HELD AT ONCE — SPEC §23 sizes the cable lane's whole -// memory ceiling off it, "(pump depth + 4 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × +// memory ceiling off it, "(pump depth + 5 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × // EVENT_FEED_MAX_FRAME_BYTES" — so a drain must not be able to hold a batch // outside the buffer while the buffer refills to capacity behind it. // diff --git a/scripts/check-known-defect-issues-open b/scripts/check-known-defect-issues-open index 8ff059578..c520beab2 100755 --- a/scripts/check-known-defect-issues-open +++ b/scripts/check-known-defect-issues-open @@ -76,6 +76,10 @@ unless File.exist?(allow_path) exit 1 end allow = YAML.safe_load(File.read(allow_path)) || {} +unless allow.is_a?(Hash) + warn "ERROR: #{allow_path} must be a YAML mapping of route lists, got #{allow.class}" + exit 1 +end # REQUIRED, not optional. Treating an absent registry as an empty one is a # fail-OPEN path in a gate whose entire purpose is failing closed: delete the @@ -90,6 +94,12 @@ unless File.exist?(tracking_path) exit 1 end tracking = YAML.safe_load(File.read(tracking_path)) || {} +# A non-mapping top level ([] or a bare string) would crash on .key? below — +# nonzero, but a backtrace is not a diagnosis. Same fail-closed, better words. +unless tracking.is_a?(Hash) + warn "ERROR: #{tracking_path} must be a YAML mapping with a prose_tracking_issues list, got #{tracking.class}" + exit 1 +end # (issue number, human-readable site) pairs, so a failure names the entry that # has to change rather than just the number. diff --git a/scripts/test-check-known-defect-issues-open b/scripts/test-check-known-defect-issues-open index c7a02c64e..59b77af84 100755 --- a/scripts/test-check-known-defect-issues-open +++ b/scripts/test-check-known-defect-issues-open @@ -424,6 +424,24 @@ out, status, _ = run_checker( ) expect_fail(failures, "N+6b. a non-string file fails", out, status, "needs a `file`") +# --- N+6c/d. A non-mapping YAML top level fails with the gate's own words --------- +# +# YAML.safe_load of "- x" returns an Array, and Array#key? / Array#[String] +# raise — the gate exited nonzero, but through a backtrace rather than a +# diagnosis, telling the next reader nothing about which file is malformed or +# how. Fail closed WITH the diagnostics. + +out, status, _ = run_checker( + allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, + tracking: [ "not", "a", "mapping" ] +) +expect_fail(failures, "N+6c. a non-mapping registry file fails with a diagnosis", out, status, + "must be a YAML mapping") + +out, status, _ = run_checker(allowlist: [ "not", "a", "mapping" ]) +expect_fail(failures, "N+6d. a non-mapping allowlist fails with a diagnosis", out, status, + "must be a YAML mapping") + # --- N+7. A registry file without the key fails ----------------------------------- # # The missing-FILE case (N+3) fails closed; a file whose key is misspelled or From 4d5382fc62b1068530dbf8c4c06bb5af4b3680ce Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Mon, 24 Aug 2026 01:15:36 -0700 Subject: [PATCH 24/55] Give unparseable YAML the gate's own diagnosis MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot's seventh-round suppressed finding: YAML.safe_load raises Psych::SyntaxError for malformed syntax, so a broken registry or allowlist still exited through a Ruby backtrace naming psych/parser.rb rather than the file to fix — fail-closed, but useless. Both loads now rescue Psych's exception family (syntax errors, forbidden tags, and the aliases safe_load refuses are all the same "this file cannot be trusted" verdict) and fail with the gate's message naming the path. Red-proven: the two new self-test cases were run against the previous checker first and both failed through the literal Psych::SyntaxError backtrace; all 29 cases pass after the fix. The self-test harness gains raw-bytes overrides for both inputs, since YAML.dump can never produce the malformed file the cases need. The round's other two findings are handled outside this commit: the G1b ownership claim needs an issue filed before it can be cited and registered (reported to the coordinator with the prepared rewrite), and the PR description correction is the coordinator's to apply. Gates: tracking-issues self-test 29/29; live checker green (sweep clean, all 22 entries verified, five issues open); doc-constants-check green under LC_ALL=C. --- scripts/check-known-defect-issues-open | 18 ++++++++++++-- scripts/test-check-known-defect-issues-open | 27 ++++++++++++++++++--- 2 files changed, 40 insertions(+), 5 deletions(-) diff --git a/scripts/check-known-defect-issues-open b/scripts/check-known-defect-issues-open index c520beab2..18e3f7a7d 100755 --- a/scripts/check-known-defect-issues-open +++ b/scripts/check-known-defect-issues-open @@ -75,7 +75,16 @@ unless File.exist?(allow_path) warn "ERROR: allowlist not found at #{allow_path}" exit 1 end -allow = YAML.safe_load(File.read(allow_path)) || {} +# Psych's exception family — syntax errors, forbidden tags, aliases (which +# safe_load refuses) — is the same fail-closed verdict as a malformed shape, +# and gets the same treatment: the gate's own words, naming the file to fix, +# never a psych.rb backtrace. +begin + allow = YAML.safe_load(File.read(allow_path)) || {} +rescue Psych::Exception => e + warn "ERROR: #{allow_path} cannot be parsed as YAML: #{e.message}" + exit 1 +end unless allow.is_a?(Hash) warn "ERROR: #{allow_path} must be a YAML mapping of route lists, got #{allow.class}" exit 1 @@ -93,7 +102,12 @@ unless File.exist?(tracking_path) warn ' from one whose claims are all unverified. Restore it, or empty its list.' exit 1 end -tracking = YAML.safe_load(File.read(tracking_path)) || {} +begin + tracking = YAML.safe_load(File.read(tracking_path)) || {} +rescue Psych::Exception => e + warn "ERROR: #{tracking_path} cannot be parsed as YAML: #{e.message}" + exit 1 +end # A non-mapping top level ([] or a bare string) would crash on .key? below — # nonzero, but a backtrace is not a diagnosis. Same fail-closed, better words. unless tracking.is_a?(Hash) diff --git a/scripts/test-check-known-defect-issues-open b/scripts/test-check-known-defect-issues-open index 59b77af84..56651ed67 100755 --- a/scripts/test-check-known-defect-issues-open +++ b/scripts/test-check-known-defect-issues-open @@ -99,7 +99,7 @@ failures = [] # case gets a corpus — empty by default — so the sweep can never read the REAL # repository against a synthetic registry, which would fail every case here on # facts about the tree rather than about the checker. -def run_checker(allowlist:, tracking: { "prose_tracking_issues" => [] }, sweep: {}, states: nil, gh_fail: false, repo: nil, github_repository: nil) +def run_checker(allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, allowlist_raw: nil, tracking: { "prose_tracking_issues" => [] }, tracking_raw: nil, sweep: {}, states: nil, gh_fail: false, repo: nil, github_repository: nil) Dir.mktmpdir("known-defect-selftest") do |dir| path_dir = File.join(dir, "bin") Dir.mkdir(path_dir) @@ -112,7 +112,9 @@ def run_checker(allowlist:, tracking: { "prose_tracking_issues" => [] }, sweep: end allow_path = File.join(dir, "bc3-route-allowlist.yml") - File.write(allow_path, YAML.dump(allowlist)) + # allowlist_raw writes bytes verbatim — the malformed-SYNTAX cases need a + # file YAML.dump could never produce. + File.write(allow_path, allowlist_raw || YAML.dump(allowlist)) # Pointed at a temp file even when the case supplies nothing, so the # checker can never read the repository's REAL registry mid-self-test — @@ -121,7 +123,12 @@ def run_checker(allowlist:, tracking: { "prose_tracking_issues" => [] }, sweep: tracking_path = File.join(dir, "tracking-issues.yml") # nil means "do not create the file", which is how the missing-registry # case is expressed without deleting the repository's real one. - File.write(tracking_path, YAML.dump(tracking)) unless tracking.nil? + # tracking_raw writes bytes verbatim, for the malformed-syntax case. + if tracking_raw + File.write(tracking_path, tracking_raw) + elsif !tracking.nil? + File.write(tracking_path, YAML.dump(tracking)) + end sweep_root = File.join(dir, "corpus") Dir.mkdir(sweep_root) @@ -442,6 +449,20 @@ out, status, _ = run_checker(allowlist: [ "not", "a", "mapping" ]) expect_fail(failures, "N+6d. a non-mapping allowlist fails with a diagnosis", out, status, "must be a YAML mapping") +# --- N+6e/f. YAML that does not PARSE fails with the gate's words too ------------ +# +# Psych::SyntaxError (and safe_load's tag/alias refusals) exited through a +# backtrace: nonzero, fail-closed, and useless — naming a psych.rb line rather +# than the file to fix. Same verdict, the gate's own diagnosis. + +out, status, _ = run_checker(tracking_raw: "prose_tracking_issues: [unclosed\n") +expect_fail(failures, "N+6e. unparseable registry YAML fails with a diagnosis", out, status, + "cannot be parsed as YAML") + +out, status, _ = run_checker(allowlist_raw: "sdk_routes_known_defective: [unclosed\n") +expect_fail(failures, "N+6f. unparseable allowlist YAML fails with a diagnosis", out, status, + "cannot be parsed as YAML") + # --- N+7. A registry file without the key fails ----------------------------------- # # The missing-FILE case (N+3) fails closed; a file whose key is misspelled or From b042239cae3d3070f3adabb3003c3c57f41e0b70 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Mon, 24 Aug 2026 01:21:27 -0700 Subject: [PATCH 25/55] Cite #819 where the prose said G1b, and let the gate hold it open MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Layer-1 seam-adapter obligation — the 302 redirect-refusal test that fixture 30's corrections assign below the poll seam — was cited five times as "tracked for G1b", a program-milestone label defined nowhere in this repository and linked to no issue: undiscoverable, and invisible to the tracking registry. #819 now exists and owns it. All five spans (README rows 15 and 30, the row-15 note, fixture 30's description, schema.json's fixture-30 respond variant) now say "tracked in #819" in the canonical grammar, and the three files carry registry rows, so the discovery sweep enforces the citations and the gate fails the day #819 closes without the adapters landing — which is the forgettable-silently failure the review named. Gates: the live checker verifies #819 open in all three rows and reports all 6 referenced tracking issues open; tracking-issues self-test 29/29; event-feed-fixtures-check green including both pin probes (fixture 30 and schema.json changed); doc-constants-check green under LC_ALL=C. --- conformance/event-feed/README.md | 6 +++--- .../fixtures/30-continuation-redirect-cross-origin.json | 3 +-- conformance/event-feed/schema.json | 2 +- spec/tracking-issues.yml | 9 +++++++++ 4 files changed, 14 insertions(+), 6 deletions(-) diff --git a/conformance/event-feed/README.md b/conformance/event-feed/README.md index 0c995fa2e..a50b31320 100644 --- a/conformance/event-feed/README.md +++ b/conformance/event-feed/README.md @@ -285,7 +285,7 @@ revoked-mint threshold). | 27 | `27-hostile-resume-cross-origin.json` | accepted 410 with a cross-origin `resume` → Terminal(`invalid_continuation`), zero foreign requests | | 28 | `28-checkpoint-load-failure.json` | store load Failed → Terminal(`checkpoint_load`) with ZERO wire attempts; distinct from Missing (which proceeds to a present entry) | | 29 | `29-checkpoint-save-failure-continues.json` | save Failed → feed continues and a SUBSEQUENT save is attempted (exact store-call script: no save circuit breaker) | -| 30 | `30-continuation-redirect-cross-origin.json` | validated same-origin `next` answering 302 + cross-origin Location → Terminal(`invalid_continuation`); zero foreign egress holds by construction of the seam here, and proving it against a real redirect is ASSIGNED to Layer 1, whose adapters are still pending (G1b) — see the row-15 note | +| 30 | `30-continuation-redirect-cross-origin.json` | validated same-origin `next` answering 302 + cross-origin Location → Terminal(`invalid_continuation`); zero foreign egress holds by construction of the seam here, and proving it against a real redirect is ASSIGNED to Layer 1, whose adapters are still pending, tracked in #819 — see the row-15 note | | 31 | `31-post-snapshot-straggler-below-served-id.json` | post-snapshot straggler with an id BELOW the entry page's served id delivered live; the re-push of that served id still suppressed | | 34 | `34-filter-changed-409-reenters-at-the-present.json` | 409 with both digests → `Observer.filterConflict` (digests pinned) before `Observer.positionRejected(filter_changed)`; the held position is discarded and the walk re-enters at `since=now` (present-class, no poll-served id) | @@ -399,7 +399,7 @@ exactly the class of claim this family exists to check. | 12 | `bypass-configured-handler` (handler registered but skipped; default-terminal applied) | 24, 25 (via `handlerInvocations` exact-set) | | 13 | `follow-cross-origin-continuation` (skips §8 validation, polls the hostile URL) | 26, 27 | | 14 | `collapse-load-error-to-missing` | 28 | -| 15 | `follow-cross-origin-redirect` (follows a 302 to a foreign Location) | **not killed at tier 2** — below the poll seam; assigned to Layer 1, whose adapters are still pending (G1b). Fixture 30 pins a different fault class above the seam. See the note under this table. | +| 15 | `follow-cross-origin-redirect` (follows a 302 to a foreign Location) | **not killed at tier 2** — below the poll seam; assigned to Layer 1, whose adapters are still pending, tracked in #819. Fixture 30 pins a different fault class above the seam. See the note under this table. | | 16 | `discard-live-id-at-or-below-served-id` (streaming lane orders live ids against the highest poll-served id) | 31 — and 31 alone: verified to pass all of 01–30, because every other straggler either arrives with nothing yet served (20) or is buffered pre-cut (01, 12, 19) | **Row 15 is not killed at tier 2, and the reason is structural.** In tier 2 the @@ -437,7 +437,7 @@ target is a Layer-1 property, and proving it is ASSIGNED to the Layer-1 seam adapter's own 302 test, where a real generated `PollEvents` call will meet a real redirect. Those adapters have not landed — `go/pkg/basecamp/eventfeed/doc.go` lists them among the pieces still to come — so this is a recorded obligation, -not a proof the repository contains today. Tracked for G1b. +not a proof the repository contains today. Tracked in #819. Auto-continue-past-unhandled-gap needs no separate mutation — fixture 23's exact-set `finally` is its direct test. Fixture 29's exact store-call script is the diff --git a/conformance/event-feed/fixtures/30-continuation-redirect-cross-origin.json b/conformance/event-feed/fixtures/30-continuation-redirect-cross-origin.json index aa7abaf3e..5d3b517f7 100644 --- a/conformance/event-feed/fixtures/30-continuation-redirect-cross-origin.json +++ b/conformance/event-feed/fixtures/30-continuation-redirect-cross-origin.json @@ -1,7 +1,6 @@ { "name": "30-continuation-redirect-cross-origin", - "description": "A VALIDATED same-origin `next` answers 302 with a cross-origin Location: the poll seam suppresses automatic redirect-following, and the foreign Location is Terminal(invalid_continuation). {{NEXT:1}} substitutes same-origin, so the pre-poll validation PASSES and the second poll seam call is made (contrast fixture 26, where no request reaches the URL at all) — the redirect answer is where the per-hop rule bites. What tier 2 verifies is the per-hop refusal DECISION (the driver classifies the scripted Location through the shipped predicate) and the loop's response: the continuation terminal, the socket explicitly closed, and no further seam calls. That the adapter's HTTP client makes zero requests to the refused URL is below the poll seam and owned by Layer-1 adapter conformance; the Location host is literal, never substituted, and never served. Kills follow-cross-origin-redirect.", - "config": { +"description": "A VALIDATED same-origin `next` answers 302 with a cross-origin Location: the poll seam suppresses automatic redirect-following, and the foreign Location is Terminal(invalid_continuation). {{NEXT:1}} substitutes same-origin, so the pre-poll validation PASSES and the second poll seam call is made (contrast fixture 26, where no request reaches the URL at all) \u2014 the redirect answer is where the per-hop rule bites. What tier 2 verifies is the per-hop refusal DECISION \u2014 the driver classifies the scripted Location through the SHIPPED predicate (checkContinuation, via a test export), never an ad-hoc re-implementation \u2014 and the loop's response: the continuation terminal, the socket explicitly closed, and no further seam calls. That the adapter's HTTP client makes zero requests to the refused URL is below the poll seam and owned by Layer-1 adapter conformance \u2014 a recorded obligation, not an existing proof: those adapters are still pending (tracked in #819), so follow-cross-origin-redirect itself remains NOT killed at tier 2 (row 15). The Location host is literal, never substituted, and never served. Redaction is deliberately not claimed here: the driver reduces the Location to its origin best-effort (an unreducible Location carries none) before the connector runs, so no path or query text exists for a connector to over-echo; TestRedirectRefusalExposesOnlyTheLocationOrigin owns that proof. See the row-15 note in README.md.", "config": { "position": "{{POS:0}}" }, "steps": [ diff --git a/conformance/event-feed/schema.json b/conformance/event-feed/schema.json index d8b93f298..b0e77eefb 100644 --- a/conformance/event-feed/schema.json +++ b/conformance/event-feed/schema.json @@ -989,7 +989,7 @@ "status", "headers" ], - "description": "Redirect on a continuation (fixture 30): the seam suppresses automatic following; a cross-origin/downgraded Location is Terminal(invalid_continuation). At tier 2 the poll lane is a SEAM, so the driver forms the redirect-refused verdict and the connector never sees a Location header: this pins the fault class above the seam (mishandling the verdict — retrying it or misclassifying it), not redirect-following itself, which lives below the seam, is unreachable at tier 2, and is ASSIGNED to the Layer-1 adapter's 302 test — a recorded obligation, not an existing proof: those adapters are still pending (tracked for G1b). Redaction is not pinned here: the driver reduces the Location to its origin before the connector runs, so no path or query text reaches it. No sentinel-listener obligation is imposed on the harness: the foreign origin is unreachable by construction of the harness, so a silent sentinel would be a statement about the driver rather than about the connector. See the row-15 note in README.md.", + "description": "Redirect on a continuation (fixture 30): the seam suppresses automatic following; a cross-origin/downgraded Location is Terminal(invalid_continuation). At tier 2 the poll lane is a SEAM, so the driver forms the redirect-refused verdict and the connector never sees a Location header: this pins the fault class above the seam (mishandling the verdict — retrying it or misclassifying it), not redirect-following itself, which lives below the seam, is unreachable at tier 2, and is ASSIGNED to the Layer-1 adapter's 302 test — a recorded obligation, not an existing proof: those adapters are still pending (tracked in #819). Redaction is not pinned here: the driver reduces the Location to its origin before the connector runs, so no path or query text reaches it. No sentinel-listener obligation is imposed on the harness: the foreign origin is unreachable by construction of the harness, so a silent sentinel would be a statement about the driver rather than about the connector. See the row-15 note in README.md.", "properties": { "status": { "const": 302 diff --git a/spec/tracking-issues.yml b/spec/tracking-issues.yml index 9abb6baf6..e20bc076c 100644 --- a/spec/tracking-issues.yml +++ b/spec/tracking-issues.yml @@ -94,3 +94,12 @@ prose_tracking_issues: - issue: 798 file: "MIGRATING.md" site: "MIGRATING.md — the generated copy of the parse is untouched pending #798" + - issue: 819 + file: "conformance/event-feed/README.md" + site: "conformance/event-feed/README.md — Layer-1 seam-adapter conformance (rows 15/30 and the row-15 note) is pending #819" + - issue: 819 + file: "conformance/event-feed/fixtures/30-continuation-redirect-cross-origin.json" + site: "fixture 30's description — the below-seam zero-request proof is pending #819's Layer-1 adapters" + - issue: 819 + file: "conformance/event-feed/schema.json" + site: "schema.json fixture-30 respond variant — the Layer-1 302 test is pending #819" From cff23ae81091b0f479c618c38a6e07ef0676e269 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Mon, 24 Aug 2026 09:53:57 -0700 Subject: [PATCH 26/55] Teach the sweep blockquotes, and give SPEC's Layer-1 assignment its citation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot's eighth round: two findings, both taken. * The sweep's line-leader class knew comments but not quotes: a canonical promise wrapped in a Markdown blockquote — "tracked\n> separately in #N" — read as absence, in the rendered prose where a promise reads most like one. The leader class gains `(?:>[ \t]*)+`, covering nested quotes. Red-proven: the blockquote corpus case was run against the previous detector first and failed with "expected FAILURE but checker passed"; all 30 cases pass after the widening. * SPEC §23's zero-egress paragraph was the one changed prose site that assigned the below-seam obligation to Layer-1 without citing #819 — so after #819 closed, the other five sites' registry rows could be cleaned up while this paragraph stayed stale. It now ends with the canonical citation, and SPEC.md carries the registry row, so the sweep enforces it with the rest. Gates: tracking-issues self-test 30/30; the live checker verifies all 26 entries and reports all 6 referenced tracking issues open; doc-constants-check green under LC_ALL=C. --- SPEC.md | 2 +- scripts/check-known-defect-issues-open | 12 ++++++------ scripts/test-check-known-defect-issues-open | 13 +++++++++++++ spec/tracking-issues.yml | 3 +++ 4 files changed, 23 insertions(+), 7 deletions(-) diff --git a/SPEC.md b/SPEC.md index 2a1dc40d1..bb1decc0a 100644 --- a/SPEC.md +++ b/SPEC.md @@ -4063,7 +4063,7 @@ asserting something about itself. That obligation belongs to the Layer-1 seam adapter's own 302 test, where a real generated `PollEvents` call meets a real redirect against an adapter with automatic redirect-following disabled. `conformance/event-feed/README.md`'s row-15 note records it as a pending obligation -rather than a proof the repository contains. +rather than a proof the repository contains; the adapters are tracked in #819. ### Clock, Timers, and Virtual Time `[conformance]` diff --git a/scripts/check-known-defect-issues-open b/scripts/check-known-defect-issues-open index 18e3f7a7d..9f5a9bcd6 100755 --- a/scripts/check-known-defect-issues-open +++ b/scripts/check-known-defect-issues-open @@ -185,12 +185,12 @@ end # with a plain directory walk as the fallback for a sweep root that is not a # repository (the self-test corpus). Binary files are skipped by NUL sniff. -# The comment leader is matched by CLASS, not by count: Swift/Kotlin doc -# comments are `///`, box comments stack `*`, and a leader that consumed an -# exact number of marks would leave the surplus mark in front of the next word -# and read a wrapped promise as absence. `#+` refuses a following digit so an -# issue reference is never eaten as a leader. -SWEEP_GAP = /(?:[ \t]|\r?\n[ \t]*(?:#+(?!\d)|\/{2,}|\*+|-{2,})?[ \t]*)+/ +# The line leader is matched by CLASS, not by count: Swift/Kotlin doc +# comments are `///`, box comments stack `*`, Markdown blockquotes nest `>`, +# and a leader that consumed an exact number of marks would leave the surplus +# mark in front of the next word and read a wrapped promise as absence. `#+` +# refuses a following digit so an issue reference is never eaten as a leader. +SWEEP_GAP = /(?:[ \t]|\r?\n[ \t]*(?:#+(?!\d)|\/{2,}|\*+|-{2,}|(?:>[ \t]*)+)?[ \t]*)+/ SWEEP_MENTION = /\btracked(?:#{SWEEP_GAP}separately)?#{SWEEP_GAP}in#{SWEEP_GAP}#(\d+)/i def sweep_files(root) diff --git a/scripts/test-check-known-defect-issues-open b/scripts/test-check-known-defect-issues-open index 56651ed67..40d0bc371 100755 --- a/scripts/test-check-known-defect-issues-open +++ b/scripts/test-check-known-defect-issues-open @@ -551,6 +551,19 @@ out, status, _ = run_checker( expect_fail(failures, "N+11a. a Swift doc-comment (///) wrapped mention is still caught", out, status, "not registered") +# --- N+11b. The wrap is caught behind a Markdown blockquote too ------------------ +# +# `> ` opens a blockquote continuation line, and rendered prose is exactly +# where a tracking promise reads most like a promise. A leader class that +# knows comments but not quotes reads the wrapped claim as absence. + +out, status, _ = run_checker( + allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, + sweep: { "docs/note.md" => "> the six SDKs disagree and this is tracked\n> separately in #909.\n" } +) +expect_fail(failures, "N+11b. a blockquote-wrapped mention is still caught", out, status, + "not registered") + # --- N+12. A row with no file fails ---------------------------------------------- # # The sweep keys coverage on [file, issue], so an entry without a file could diff --git a/spec/tracking-issues.yml b/spec/tracking-issues.yml index e20bc076c..22fd3b8b5 100644 --- a/spec/tracking-issues.yml +++ b/spec/tracking-issues.yml @@ -94,6 +94,9 @@ prose_tracking_issues: - issue: 798 file: "MIGRATING.md" site: "MIGRATING.md — the generated copy of the parse is untouched pending #798" + - issue: 819 + file: "SPEC.md" + site: "SPEC.md §23 zero-egress paragraph — the Layer-1 302 test's adapters are pending #819" - issue: 819 file: "conformance/event-feed/README.md" site: "conformance/event-feed/README.md — Layer-1 seam-adapter conformance (rows 15/30 and the row-15 note) is pending #819" From a4a66a5ccae693c67b95ae2f17b0db16bd2d761b Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Mon, 24 Aug 2026 19:21:23 -0700 Subject: [PATCH 27/55] Bound every ms field at 10 virtual years, in the schema and at load MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot's ninth round, finding 1: the schema's ms fields carried no maximum, so a schema-valid advance.ms of 9223372036855 — one past the largest ms count whose int64-nanosecond product fits — multiplied NEGATIVE in Go's millis(), and AdvanceIfQuiet, asked to advance by a negative window, silently REWOUND virtual time. Drivers diverge on an input the schema blesses. Both sides now bound it. Every ms-carrying field (the five config durations, advance.ms, both assertDelayMs bounds — the whole duration-carrying class; liveBufferCapacity and dedupeCapacity are counts, not durations) carries maximum: 315576000000. The bound is 10 virtual years, and it is a DOMAIN bound, stated in the schema and the driver const: scripts age tickets by minutes to days (largest today ~11 days, 300× headroom), and the schema should say what a script can MEAN rather than restate one language's integer layout — while sitting ~29× under the int64-nanosecond line so no conforming representation can overflow. The Go driver enforces the range at load through one helper (checkScenarioMs), so an out-of-range value is a fixture error in every driver, not a representation accident in one. Red-proven: three new load-rejection selftests (advance.ms, the assertDelayMs envelope, config stalenessMs) were run against the previous driver first and all failed — the overflow advance case consumed its 5s watchdog, the accepted-rewind in action. Green after, with event-feed-fixtures-check green (schema changed; all 31 fixtures and both pin probes verify). --- conformance/event-feed/schema.json | 15 ++++- .../eventfeed/scenario_fixture_test.go | 55 ++++++++++++++++++- .../eventfeed/scenario_selftest_test.go | 20 +++++++ 3 files changed, 84 insertions(+), 6 deletions(-) diff --git a/conformance/event-feed/schema.json b/conformance/event-feed/schema.json index b0e77eefb..e4fe02303 100644 --- a/conformance/event-feed/schema.json +++ b/conformance/event-feed/schema.json @@ -314,26 +314,31 @@ "confirmationDeadlineMs": { "type": "integer", "minimum": 1, + "maximum": 315576000000, "description": "Default 10000." }, "repairPollBaseMs": { "type": "integer", "minimum": 1, + "maximum": 315576000000, "description": "Repair interval base. Default 60000, ±20% jitter per cycle. Override large to keep repair-poll out of an advance window." }, "backoffBaseMs": { "type": "integer", "minimum": 1, + "maximum": 315576000000, "description": "Reconnect/poll-retry full-jitter base. Default 1000." }, "backoffCapMs": { "type": "integer", "minimum": 1, + "maximum": 315576000000, "description": "Local jitter-draw cap. Default 60000. Server-directed Retry-After is exempt per §7." }, "stalenessMs": { "type": "integer", "minimum": 1, + "maximum": 315576000000, "description": "Default 7500. Override large when a scenario advances virtual time without scripting frames." }, "liveBufferCapacity": { @@ -1326,7 +1331,9 @@ "properties": { "ms": { "type": "integer", - "minimum": 1 + "minimum": 1, + "maximum": 315576000000, + "description": "Every ms field in this schema shares this maximum: 10 virtual years, a DOMAIN bound (scripts age tickets by minutes to days) chosen over the representation-derived 9223372036854 because the schema states what a script can mean, not one language's integer layout. It sits ~29× under the int64-nanosecond overflow line, so no conforming driver's duration representation can overflow — one past that line, a naive ms-to-duration multiply goes negative and an accepted advance would silently REWIND virtual time. Drivers enforce the range at load." } } }, @@ -1352,11 +1359,13 @@ "properties": { "min": { "type": "integer", - "minimum": 0 + "minimum": 0, + "maximum": 315576000000 }, "max": { "type": "integer", - "minimum": 0 + "minimum": 0, + "maximum": 315576000000 } } } diff --git a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go index 539cf50d8..0e980f24a 100644 --- a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go @@ -253,6 +253,29 @@ type expectFilterConflictStep struct { // --- loading ------------------------------------------------------------- +// maxScenarioMs is the schema's shared `maximum` for every ms field: 10 +// virtual years. It is a DOMAIN bound — scripts age tickets by minutes to +// days, so the largest value today (~11 days) has 300× headroom — chosen over +// the representation-derived 9,223,372,036,854 (the largest ms count whose +// int64-nanosecond product does not overflow) because the schema should say +// what a script can MEAN, not restate one language's integer layout. It sits +// ~29× under that overflow line, so no conforming driver's duration +// representation can overflow — the failure this bound exists to make a +// fixture error rather than a representation accident: one past the int64 +// line, time.Duration(ms)*time.Millisecond goes negative and an accepted +// advance would silently REWIND virtual time. +const maxScenarioMs = 315_576_000_000 + +// checkScenarioMs enforces the schema's [floor, maxScenarioMs] range on one +// ms field at load, so every driver rejects the same values for the same +// stated reason. +func checkScenarioMs(what string, v, floor int) error { + if v < floor || v > maxScenarioMs { + return fmt.Errorf("%s must be in [%d, %d] (10 virtual years): got %d", what, floor, maxScenarioMs, v) + } + return nil +} + // parseScenario decodes one substituted fixture, failing on anything the // driver does not model. func parseScenario(raw []byte, file string) (*scenario, error) { @@ -387,14 +410,25 @@ func decodeDirective(kind string, body json.RawMessage) (any, error) { return &expectClientCloseStep{}, decodeStrict(body, &empty) case "advance": step := &advanceStep{} - return step, decodeStrict(body, step) + if err := decodeStrict(body, step); err != nil { + return nil, err + } + return step, checkScenarioMs("advance ms", step.Ms, 1) case "fireTimer": step := &fireTimerStep{} if err := decodeStrict(body, step); err != nil { return nil, err } - if step.AssertDelayMs != nil && step.AssertDelayMs.Min > step.AssertDelayMs.Max { - return nil, fmt.Errorf("assertDelayMs min %d exceeds max %d", step.AssertDelayMs.Min, step.AssertDelayMs.Max) + if step.AssertDelayMs != nil { + if step.AssertDelayMs.Min > step.AssertDelayMs.Max { + return nil, fmt.Errorf("assertDelayMs min %d exceeds max %d", step.AssertDelayMs.Min, step.AssertDelayMs.Max) + } + if err := checkScenarioMs("assertDelayMs min", step.AssertDelayMs.Min, 0); err != nil { + return nil, err + } + if err := checkScenarioMs("assertDelayMs max", step.AssertDelayMs.Max, 0); err != nil { + return nil, err + } } return step, validateTimerKind(step.Kind) case "expectDelivered": @@ -489,6 +523,21 @@ type ( // --- validation ---------------------------------------------------------- func validateConfig(cfg scenarioConfig) error { + // Absent decodes as 0 and means "default", so only set values are ranged. + for _, f := range []struct { + name string + v int + }{ + {"confirmationDeadlineMs", cfg.ConfirmationDeadlineMs}, + {"repairPollBaseMs", cfg.RepairPollBaseMs}, + {"stalenessMs", cfg.StalenessMs}, + } { + if f.v != 0 { + if err := checkScenarioMs(f.name, f.v, 1); err != nil { + return err + } + } + } if cfg.BackoffBaseMs != 0 || cfg.BackoffCapMs != 0 { return fmt.Errorf("backoffBaseMs/backoffCapMs are not modeled: SPEC §23 pins the Go connector's full-jitter base and cap as constants, with no construction option to override") } diff --git a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go index 00053a20d..e36c5158c 100644 --- a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go @@ -442,6 +442,26 @@ func TestScenarioDriverRejectsUnmodelledScripts(t *testing.T) { script: `{"name":"x","description":"d","steps":[{"fireTimer":{"kind":"backoff","assertDelayMs":{"min":1000,"max":10}}}],"finally":{"state":"closed"}}`, wants: "exceeds max", }, + { + // The schema's shared ms maximum, driver-enforced: one past Go's + // int64-nanosecond line, time.Duration(ms)*time.Millisecond goes + // NEGATIVE and an accepted advance would REWIND virtual time. The + // bound is checked at load so the overflow is a fixture error in + // every driver, not a representation accident in one. + name: "advance ms beyond the 10-virtual-year maximum", + script: `{"name":"x","description":"d","steps":[{"advance":{"ms":9223372036855}}],"finally":{"state":"closed"}}`, + wants: "10 virtual years", + }, + { + name: "fireTimer envelope beyond the 10-virtual-year maximum", + script: `{"name":"x","description":"d","steps":[{"fireTimer":{"kind":"backoff","assertDelayMs":{"min":0,"max":9223372036855}}}],"finally":{"state":"closed"}}`, + wants: "10 virtual years", + }, + { + name: "config stalenessMs beyond the 10-virtual-year maximum", + script: `{"name":"x","description":"d","config":{"stalenessMs":9223372036855},"steps":[{"advance":{"ms":1}}],"finally":{"state":"closed"}}`, + wants: "10 virtual years", + }, { name: "droppedCount disagreeing with droppedIds", script: `{"name":"x","description":"d","steps":[{"expectSignal":{"kind":"bufferOverflow","droppedIds":[1],"droppedCount":2}}],"finally":{"state":"closed"}}`, From d5e22d19a5d5df1441885c0fdc6124df1b20b2c5 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Mon, 24 Aug 2026 19:23:54 -0700 Subject: [PATCH 28/55] Commit the mention counts, so a covered file cannot absorb new promises MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot's ninth round, finding 2: the sweep's [file, issue] coverage was a grant without a number — once a pair was registered, every FUTURE canonical mention of that issue in that file was silently covered, violating the registry's one-entry-per-claim contract (README carries three #819 mentions on one row, python's todolists.py two #578). The repository's instrument for exactly this is doc-constants' committed marker counts, and the registry now uses it. Every row commits `mentions:` — the number of canonical "tracked in #N" mentions it stands for, 0 for a promise phrased outside the canonical grammar — validated like `issue`/`site`/`file`, with duplicate [file, issue] rows rejected (they would make the count ambiguous). The sweep verifies the number in both directions: a second mention fails until the count moves, and a deleted sentence fails until it moves back. All 26 rows carry their true counts (13 canonical mentions across the tree), computed with the sweep's own grammar. Red-proven: four new self-test cases were run against the previous checker first and all failed with "expected FAILURE but checker passed" — the drifted count (2 swept, 1 registered), the deleted mention (0 swept, 1 registered), the missing count, and the duplicate pair — while the matching-count control passed both sides. All 35 cases pass after the fix. Gates: tracking-issues self-test 35/35; the live checker sweeps clean with counts verified and reports all 6 referenced tracking issues open; doc-constants-check green under LC_ALL=C. --- scripts/check-known-defect-issues-open | 41 ++++++++++-- scripts/test-check-known-defect-issues-open | 71 +++++++++++++++++++-- spec/tracking-issues.yml | 39 +++++++++-- 3 files changed, 137 insertions(+), 14 deletions(-) diff --git a/scripts/check-known-defect-issues-open b/scripts/check-known-defect-issues-open index 9f5a9bcd6..8653b157c 100755 --- a/scripts/check-known-defect-issues-open +++ b/scripts/check-known-defect-issues-open @@ -167,9 +167,27 @@ rows.each_with_index do |e, i| 'string naming the path the sentence lives in (the sweep keys coverage on [file, issue]) — got ' + e['file'].inspect exit 1 end + # The count is committed, exactly as doc-constants commits its marker + # counts: a bare [file, issue] pair would silently cover every FUTURE + # canonical mention in the file, so each row records how many it stands for + # (0 for a promise phrased outside the canonical grammar) and the sweep + # verifies the number in both directions. + unless e['mentions'].is_a?(Integer) && e['mentions'] >= 0 + warn "ERROR: prose_tracking_issues[#{i}] (##{e['issue']}) in #{tracking_path} needs a `mentions` count — " \ + 'the number of canonical "tracked in #N" mentions of this issue in `file` — got ' + e['mentions'].inspect + exit 1 + end refs << [e['issue'], e['site']] end +# One row per [file, issue]: a duplicate would make the committed count +# ambiguous (which row does the sweep charge a mention to?). +dup = rows.map { |e| [ e['file'], e['issue'] ] }.tally.select { |_, n| n > 1 }.keys +unless dup.empty? + dup.each { |f, n| warn "ERROR: duplicate prose_tracking_issues rows for [#{f}, ##{n}] in #{tracking_path} — merge them and sum `mentions`" } + exit 1 +end + # --- Discovery sweep -------------------------------------------------------------- # # Hand-enumeration cannot prove the registry covers the tree, so the one @@ -215,8 +233,8 @@ def sweep_files(root) end sweep_root = ENV['PROSE_SWEEP_ROOT'] || ROOT -registered_pairs = rows.map { |e| [ e['file'], e['issue'] ] } -violations = [] +registered_mentions = rows.to_h { |e| [ [ e['file'], e['issue'] ], e['mentions'] ] } +swept = Hash.new { |h, k| h[k] = [] } sweep_files(sweep_root).each do |rel| next if rel == 'spec/tracking-issues.yml' || rel == 'scripts/test-check-known-defect-issues-open' || @@ -231,12 +249,25 @@ sweep_files(sweep_root).each do |rel| content = content.force_encoding(Encoding::UTF_8).scrub content.to_enum(:scan, SWEEP_MENTION).each do m = Regexp.last_match - next if registered_pairs.include?([ rel, m[1].to_i ]) + swept[[ rel, m[1].to_i ]] << content[0...m.begin(0)].count("\n") + 1 + end +end - line = content[0...m.begin(0)].count("\n") + 1 - violations << "#{rel}:#{line} — a canonical \"tracked in ##{m[1]}\" claim, and [#{rel}, ##{m[1]}] is not registered" +violations = [] +swept.each do |(rel, issue), lines| + unless registered_mentions.key?([ rel, issue ]) + lines.each { |line| violations << "#{rel}:#{line} — a canonical \"tracked in ##{issue}\" claim, and [#{rel}, ##{issue}] is not registered" } end end +# Both directions, so a new mention cannot ride an existing row and a deleted +# sentence cannot leave its count claiming more than the file holds. +registered_mentions.each do |(rel, issue), expected| + found = swept.fetch([ rel, issue ], []) + next if found.length == expected + + violations << "#{rel} ##{issue}: the registry records #{expected} canonical mention(s), the sweep found " \ + "#{found.length}#{found.empty? ? '' : " (line #{found.join(', ')})"} — update `mentions:` alongside the prose" +end unless violations.empty? warn 'ERROR: canonical tracking claims exist outside the registry:' violations.each { |v| warn " - #{v}" } diff --git a/scripts/test-check-known-defect-issues-open b/scripts/test-check-known-defect-issues-open index 40d0bc371..9047d28a4 100755 --- a/scripts/test-check-known-defect-issues-open +++ b/scripts/test-check-known-defect-issues-open @@ -335,7 +335,7 @@ expect_fail(failures, "9. missing gh fails closed", out, status, out, status, calls = run_checker( allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, - tracking: { "prose_tracking_issues" => [ { "issue" => 303, "file" => "README.md", "site" => "README.md — a gap" } ] }, + tracking: { "prose_tracking_issues" => [ { "issue" => 303, "file" => "README.md", "site" => "README.md — a gap", "mentions" => 0 } ] }, states: "303=open" ) expect_pass(failures, "N. prose registry entries are collected", out, status, @@ -352,7 +352,7 @@ end out, status, _ = run_checker( allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, - tracking: { "prose_tracking_issues" => [ { "issue" => 404, "file" => "SPEC.md", "site" => "SPEC.md — a bound" } ] }, + tracking: { "prose_tracking_issues" => [ { "issue" => 404, "file" => "SPEC.md", "site" => "SPEC.md — a bound", "mentions" => 0 } ] }, states: "404=closed" ) expect_fail(failures, "N+1. a CLOSED prose tracking issue fails", out, status, @@ -365,7 +365,7 @@ expect_fail(failures, "N+1. a CLOSED prose tracking issue fails", out, status, out, status, _ = run_checker( allowlist: { "sdk_routes_known_defective" => [ defective(101) ], "bc3_routes_not_modeled" => [] }, - tracking: { "prose_tracking_issues" => [ { "issue" => 303, "file" => "README.md", "site" => "README.md — a gap" } ] }, + tracking: { "prose_tracking_issues" => [ { "issue" => 303, "file" => "README.md", "site" => "README.md — a gap", "mentions" => 0 } ] }, states: "101=open,303=closed" ) expect_fail(failures, "N+2. a closed registry entry fails even with a green allowlist", out, status, @@ -500,7 +500,7 @@ expect_fail(failures, "N+8. an unregistered canonical mention fails the sweep", out, status, calls = run_checker( allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, tracking: { "prose_tracking_issues" => [ - { "issue" => 303, "file" => "docs/note.md", "site" => "docs/note.md — a gap" } + { "issue" => 303, "file" => "docs/note.md", "site" => "docs/note.md — a gap", "mentions" => 1 } ] }, sweep: { "docs/note.md" => "The gap is tracked in #303.\n" }, states: "303=open" @@ -564,6 +564,69 @@ out, status, _ = run_checker( expect_fail(failures, "N+11b. a blockquote-wrapped mention is still caught", out, status, "not registered") +# --- N+13..16. Counts are committed, like doc-constants' marker counts ----------- +# +# [file, issue] pair coverage silently absorbed every FUTURE matching promise +# in a covered file: a second "tracked in #N" could appear and ride the +# existing row unregistered. The repo's instrument for exactly this is the +# committed COUNT — a row records how many canonical mentions it stands for, +# the sweep verifies the number in both directions, and a new mention (or a +# deleted one) fails until someone updates the row. + +two = "one gap is tracked in #303 here.\nThe other is also tracked in #303.\n" + +out, status, _ = run_checker( + allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, + tracking: { "prose_tracking_issues" => [ + { "issue" => 303, "file" => "docs/note.md", "site" => "docs/note.md — a gap", "mentions" => 1 } + ] }, + sweep: { "docs/note.md" => two }, + states: "303=open" +) +expect_fail(failures, "N+13. a second canonical mention fails until the count moves", out, status, + "records 1 canonical mention(s), the sweep found 2") + +out, status, _ = run_checker( + allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, + tracking: { "prose_tracking_issues" => [ + { "issue" => 303, "file" => "docs/note.md", "site" => "docs/note.md — two gaps", "mentions" => 2 } + ] }, + sweep: { "docs/note.md" => two }, + states: "303=open" +) +expect_pass(failures, "N+14. a matching count passes", out, status) + +out, status, _ = run_checker( + allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, + tracking: { "prose_tracking_issues" => [ + { "issue" => 303, "file" => "docs/note.md", "site" => "docs/note.md — a gap", "mentions" => 1 } + ] }, + states: "303=open" +) +expect_fail(failures, "N+15. a deleted mention fails until the count moves", out, status, + "records 1 canonical mention(s), the sweep found 0") + +out, status, _ = run_checker( + allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, + tracking: { "prose_tracking_issues" => [ + { "issue" => 303, "file" => "README.md", "site" => "README.md — a gap" } + ] }, + states: "303=open" +) +expect_fail(failures, "N+16. a row without a mentions count fails", out, status, + "needs a `mentions`") + +out, status, _ = run_checker( + allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, + tracking: { "prose_tracking_issues" => [ + { "issue" => 303, "file" => "README.md", "site" => "README.md — a gap", "mentions" => 0 }, + { "issue" => 303, "file" => "README.md", "site" => "README.md — the same gap again", "mentions" => 0 } + ] }, + states: "303=open" +) +expect_fail(failures, "N+17. duplicate [file, issue] rows fail", out, status, + "duplicate") + # --- N+12. A row with no file fails ---------------------------------------------- # # The sweep keys coverage on [file, issue], so an entry without a file could diff --git a/spec/tracking-issues.yml b/spec/tracking-issues.yml index 22fd3b8b5..d300723da 100644 --- a/spec/tracking-issues.yml +++ b/spec/tracking-issues.yml @@ -17,11 +17,14 @@ # need an ever-growing exclusion list to tell the two apart. The registry makes # the author say which kind they meant, once, where the reason is visible. # -# Every entry names the `file` its sentence lives in, because one detector -# backs this registry: the checker sweeps tracked text for the canonical -# grammar "tracked [separately] in #N" and fails on any mention whose -# [file, issue] pair is not registered here — the canonical form cannot be -# written without being registered. Non-canonical phrasings are out of the +# Every entry names the `file` its sentence lives in and commits a `mentions` +# count, because one detector backs this registry: the checker sweeps tracked +# text for the canonical grammar "tracked [separately] in #N" and fails on any +# mention whose [file, issue] pair is not registered here — and on any pair +# whose swept count differs from the committed one, in either direction, so a +# NEW mention cannot ride an existing row and a deleted sentence cannot leave +# its row claiming more than the file holds (`mentions: 0` marks a promise +# phrased outside the canonical grammar, registered by hand). Non-canonical phrasings are out of the # detector's reach by design (an open vocabulary cannot be enumerated); they # still go through this registry by hand, on the honor the header describes. # @@ -31,78 +34,104 @@ prose_tracking_issues: - issue: 589 file: "Makefile" site: "Makefile — the vendored bc3-route table's freshness gate needs BC3_REPO_PATH, so it is not in CI" + mentions: 1 - issue: 775 file: "SPEC.md" site: "SPEC.md §6 Retry-After — which statuses honour the header is divergent across the six SDKs" + mentions: 0 - issue: 775 file: "typescript/tests/retry-after.test.ts" site: "typescript/tests/retry-after.test.ts — pins TypeScript's side of the Retry-After status divergence" + mentions: 1 - issue: 775 file: "MIGRATING.md" site: "MIGRATING.md — the Retry-After migration note repeats the status-divergence promise" + mentions: 1 - issue: 578 file: "ruby/lib/basecamp/services/todolists_extensions.rb" site: "ruby/lib/basecamp/services/todolists_extensions.rb — the generated validating layer that would retire writable_string" + mentions: 1 - issue: 578 file: "python/src/basecamp/services/todolists.py" site: "python/src/basecamp/services/todolists.py — making the flat todolist read structurally safe" + mentions: 2 - issue: 578 file: "typescript/src/services/todolists-extensions.ts" site: "typescript/src/services/todolists-extensions.ts — structural safety for this SDK" + mentions: 0 - issue: 578 file: "python/tests/services/test_todolist_groups_service.py" site: "python/tests/services/test_todolist_groups_service.py — giving Python a structural decoder" + mentions: 0 - issue: 578 file: "typescript/tests/services/todolists.test.ts" site: "typescript/tests/services/todolists.test.ts — structural safety for this SDK" + mentions: 0 - issue: 578 file: "ruby/lib/basecamp/services/merge_safe.rb" site: "ruby/lib/basecamp/services/merge_safe.rb — the generated validating layer as the guards' intended end state" + mentions: 0 - issue: 578 file: "python/src/basecamp/services/_merge_safe.py" site: "python/src/basecamp/services/_merge_safe.py — the generated validating layer as the guards' intended end state" + mentions: 0 - issue: 578 file: "typescript/src/services/merge-safe.ts" site: "typescript/src/services/merge-safe.ts — the generated validating layer as the guards' intended end state" + mentions: 0 - issue: 775 file: "kotlin/sdk/src/commonMain/kotlin/com/basecamp/sdk/Pagination.kt" site: "kotlin/sdk/src/commonMain/kotlin/com/basecamp/sdk/Pagination.kt — #775 carries the six-SDK Retry-After table" + mentions: 0 - issue: 775 file: "kotlin/sdk/src/commonTest/kotlin/com/basecamp/sdk/PaginationTest.kt" site: "kotlin/sdk/src/commonTest/kotlin/com/basecamp/sdk/PaginationTest.kt — #775 carries the six-SDK Retry-After table" + mentions: 0 - issue: 775 file: "typescript/src/errors.ts" site: "typescript/src/errors.ts — the strictness gate is recorded against the #775 divergence" + mentions: 0 - issue: 799 file: "SPEC.md" site: "SPEC.md §6 — Python and Ruby still truncate the delay; #799 tracks the rounding convergence" + mentions: 1 - issue: 799 file: "MIGRATING.md" site: "MIGRATING.md — #799 tracks the cross-SDK convergence on Retry-After rounding" + mentions: 0 - issue: 775 file: "go/pkg/basecamp/client.go" site: "go/pkg/basecamp/client.go — widening the set of statuses that carry a parsed Retry-After is #775's" + mentions: 0 - issue: 799 file: "go/pkg/basecamp/client.go" site: "go/pkg/basecamp/client.go — the rounding and over-range halves of the divergence are #799's" + mentions: 0 - issue: 799 file: "go/pkg/basecamp/client_retry_after_test.go" site: "go/pkg/basecamp/client_retry_after_test.go — pins Go's side pending #799's convergence" + mentions: 0 - issue: 798 file: "SPEC.md" site: "SPEC.md §6 — the generated Go parser stays Atoi until #798 lands the template change" + mentions: 0 - issue: 798 file: "MIGRATING.md" site: "MIGRATING.md — the generated copy of the parse is untouched pending #798" + mentions: 0 - issue: 819 file: "SPEC.md" site: "SPEC.md §23 zero-egress paragraph — the Layer-1 302 test's adapters are pending #819" + mentions: 1 - issue: 819 file: "conformance/event-feed/README.md" site: "conformance/event-feed/README.md — Layer-1 seam-adapter conformance (rows 15/30 and the row-15 note) is pending #819" + mentions: 3 - issue: 819 file: "conformance/event-feed/fixtures/30-continuation-redirect-cross-origin.json" site: "fixture 30's description — the below-seam zero-request proof is pending #819's Layer-1 adapters" + mentions: 1 - issue: 819 file: "conformance/event-feed/schema.json" site: "schema.json fixture-30 respond variant — the Layer-1 302 test is pending #819" + mentions: 1 From b9e61bf728e4f1c3f5138af8cfbdf4c9aad1a37b Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Mon, 24 Aug 2026 19:38:24 -0700 Subject: [PATCH 29/55] Model scenario ms values as int64, end to end MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot's tenth-round finding: maxScenarioMs (315,576,000,000) exceeds MaxInt32 while the fixture structs and validator used platform-width int, so GOARCH=386 fails to COMPILE at the comparison — and typing only the constant would trade the compile error for a decode error, since schema-valid values above MaxInt32 could not land in the int structs. The ms values are now int64 end to end: the five config durations, advance.Ms, the assertDelayMs envelope, checkScenarioMs, and millis() itself, with the width rationale on the validator. Red-proven with the 32-bit compile check: GOOS=linux GOARCH=386 go vet against the previous code fails with "maxScenarioMs (untyped int constant 315576000000) overflows int" at the cited line (darwin/386 is not a valid pair, hence the linux GOOS); after the conversion — and after typing the constant itself int64, which the first pass missed because the untyped constant still defaulted to int inside fmt.Errorf's variadic — the same vet exits 0. Gates: GOOS=linux GOARCH=386 go vet clean; native suite green for both eventfeed packages, plain and -race; gofmt clean. --- .../eventfeed/scenario_fixture_test.go | 27 ++++++++++--------- .../eventfeed/scenario_harness_test.go | 2 +- 2 files changed, 16 insertions(+), 13 deletions(-) diff --git a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go index 0e980f24a..27fb5b666 100644 --- a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go @@ -46,11 +46,11 @@ type scenarioConfig struct { ExcludePerformers []int64 `json:"exclude_performers"` ActorTypes []string `json:"actorTypes"` Position string `json:"position"` - ConfirmationDeadlineMs int `json:"confirmationDeadlineMs"` - RepairPollBaseMs int `json:"repairPollBaseMs"` - BackoffBaseMs int `json:"backoffBaseMs"` - BackoffCapMs int `json:"backoffCapMs"` - StalenessMs int `json:"stalenessMs"` + ConfirmationDeadlineMs int64 `json:"confirmationDeadlineMs"` + RepairPollBaseMs int64 `json:"repairPollBaseMs"` + BackoffBaseMs int64 `json:"backoffBaseMs"` + BackoffCapMs int64 `json:"backoffCapMs"` + StalenessMs int64 `json:"stalenessMs"` LiveBufferCapacity int `json:"liveBufferCapacity"` DedupeCapacity int `json:"dedupeCapacity"` SignalDisposition map[string]string `json:"signalDisposition"` @@ -195,7 +195,7 @@ type goneBody struct { } type advanceStep struct { - Ms int `json:"ms"` + Ms int64 `json:"ms"` } type fireTimerStep struct { @@ -204,8 +204,8 @@ type fireTimerStep struct { } type delayEnvelope struct { - Min int `json:"min"` - Max int `json:"max"` + Min int64 `json:"min"` + Max int64 `json:"max"` } type expectCheckpointStep struct { @@ -264,12 +264,15 @@ type expectFilterConflictStep struct { // fixture error rather than a representation accident: one past the int64 // line, time.Duration(ms)*time.Millisecond goes negative and an accepted // advance would silently REWIND virtual time. -const maxScenarioMs = 315_576_000_000 +const maxScenarioMs int64 = 315_576_000_000 // checkScenarioMs enforces the schema's [floor, maxScenarioMs] range on one // ms field at load, so every driver rejects the same values for the same -// stated reason. -func checkScenarioMs(what string, v, floor int) error { +// stated reason. ms values are int64 END TO END (fixture structs, this check, +// millis): the maximum exceeds MaxInt32, so a platform-width int fails to +// compile on 32-bit (an int64-typed constant alone would instead make +// schema-valid values above MaxInt32 fail decode into int structs). +func checkScenarioMs(what string, v, floor int64) error { if v < floor || v > maxScenarioMs { return fmt.Errorf("%s must be in [%d, %d] (10 virtual years): got %d", what, floor, maxScenarioMs, v) } @@ -526,7 +529,7 @@ func validateConfig(cfg scenarioConfig) error { // Absent decodes as 0 and means "default", so only set values are ranged. for _, f := range []struct { name string - v int + v int64 }{ {"confirmationDeadlineMs", cfg.ConfirmationDeadlineMs}, {"repairPollBaseMs", cfg.RepairPollBaseMs}, diff --git a/go/pkg/basecamp/eventfeed/scenario_harness_test.go b/go/pkg/basecamp/eventfeed/scenario_harness_test.go index c153700e8..d556561e6 100644 --- a/go/pkg/basecamp/eventfeed/scenario_harness_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_harness_test.go @@ -778,4 +778,4 @@ func (h *scenarioHarness) signalHandler(dispositions map[string]string) eventfee } } -func millis(ms int) time.Duration { return time.Duration(ms) * time.Millisecond } +func millis(ms int64) time.Duration { return time.Duration(ms) * time.Millisecond } From 53271c7f355b742d5bcdc9edb6d3da85e026f786 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Mon, 24 Aug 2026 19:58:36 -0700 Subject: [PATCH 30/55] Scope the ceiling to Go, describe the registry's real unit, refuse dup keys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot's eleventh round: three findings taken, one already fixed. * The cable-lane retention ceiling was presented inside the shared six-SDK contract while its proof is Go-shaped — goroutines, json.RawMessage, a copying decoder — and Appendix F already records a lane where the per-item premise fails: TypeScript's default global-WebSocket transport cannot pre-bound a read, so an oversized message is allocated whole at receipt, before the max-frame check drops it. The formula is now scoped explicitly to the GO REFERENCE IMPLEMENTATION and transports with bounded reads: every SDK inherits the shape (bounded queue, blocking hand-off, single deferral slot, buffer as the only drop point) but re-derives its own weights, the TS divergence is cited where the ceiling is stated, and no universal cross-SDK byte ceiling is published. * The registry header still said "one entry per sentence" while the validator rejects duplicate [file, issue] rows and commits one mentions count — instructions that would lead a maintainer to write rows the gate refuses. The header now describes the actual unit: one [file, issue] entry, sentences in the same file sharing the row and summing in its count, with bump/decrement as the maintenance verbs. * YAML keeps only the LAST of duplicate mapping keys, so a second sdk_routes_known_defective: key REPLACED the first list — every issue reference in it gone, with nothing left malformed for shape validation to see. Both loads now walk Psych's parse tree (which still holds both keys) before safe_load flattens them, failing with the gate's diagnostic on any duplicated key at any depth. Red-proven: the two new self-test cases — a duplicated allowlist key hiding a defective route, a duplicated registry key hiding a promise — were run against the previous checker first and both passed it ("expected FAILURE but checker passed"); all 37 cases pass after the fix. Already fixed, no change: the 32-bit int overflow finding was reviewed against fed7b2049, one commit before e42277e33 modeled ms values as int64 end to end; GOOS=linux GOARCH=386 go vet is clean at this head. Gates: tracking-issues self-test 37/37; live checker green (sweep and counts verified, all 6 issues open); doc-constants-check green under LC_ALL=C. --- SPEC.md | 14 ++++++-- scripts/check-known-defect-issues-open | 36 +++++++++++++++++++-- scripts/test-check-known-defect-issues-open | 19 +++++++++++ spec/tracking-issues.yml | 14 +++++--- 4 files changed, 74 insertions(+), 9 deletions(-) diff --git a/SPEC.md b/SPEC.md index bb1decc0a..17729249e 100644 --- a/SPEC.md +++ b/SPEC.md @@ -3615,8 +3615,18 @@ Two dispatch clarifications, pinned: (implementation-chosen; the Go reference uses 256). At capacity the pump **blocks** — back-pressure propagates to the socket and TCP — rather than dropping: the state-machine-owned live buffer is the only place a frame can ever be dropped, and its - overflow signal is the only drop signal. Worst-case cable-lane retention is therefore - bounded multiplicatively — every retained item is itself bounded by + overflow signal is the only drop signal. **The retention ceiling below is the GO + REFERENCE IMPLEMENTATION'S**, stated in its own terms — two goroutines, a + `json.RawMessage` copy, a copying decoder — **and it presumes a transport with + bounded reads**. Every SDK's cable lane inherits the shape (bounded queue, blocking + hand-off, single deferral slot, the buffer as the only drop point) but re-derives its + own weights, and one recorded divergence already breaks the per-item premise + elsewhere: TypeScript's default global-`WebSocket` lane cannot pre-bound a read, so a + single oversized message is allocated whole at receipt, before the + `EVENT_FEED_MAX_FRAME_BYTES` check drops it — the scenario-lane table in Appendix F + records that as an accepted divergence, and no universal cross-SDK byte ceiling is + published here. In the Go reference, worst-case cable-lane retention is bounded + multiplicatively — every retained item is itself bounded by `EVENT_FEED_MAX_FRAME_BYTES`, and retention is an enumeration by HOLDER, which is what closes the count: a frame lives in the hand-off queue (≤ pump depth), in the live buffer (≤ `EVENT_FEED_LIVE_BUFFER_CAPACITY`), in the single deferral slot (≤ 1), or in diff --git a/scripts/check-known-defect-issues-open b/scripts/check-known-defect-issues-open index 8653b157c..b514eda64 100755 --- a/scripts/check-known-defect-issues-open +++ b/scripts/check-known-defect-issues-open @@ -75,12 +75,42 @@ unless File.exist?(allow_path) warn "ERROR: allowlist not found at #{allow_path}" exit 1 end +# YAML keeps only the LAST of duplicate mapping keys, so a second +# `sdk_routes_known_defective:` key would REPLACE the first list — every issue +# reference in it gone, with nothing malformed left for the shape validation +# below to see. Psych's parse tree still holds both, so duplicates are +# detected there, before safe_load flattens them. All mappings are walked, +# not just the root: a duplicated key inside one entry silently drops a field +# the same way. +def reject_duplicate_keys!(path, text) + doc = Psych.parse(text) + return if doc.nil? + + stack = [ doc.root ] + until stack.empty? + node = stack.pop + if node.is_a?(Psych::Nodes::Mapping) + dup = node.children.each_slice(2) + .map { |k, _| k.value if k.respond_to?(:value) } + .compact.tally.select { |_, n| n > 1 }.keys + unless dup.empty? + warn "ERROR: #{path} repeats mapping key(s): #{dup.join(', ')} — YAML keeps only the last, " \ + 'silently discarding the first' + exit 1 + end + end + stack.concat(node.children) if node.children + end +end + # Psych's exception family — syntax errors, forbidden tags, aliases (which # safe_load refuses) — is the same fail-closed verdict as a malformed shape, # and gets the same treatment: the gate's own words, naming the file to fix, # never a psych.rb backtrace. +allow_text = File.read(allow_path) begin - allow = YAML.safe_load(File.read(allow_path)) || {} + reject_duplicate_keys!(allow_path, allow_text) + allow = YAML.safe_load(allow_text) || {} rescue Psych::Exception => e warn "ERROR: #{allow_path} cannot be parsed as YAML: #{e.message}" exit 1 @@ -102,8 +132,10 @@ unless File.exist?(tracking_path) warn ' from one whose claims are all unverified. Restore it, or empty its list.' exit 1 end +tracking_text = File.read(tracking_path) begin - tracking = YAML.safe_load(File.read(tracking_path)) || {} + reject_duplicate_keys!(tracking_path, tracking_text) + tracking = YAML.safe_load(tracking_text) || {} rescue Psych::Exception => e warn "ERROR: #{tracking_path} cannot be parsed as YAML: #{e.message}" exit 1 diff --git a/scripts/test-check-known-defect-issues-open b/scripts/test-check-known-defect-issues-open index 9047d28a4..974210eea 100755 --- a/scripts/test-check-known-defect-issues-open +++ b/scripts/test-check-known-defect-issues-open @@ -463,6 +463,25 @@ out, status, _ = run_checker(allowlist_raw: "sdk_routes_known_defective: [unclos expect_fail(failures, "N+6f. unparseable allowlist YAML fails with a diagnosis", out, status, "cannot be parsed as YAML") +# --- N+6g/h. Duplicate mapping keys fail instead of silently last-winning -------- +# +# YAML keeps only the LAST of duplicate keys, so a second +# `sdk_routes_known_defective:` REPLACES the first list — every issue +# reference in it gone, with nothing malformed left for shape validation to +# see. Psych's parse tree still holds both, so the gate looks there first. + +out, status, _ = run_checker( + allowlist_raw: "sdk_routes_known_defective:\n - method: GET\n path: /x\n issue: 101\nsdk_routes_known_defective: []\n" +) +expect_fail(failures, "N+6g. a duplicate allowlist key fails instead of last-winning", out, status, + "repeats mapping key") + +out, status, _ = run_checker( + tracking_raw: "prose_tracking_issues:\n - issue: 303\n file: \"a.md\"\n site: \"a gap\"\n mentions: 0\nprose_tracking_issues: []\n" +) +expect_fail(failures, "N+6h. a duplicate registry key fails instead of last-winning", out, status, + "repeats mapping key") + # --- N+7. A registry file without the key fails ----------------------------------- # # The missing-FILE case (N+3) fails closed; a file whose key is misspelled or diff --git a/spec/tracking-issues.yml b/spec/tracking-issues.yml index d300723da..ffde60dfa 100644 --- a/spec/tracking-issues.yml +++ b/spec/tracking-issues.yml @@ -7,9 +7,12 @@ # it (see scripts/check-known-defect-issues-open). # # The route allowlist gets that check because its entries are structured data. -# Prose is not, so the claims are registered here instead: one entry per -# sentence that leans on an issue, verified OPEN by the same gate, failing -# closed the same way. +# Prose is not, so the claims are registered here instead — one entry per +# [file, issue] pair, committing in `mentions` how many canonical sentences it +# stands for — verified OPEN by the same gate, failing closed the same way. +# (Duplicate [file, issue] rows are rejected: they would make the committed +# count ambiguous. Several sentences in one file leaning on one issue share +# the row and sum in its count.) # # This is deliberately a REGISTRY and not a scanner. A regex over prose for # "#\d+" would sweep up every historical citation — "shipped in #12380", "the @@ -28,8 +31,9 @@ # detector's reach by design (an open vocabulary cannot be enumerated); they # still go through this registry by hand, on the honor the header describes. # -# Add an entry when you write a sentence promising an issue owns something. -# Remove it when the sentence goes, and the gate stops asking. +# Add an entry — or bump an existing row's `mentions` — when you write a +# sentence promising an issue owns something. Decrement or remove when the +# sentence goes, and the gate stops asking. prose_tracking_issues: - issue: 589 file: "Makefile" From 36da938572d44101bf4231564d48d98902e26012 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Mon, 24 Aug 2026 20:13:59 -0700 Subject: [PATCH 31/55] Drop a stray possessive in the ceiling's holder sentence --- SPEC.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/SPEC.md b/SPEC.md index 17729249e..4a0bcd827 100644 --- a/SPEC.md +++ b/SPEC.md @@ -3631,7 +3631,7 @@ Two dispatch clarifications, pinned: closes the count: a frame lives in the hand-off queue (≤ pump depth), in the live buffer (≤ `EVENT_FEED_LIVE_BUFFER_CAPACITY`), in the single deferral slot (≤ 1), or in the hands of one of the exactly two goroutines that touch frames. The pump's hand - holds one frame. The state machine's holds up to THREE frame-sized allocations at + holds one frame. The state machine holds up to THREE frame-sized allocations at once, and three is not a discovered constant but the decode chain's REPRESENTATION COUNT: a message frame exists as the wire bytes, as `parseFrame`'s `json.RawMessage` payload copy, and as the decoded `Event`'s strings — each step's output allocated From 44ec5275b51b520c89f3612ee21d1f6142c78f1f Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Mon, 24 Aug 2026 21:04:12 -0700 Subject: [PATCH 32/55] The advance rendezvous is authored: expectTimers precedes it, at load MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Codex's P2 plus three Copilot findings, all taken. * The firing-keyed advance guard read the due set from a settled state, but nothing guaranteed the state HAD settled: an action's completion can precede the timer arms its transition causes — expectConnect returns when the dial is recorded, while the handshake deadline arms on the connector's goroutine after — so an advance right behind an action raced those arms, rejected on one schedule and accepted with time moved past a deadline about to arm on the other. The earlier rounds' conclusion stands (pending arms are undetectable in general); what is new is that the DRIVER controls the preceding expectation, so the rendezvous is AUTHORED rather than guessed: every advance must be the scenario's first step or immediately follow expectTimers, whose exact-set match is the settle (§23's per-state exact-set invariants are what make a match mean settled), enforced at fixture load and stated in the schema, README, and SPEC §23. The racing self-test script and fixture 05 gain their rendezvous steps (the fixture's expectState observed the state, not the timer set — the same race one door over). Proof, honestly: 60 rounds of the racing case never lost the race on this machine (the arm wins natively), so the live flake has no captured red; the deterministic red is the load rule — the unrendezvoused script, run against the previous driver, was rejected with the race-dependent "would fire [handshake-deadline]" verdict where the rule now names the missing rendezvous, and on the other schedule that same script would have advanced silently. * Psych.parse("") returns false, not nil, so an empty registry or allowlist crashed the duplicate-key walk on false.root. Falsy parse results are empty documents; the shape validation speaks in its own words. Red-proven: the empty-registry case crashed the previous checker with the literal "undefined method 'root' for false (NoMethodError)"; all 39 cases pass after. * The cable-lane ceiling counts FRAMES, and now says so against the one non-frame item: the read error that ends the pump — at most one per attempt, because the pump exits by sending it — rides the queue or the deferral slot with a SIZE the seam contract does not bound. The built-in transport's errors are bounded by construction (fixed shapes, renderings from configured text or placeholders, never server bytes); a custom transport's error is its author's to bound. Stated, not capped. * The kill-matrix invariant still said every mutation is red-proven while row 15 sits in the table pending Layer-1. It now says fifteen of sixteen, names row 15 as the recorded exception pending the Layer-1 adapters #819 tracks, and points at the note that accounts for it — phrased outside the canonical grammar, so the README's committed mention count is unchanged. Gates: both eventfeed packages green, plain and -race; event-feed-fixtures-check green (fixture 05 and the schema changed; all 31 fixtures and both pin probes verify); tracking-issues self-test 39/39; live checker green (all 6 issues open); doc-constants-check green under LC_ALL=C; gofmt clean. --- SPEC.md | 16 +++++++++++--- conformance/event-feed/README.md | 16 ++++++++++++-- .../05-fresh-ticket-reconnect-after-ttl.json | 8 +++++++ conformance/event-feed/schema.json | 2 +- .../eventfeed/scenario_fixture_test.go | 22 +++++++++++++++++++ .../eventfeed/scenario_selftest_test.go | 16 +++++++++++++- scripts/check-known-defect-issues-open | 4 +++- scripts/test-check-known-defect-issues-open | 15 +++++++++++++ 8 files changed, 91 insertions(+), 8 deletions(-) diff --git a/SPEC.md b/SPEC.md index 4a0bcd827..7d9380176 100644 --- a/SPEC.md +++ b/SPEC.md @@ -3625,7 +3625,13 @@ Two dispatch clarifications, pinned: single oversized message is allocated whole at receipt, before the `EVENT_FEED_MAX_FRAME_BYTES` check drops it — the scenario-lane table in Appendix F records that as an accepted divergence, and no universal cross-SDK byte ceiling is - published here. In the Go reference, worst-case cable-lane retention is bounded + published here. The ceiling also counts FRAMES, not errors: exactly one non-frame item + can ride the queue or the deferral slot — the read error that ends the pump, at most + one per attempt because the pump exits by sending it — and its SIZE is + transport-authored, unbounded by the seam contract. The built-in transport's errors + are bounded by construction (fixed shapes whose renderings are configured text or + placeholders, never server bytes); a custom transport's error is its author's to + bound. In the Go reference, worst-case cable-lane retention is bounded multiplicatively — every retained item is itself bounded by `EVENT_FEED_MAX_FRAME_BYTES`, and retention is an enumeration by HOLDER, which is what closes the count: a frame lives in the hand-off queue (≤ pump depth), in the live @@ -4147,8 +4153,12 @@ advance selects under. The inversion is sound because a test clock releases that across a firing's aftermath — so an advance that fires nothing never wakes anything and cannot cause an arm, leaving nothing to detect. It is stricter than an arming rule (a firing that replaces nothing is rejected too) and that is the trade: a script wanting that -firing writes `fireTimer` and names the timer. `conformance/event-feed/schema.json`'s -`$defs.advance` states it, and the driver obligation is enforced there. +firing writes `fireTimer` and names the timer. The due-set read also needs a settled set +to read — an action's completion can precede the timer arms its transition causes — so +every `advance` must be the scenario's first step or immediately follow `expectTimers`, +whose exact-set match is the authored settle; drivers enforce the adjacency at fixture +load. `conformance/event-feed/schema.json`'s +`$defs.advance` states both, and the driver obligation is enforced there. Teardown discipline: disposing a connection attempt — deadline lapse, staleness, socket death, terminal — cancels the frame pump, **cancels any in-flight seam call belonging to diff --git a/conformance/event-feed/README.md b/conformance/event-feed/README.md index a50b31320..c67e2a4fa 100644 --- a/conformance/event-feed/README.md +++ b/conformance/event-feed/README.md @@ -183,6 +183,16 @@ says which timer it means, which is more legible anyway. The Go driver self-tests three arms — the rejection, an ordinary quiet-window advance still passing, and a firing that arms nothing being rejected all the same. +**And the due-set read needs a settled set to read.** An action's completion +can precede the timer arms its transition causes — a connect is observable +before the handshake deadline is armed on the connector's own thread — so an +advance placed right behind an action races those arms: rejected on one +schedule, accepted with time moved past a deadline about to arm on another. +The rendezvous is authored, not guessed: every `advance` must be the +scenario's first step or immediately follow `expectTimers`, whose exact-set +match is the settle (the per-state exact-set invariants are what make a match +mean settled), and drivers enforce the adjacency at fixture load. + ## Contract notes the fixtures encode (SDK-owned, final) - **Connect-to-mint-URL-verbatim.** The connector never assembles cable topology @@ -372,8 +382,10 @@ when every line is done: ## Mutation kill matrix (sixteen) -Each mutation is shown red against at least one fixture in the reference -implementation PR's body before it counts. +Fifteen of the sixteen mutations are shown red against at least one fixture in +the reference implementation PR's body before they count. Row 15 is the +recorded exception — not killed at tier 2, pending the Layer-1 adapters #819 +tracks — and the note below is its account. **One row is an exception, and it is the reason this heading is worth reading twice.** Row 15's mutation is **not killed at tier 2 at all** — it lives below diff --git a/conformance/event-feed/fixtures/05-fresh-ticket-reconnect-after-ttl.json b/conformance/event-feed/fixtures/05-fresh-ticket-reconnect-after-ttl.json index 6255dc130..f35b8f3fe 100644 --- a/conformance/event-feed/fixtures/05-fresh-ticket-reconnect-after-ttl.json +++ b/conformance/event-feed/fixtures/05-fresh-ticket-reconnect-after-ttl.json @@ -63,6 +63,14 @@ "is": "streaming" } }, + { + "expectTimers": { + "exact": { + "staleness": 1, + "repair-poll": 1 + } + } + }, { "advance": { "ms": 121000 diff --git a/conformance/event-feed/schema.json b/conformance/event-feed/schema.json index e4fe02303..79ac7e037 100644 --- a/conformance/event-feed/schema.json +++ b/conformance/event-feed/schema.json @@ -1327,7 +1327,7 @@ "required": [ "ms" ], - "description": "Advance virtual now by ms. The normative virtual-advance algorithm (README) fires due timers in deadline order, re-evaluating after each fire, with timers scheduled during the advance whose deadlines land inside the window also firing and ties breaking by creation order. That reentrant clause is UNSCRIPTABLE wherever the connector runs concurrently with the driver, so no fixture may rely on it: every driver MUST REJECT an advance whose window would fire ANY timer, naming fireTimer as the deterministic alternative. The test is what would FIRE, decided from the clock's state before time moves — not what gets armed, which happens on the connector's schedule and can only be sampled. An advance that fires nothing never wakes anything, so it cannot cause an arm; that is what makes the check complete rather than probabilistic. A firing that replaces nothing is rejected too.", + "description": "Advance virtual now by ms. The normative virtual-advance algorithm (README) fires due timers in deadline order, re-evaluating after each fire, with timers scheduled during the advance whose deadlines land inside the window also firing and ties breaking by creation order. That reentrant clause is UNSCRIPTABLE wherever the connector runs concurrently with the driver, so no fixture may rely on it: every driver MUST REJECT an advance whose window would fire ANY timer, naming fireTimer as the deterministic alternative. The test is what would FIRE, decided from the clock's state before time moves — not what gets armed, which happens on the connector's schedule and can only be sampled. An advance that fires nothing never wakes anything, so it cannot cause an arm; that is what makes the check complete rather than probabilistic. A firing that replaces nothing is rejected too. And an advance is deterministic only from a scripted rendezvous point: an action's completion can precede the timer arms its transition causes, so every advance must be the scenario's first step or immediately follow expectTimers — whose exact-set match is the authored settle — and drivers enforce that adjacency at load.", "properties": { "ms": { "type": "integer", diff --git a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go index 27fb5b666..05bfaaee7 100644 --- a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go @@ -321,6 +321,28 @@ func parseScenario(raw []byte, file string) (*scenario, error) { } sc.Steps = append(sc.Steps, step) } + + // An advance is deterministic only from a scripted rendezvous point. An + // action's completion can precede the timer arms its transition causes — + // expectConnect returns when the dial is recorded, while the handshake + // deadline arms on the connector's goroutine after — so an advance right + // behind an action races those arms: the same script is rejected on one + // schedule ("would fire") and accepted on another, with time moved past a + // deadline about to arm. The rendezvous is AUTHORED, not guessed: + // expectTimers polls until the outstanding set exactly matches a settled + // state's set (§23's per-state exact-set invariants are what make the + // match a settle), and this load rule makes its absence unscriptable. + for i, step := range sc.Steps { + if step.Kind != "advance" || i == 0 { + continue + } + if sc.Steps[i-1].Kind != "expectTimers" { + return nil, fmt.Errorf("step %d: an advance must be the scenario's first step or immediately follow "+ + "expectTimers — an action's completion can precede the timer arms its transition causes, so an "+ + "unrendezvoused advance cannot mean the same thing on every schedule; expectTimers' exact-set "+ + "match is the settle", i+1) + } + } finRaw, ok := top["finally"] if !ok { return nil, fmt.Errorf("fixture is missing its `finally` block") diff --git a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go index e36c5158c..65391b5f9 100644 --- a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go @@ -81,7 +81,7 @@ func TestScenarioDriverRejectsMutatedFixtures(t *testing.T) { { name: "reconnect dials the previous mint's url", fixture: "05-fresh-ticket-reconnect-after-ttl.json", - path: "steps.13.expectConnect.url", + path: "steps.14.expectConnect.url", value: "{{CABLE_URL:1}}", wants: "a cable dial", }, @@ -462,6 +462,19 @@ func TestScenarioDriverRejectsUnmodelledScripts(t *testing.T) { script: `{"name":"x","description":"d","config":{"stalenessMs":9223372036855},"steps":[{"advance":{"ms":1}}],"finally":{"state":"closed"}}`, wants: "10 virtual years", }, + { + // An advance is deterministic only from a scripted rendezvous: + // an action's completion can precede the timer arms its + // transition causes (expectConnect returns when the dial is + // recorded; the handshake deadline arms on the connector's + // goroutine after), so an unrendezvoused advance races the arm — + // accepted on one schedule, rejected on another. The rendezvous + // is authored, not guessed: expectTimers' exact-set match is the + // settle, and the load rule makes its absence unscriptable. + name: "an advance not preceded by an expectTimers rendezvous", + script: `{"name":"x","description":"d","steps":[{"expectMint":{"respond":{"status":200,"body":{"ticket":"{{TICKET:1}}","expires_in":120,"url":"{{CABLE_URL:1}}"}}}},{"expectConnect":{"url":"{{CABLE_URL:1}}"}},{"advance":{"ms":30000}}],"finally":{"state":"closed"}}`, + wants: "must be the scenario's first step or immediately follow expectTimers", + }, { name: "droppedCount disagreeing with droppedIds", script: `{"name":"x","description":"d","steps":[{"expectSignal":{"kind":"bufferOverflow","droppedIds":[1],"droppedCount":2}}],"finally":{"state":"closed"}}`, @@ -556,6 +569,7 @@ func TestScenarioDriverRejectsSchedulingDependentAdvance(t *testing.T) { script := `{"name":"x","description":"d","steps":[ {"expectMint":{"respond":{"status":200,"body":{"ticket":"{{TICKET:1}}","expires_in":120,"url":"{{CABLE_URL:1}}"}}}}, {"expectConnect":{"url":"{{CABLE_URL:1}}"}}, + {"expectTimers":{"exact":{"handshake-deadline":1}}}, {"advance":{"ms":30000}}], "finally":{"state":"backoff"}}` err := underShortWatchdog(func() error { return runScenarioBytes([]byte(script), "x.json") }) diff --git a/scripts/check-known-defect-issues-open b/scripts/check-known-defect-issues-open index b514eda64..589a4a415 100755 --- a/scripts/check-known-defect-issues-open +++ b/scripts/check-known-defect-issues-open @@ -84,7 +84,9 @@ end # the same way. def reject_duplicate_keys!(path, text) doc = Psych.parse(text) - return if doc.nil? + # Psych.parse("") returns false, not nil: every falsy result is an empty + # document with no keys to duplicate — the shape validation below speaks. + return unless doc stack = [ doc.root ] until stack.empty? diff --git a/scripts/test-check-known-defect-issues-open b/scripts/test-check-known-defect-issues-open index 974210eea..d8d26ed40 100755 --- a/scripts/test-check-known-defect-issues-open +++ b/scripts/test-check-known-defect-issues-open @@ -482,6 +482,21 @@ out, status, _ = run_checker( expect_fail(failures, "N+6h. a duplicate registry key fails instead of last-winning", out, status, "repeats mapping key") +# --- N+6i/j. An EMPTY file is an empty document, not a crash --------------------- +# +# Psych.parse("") returns false, not nil, so an empty file reached the parse +# tree walk and crashed on false.root — nonzero, but a NoMethodError is not +# the gate's diagnosis. Falsy parse results are empty documents; the shape +# validation below then speaks in its own words. + +out, status, _ = run_checker(tracking_raw: "") +expect_fail(failures, "N+6i. an empty registry file fails with the key diagnosis", out, status, + "must define prose_tracking_issues") + +out, status, _ = run_checker(allowlist_raw: "") +expect_pass(failures, "N+6j. an empty allowlist file is an empty allowlist", out, status, + "no tracking issues referenced") + # --- N+7. A registry file without the key fails ----------------------------------- # # The missing-FILE case (N+3) fails closed; a file whose key is misspelled or From 2ff4de3a1ccb4be251403d2209eb6413988ea97d Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Mon, 24 Aug 2026 21:26:12 -0700 Subject: [PATCH 33/55] A rendezvous must be able to mean settled, and a registry is one document MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Codex's re-raised P2 and Copilot's multi-document finding, both taken. * The premise of the re-raise is half wrong and half right, and the half matters. expectTimers is NOT an instantaneous comparison — the step dispatches to awaitTimers, which polls under the scenario watchdog until the outstanding set exactly matches (the snapshot compare at stepSatisfiedLocked is the lookahead probe, not the executor). What survives is the informationless-match class: a match can only mean settled when the authored set could not exist BEFORE the preceding transition was processed. Codex's own example is the decidable case — an empty set can never contain an arm of the preceding transition, so {"exact":{}} after a released failed mint matches before backoff arms, exactly as if the rendezvous were absent — and the load rule now rejects it: an advance's rendezvous set must be non-empty (a scenario with nothing yet armed advances as its first step). The limit no load rule can close is stated in the schema, README, and SPEC §23 instead of implied: the authored set must include at least one timer the preceding transition ARMS, and a transition that only rearms a timer of the same kind and count is invisible to set matching — such scripts use fireTimer. Red-proven: the empty-rendezvous script against the previous driver consumed its 5s watchdog (empty match accepted, advance accepted) and failed only for want of the fragment the rule now emits. * Psych.parse and safe_load read only the FIRST YAML document, so everything after a `---` separator — registry rows, allowlist entries — silently vanished from the live check. The gate now parses the STREAM and rejects a file holding more than one document with its own diagnostic. Red-proven: a two-document registry hiding a row and a two-document allowlist hiding a defective route both passed the previous checker ("expected FAILURE but checker passed"); all 41 cases pass after the fix. The empty-file path moved with it: no documents at all is the empty-document case, and the shape validation still speaks. One honest note: a single -race batch run failed once with its output discarded by the batch's redirect; three subsequent captured runs (-count=1 and -count=2) are clean with no data-race report, consistent with the wall-clock-under-load flake class #794 deflaked — not reproduced, and nothing in this commit touches connector concurrency. Gates: tracking-issues self-test 41/41; live checker green (all 6 issues open); event-feed-fixtures-check green (schema changed); both eventfeed packages green, plain and -race ×3; doc-constants-check green under LC_ALL=C; gofmt clean. --- SPEC.md | 5 ++++- conformance/event-feed/README.md | 9 ++++++++- conformance/event-feed/schema.json | 2 +- .../eventfeed/scenario_fixture_test.go | 12 +++++++++++ .../eventfeed/scenario_selftest_test.go | 11 ++++++++++ scripts/check-known-defect-issues-open | 20 ++++++++++++++----- scripts/test-check-known-defect-issues-open | 18 +++++++++++++++++ 7 files changed, 69 insertions(+), 8 deletions(-) diff --git a/SPEC.md b/SPEC.md index 7d9380176..3f4121309 100644 --- a/SPEC.md +++ b/SPEC.md @@ -4157,7 +4157,10 @@ firing writes `fireTimer` and names the timer. The due-set read also needs a set to read — an action's completion can precede the timer arms its transition causes — so every `advance` must be the scenario's first step or immediately follow `expectTimers`, whose exact-set match is the authored settle; drivers enforce the adjacency at fixture -load. `conformance/event-feed/schema.json`'s +load, and reject an empty rendezvous set with it (no arm of the preceding transition can +be in it, so its match orders nothing). The authored set must include an arm of the +preceding transition — a same-kind, same-count rearm is invisible to set matching, and a +script that would advance behind one uses `fireTimer`. `conformance/event-feed/schema.json`'s `$defs.advance` states both, and the driver obligation is enforced there. Teardown discipline: disposing a connection attempt — deadline lapse, staleness, socket diff --git a/conformance/event-feed/README.md b/conformance/event-feed/README.md index c67e2a4fa..02f4113e6 100644 --- a/conformance/event-feed/README.md +++ b/conformance/event-feed/README.md @@ -191,7 +191,14 @@ schedule, accepted with time moved past a deadline about to arm on another. The rendezvous is authored, not guessed: every `advance` must be the scenario's first step or immediately follow `expectTimers`, whose exact-set match is the settle (the per-state exact-set invariants are what make a match -mean settled), and drivers enforce the adjacency at fixture load. +mean settled), and drivers enforce the adjacency at fixture load — rejecting an +empty rendezvous set with it, since a set with no timers cannot contain an arm +of the preceding transition and so matches before that transition is processed. +The limit the load rule cannot close is authored judgment: the scripted set +must include at least one timer the preceding transition ARMS, and a +transition that only rearms a timer of the same kind and count is invisible to +set matching — a script that would advance behind one writes `fireTimer` +instead. ## Contract notes the fixtures encode (SDK-owned, final) diff --git a/conformance/event-feed/schema.json b/conformance/event-feed/schema.json index 79ac7e037..b6513611b 100644 --- a/conformance/event-feed/schema.json +++ b/conformance/event-feed/schema.json @@ -1327,7 +1327,7 @@ "required": [ "ms" ], - "description": "Advance virtual now by ms. The normative virtual-advance algorithm (README) fires due timers in deadline order, re-evaluating after each fire, with timers scheduled during the advance whose deadlines land inside the window also firing and ties breaking by creation order. That reentrant clause is UNSCRIPTABLE wherever the connector runs concurrently with the driver, so no fixture may rely on it: every driver MUST REJECT an advance whose window would fire ANY timer, naming fireTimer as the deterministic alternative. The test is what would FIRE, decided from the clock's state before time moves — not what gets armed, which happens on the connector's schedule and can only be sampled. An advance that fires nothing never wakes anything, so it cannot cause an arm; that is what makes the check complete rather than probabilistic. A firing that replaces nothing is rejected too. And an advance is deterministic only from a scripted rendezvous point: an action's completion can precede the timer arms its transition causes, so every advance must be the scenario's first step or immediately follow expectTimers — whose exact-set match is the authored settle — and drivers enforce that adjacency at load.", + "description": "Advance virtual now by ms. The normative virtual-advance algorithm (README) fires due timers in deadline order, re-evaluating after each fire, with timers scheduled during the advance whose deadlines land inside the window also firing and ties breaking by creation order. That reentrant clause is UNSCRIPTABLE wherever the connector runs concurrently with the driver, so no fixture may rely on it: every driver MUST REJECT an advance whose window would fire ANY timer, naming fireTimer as the deterministic alternative. The test is what would FIRE, decided from the clock's state before time moves — not what gets armed, which happens on the connector's schedule and can only be sampled. An advance that fires nothing never wakes anything, so it cannot cause an arm; that is what makes the check complete rather than probabilistic. A firing that replaces nothing is rejected too. And an advance is deterministic only from a scripted rendezvous point: an action's completion can precede the timer arms its transition causes, so every advance must be the scenario's first step or immediately follow expectTimers — whose exact-set match is the authored settle — and drivers enforce that adjacency at load, rejecting an empty rendezvous set too (it cannot contain an arm of the preceding transition, so its match orders nothing). The authored set must include at least one timer the preceding transition arms; a transition that only REARMS a timer of the same kind and count is invisible to set matching, and a script that would advance behind one uses fireTimer instead.", "properties": { "ms": { "type": "integer", diff --git a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go index 05bfaaee7..6e0ddc870 100644 --- a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go @@ -342,6 +342,18 @@ func parseScenario(raw []byte, file string) (*scenario, error) { "unrendezvoused advance cannot mean the same thing on every schedule; expectTimers' exact-set "+ "match is the settle", i+1) } + // The match must be able to MEAN settled: an empty set can never + // contain an arm of the preceding transition, so it matches before + // that transition is processed (a released failed mint has not armed + // backoff yet) exactly as if the rendezvous were absent. The limit + // this cannot close is stated in the contract: the authored set must + // include an arm of the preceding transition, and a same-kind rearm + // is invisible to set matching — such scripts use fireTimer. + if rv, ok := sc.Steps[i-1].Payload.(*timerSet); ok && len(rv.Exact) == 0 { + return nil, fmt.Errorf("step %d: an empty rendezvous orders nothing — an expectTimers set with no "+ + "timers cannot contain an arm of the preceding transition, so its match cannot prove the "+ + "transition settled; a scenario with nothing yet armed advances as its first step", i+1) + } } finRaw, ok := top["finally"] if !ok { diff --git a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go index 65391b5f9..ae9296236 100644 --- a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go @@ -475,6 +475,17 @@ func TestScenarioDriverRejectsUnmodelledScripts(t *testing.T) { script: `{"name":"x","description":"d","steps":[{"expectMint":{"respond":{"status":200,"body":{"ticket":"{{TICKET:1}}","expires_in":120,"url":"{{CABLE_URL:1}}"}}}},{"expectConnect":{"url":"{{CABLE_URL:1}}"}},{"advance":{"ms":30000}}],"finally":{"state":"closed"}}`, wants: "must be the scenario's first step or immediately follow expectTimers", }, + { + // An EMPTY rendezvous set can never contain an arm of the + // preceding transition, so its match orders nothing: after a + // released failed mint, {"exact":{}} matches before backoff is + // armed, and the advance races the arm exactly as if the + // rendezvous were absent. A scenario with nothing yet armed + // advances as its first step instead. + name: "an advance behind an empty expectTimers rendezvous", + script: `{"name":"x","description":"d","steps":[{"expectTimers":{"exact":{}}},{"advance":{"ms":1}}],"finally":{"state":"closed"}}`, + wants: "an empty rendezvous orders nothing", + }, { name: "droppedCount disagreeing with droppedIds", script: `{"name":"x","description":"d","steps":[{"expectSignal":{"kind":"bufferOverflow","droppedIds":[1],"droppedCount":2}}],"finally":{"state":"closed"}}`, diff --git a/scripts/check-known-defect-issues-open b/scripts/check-known-defect-issues-open index 589a4a415..55d56021c 100755 --- a/scripts/check-known-defect-issues-open +++ b/scripts/check-known-defect-issues-open @@ -83,12 +83,22 @@ end # not just the root: a duplicated key inside one entry silently drops a field # the same way. def reject_duplicate_keys!(path, text) - doc = Psych.parse(text) - # Psych.parse("") returns false, not nil: every falsy result is an empty - # document with no keys to duplicate — the shape validation below speaks. - return unless doc + # The STREAM is parsed, not the first document: Psych.parse and safe_load + # read only the first, so everything after a `---` separator — rows, + # entries, whole lists — would silently vanish from the check. A registry + # cannot mean "the first document of". + stream = Psych.parse_stream(text) + docs = stream ? stream.children : [] + if docs.length > 1 + warn "ERROR: #{path} holds #{docs.length} YAML documents — only the first is ever read, so everything " \ + 'after the `---` separator would silently vanish from the check' + exit 1 + end + # An empty file parses to no documents at all — nothing to walk; the shape + # validation below speaks. + return if docs.empty? - stack = [ doc.root ] + stack = [ docs.first ] until stack.empty? node = stack.pop if node.is_a?(Psych::Nodes::Mapping) diff --git a/scripts/test-check-known-defect-issues-open b/scripts/test-check-known-defect-issues-open index d8d26ed40..0d783e751 100755 --- a/scripts/test-check-known-defect-issues-open +++ b/scripts/test-check-known-defect-issues-open @@ -497,6 +497,24 @@ out, status, _ = run_checker(allowlist_raw: "") expect_pass(failures, "N+6j. an empty allowlist file is an empty allowlist", out, status, "no tracking issues referenced") +# --- N+6k/l. A multi-document file fails instead of dropping documents ----------- +# +# Psych.parse and safe_load read only the FIRST YAML document: everything +# after a `---` separator — rows, entries, whole lists — silently vanishes +# from the live check. A registry cannot mean "the first document of". + +out, status, _ = run_checker( + tracking_raw: "prose_tracking_issues: []\n---\nprose_tracking_issues:\n - issue: 999\n file: \"a.md\"\n site: \"a hidden promise\"\n mentions: 0\n" +) +expect_fail(failures, "N+6k. a multi-document registry fails instead of dropping rows", out, status, + "YAML documents") + +out, status, _ = run_checker( + allowlist_raw: "sdk_routes_known_defective: []\n---\nsdk_routes_known_defective:\n - method: GET\n path: /x\n issue: 101\n" +) +expect_fail(failures, "N+6l. a multi-document allowlist fails instead of dropping entries", out, status, + "YAML documents") + # --- N+7. A registry file without the key fails ----------------------------------- # # The missing-FILE case (N+3) fails closed; a file whose key is misspelled or From ee4975e3a9c2130d3960f3ba90ef4bd78f31ac41 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Mon, 24 Aug 2026 21:54:53 -0700 Subject: [PATCH 34/55] Presence is meaning for config durations, and the rendezvous is two steps MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot's explicit-zero finding and Codex's third settle round, both taken — and the second fix caught the first live instance of its own bug class inside our own self-test. * Explicit zero is not omission. The five config durations decoded into plain int64, so {"stalenessMs":0} was indistinguishable from absence: the range check skipped it and the driver silently used the default — accepting a value the schema rejects (minimum 1) and substituting another. The fields are pointers now; every SUPPLIED value is ranged, explicit zero included, and the unmodeled backoff pair is rejected at ANY supplied value (explicit zero used to slip that check too). Red-proven: four zero-boundary self-tests (the three ranged durations, one unmodeled) all failed against the previous driver. * The settle instrument, round three. Codex's counter is real: the welcome transition stops handshake-deadline and arms confirmation-deadline in separate clock acquisitions, so an exact-set match ALONE can coincide with a transient mid-surgery set. The audit of every setState site found the codebase already holds "surgery before announcement" deliberately at streaming, awaiting-welcome, and awaiting-confirmation — with Backoff and Connecting announcing before a single tail arm. That shape is what makes the TWO-STEP rendezvous sound without touching the connector: expectState blocks until the transition announces (bounding the surgery), and any timer still unarmed at an announcement is exactly what the following exact-set match waits for. Every advance must now be first or follow expectState + expectTimers, enforced at load; both steps block under the watchdog, so wrong authorship fails loudly rather than diverging. The residue is stated, not implied: a transition that announces no state change or only rearms a same-kind same-count timer is invisible to both barriers — such scripts use fireTimer. Schema, README, and SPEC §23 carry the rule. The vindication: adding the state barrier to the racing self-test immediately failed it with "outstanding timers map[handshake-deadline:1 staleness:1], want exactly map[handshake-deadline:1]" — the old expectTimers-only script had been matching {handshake-deadline:1} in the pre-staleness TRANSIENT, the precise coincidence Codex described, live in our own suite. The script now authors the true settled set. Red-proven for the rule itself: the barrier-less script against the previous driver was rejected only by the race-dependent "would fire" verdict, never the rendezvous diagnostic; fixture 05 already conformed. Gates: both eventfeed packages green, plain and -race; event-feed-fixtures-check green; tracking-issues live checker green (all 6 issues open); doc-constants-check green under LC_ALL=C; gofmt clean. --- SPEC.md | 16 +++-- conformance/event-feed/README.md | 26 ++++--- conformance/event-feed/schema.json | 2 +- .../eventfeed/scenario_fixture_test.go | 67 +++++++++++-------- .../eventfeed/scenario_harness_test.go | 12 ++-- .../eventfeed/scenario_selftest_test.go | 46 ++++++++++++- 6 files changed, 113 insertions(+), 56 deletions(-) diff --git a/SPEC.md b/SPEC.md index 3f4121309..d0a868a4d 100644 --- a/SPEC.md +++ b/SPEC.md @@ -4155,12 +4155,16 @@ cannot cause an arm, leaving nothing to detect. It is stricter than an arming ru firing that replaces nothing is rejected too) and that is the trade: a script wanting that firing writes `fireTimer` and names the timer. The due-set read also needs a settled set to read — an action's completion can precede the timer arms its transition causes — so -every `advance` must be the scenario's first step or immediately follow `expectTimers`, -whose exact-set match is the authored settle; drivers enforce the adjacency at fixture -load, and reject an empty rendezvous set with it (no arm of the preceding transition can -be in it, so its match orders nothing). The authored set must include an arm of the -preceding transition — a same-kind, same-count rearm is invisible to set matching, and a -script that would advance behind one uses `fireTimer`. `conformance/event-feed/schema.json`'s +every `advance` must be the scenario's first step or immediately follow the two-step +rendezvous `expectState` then `expectTimers`, enforced at fixture load (an empty +rendezvous set is rejected with it — it orders nothing). Neither step alone settles: a +set match can coincide with a transient mid-surgery set (timer surgery spans clock +acquisitions), and an announcement can precede a tail arm. Together they do — the +announcement bounds the surgery, and any timer still unarmed at the announcement is +exactly what the exact-set match then waits for, both blocking under the watchdog so +wrong authorship fails loudly. A transition that announces no state change, or only +rearms a timer of the same kind and count, is invisible to this rendezvous — a script +that would advance behind one uses `fireTimer`. `conformance/event-feed/schema.json`'s `$defs.advance` states both, and the driver obligation is enforced there. Teardown discipline: disposing a connection attempt — deadline lapse, staleness, socket diff --git a/conformance/event-feed/README.md b/conformance/event-feed/README.md index 02f4113e6..7fedbf04d 100644 --- a/conformance/event-feed/README.md +++ b/conformance/event-feed/README.md @@ -188,17 +188,21 @@ can precede the timer arms its transition causes — a connect is observable before the handshake deadline is armed on the connector's own thread — so an advance placed right behind an action races those arms: rejected on one schedule, accepted with time moved past a deadline about to arm on another. -The rendezvous is authored, not guessed: every `advance` must be the -scenario's first step or immediately follow `expectTimers`, whose exact-set -match is the settle (the per-state exact-set invariants are what make a match -mean settled), and drivers enforce the adjacency at fixture load — rejecting an -empty rendezvous set with it, since a set with no timers cannot contain an arm -of the preceding transition and so matches before that transition is processed. -The limit the load rule cannot close is authored judgment: the scripted set -must include at least one timer the preceding transition ARMS, and a -transition that only rearms a timer of the same kind and count is invisible to -set matching — a script that would advance behind one writes `fireTimer` -instead. +The rendezvous is authored, not guessed, and it is TWO steps: every `advance` +must be the scenario's first step or immediately follow `expectState` then +`expectTimers`, enforced at fixture load (an empty `expectTimers` set is +rejected with it — it orders nothing). Neither step alone settles. A set match +can coincide with a transient mid-surgery set: the welcome transition stops +`handshake-deadline` and arms `confirmation-deadline` in separate clock +acquisitions, so an authored set can exist in the gap. An announcement can +precede a tail arm: Backoff announces before its timer is armed. Together they +settle — the announcement bounds the surgery, and any timer still unarmed at +the announcement is exactly what the following exact-set match waits for. Both +steps block under the scenario watchdog, so a wrongly authored state or set +fails loudly instead of diverging silently. What stays outside this +rendezvous is a transition that announces no state change, or only rearms a +timer of the same kind and count — invisible to both barriers; a script that +would advance behind one writes `fireTimer` instead. ## Contract notes the fixtures encode (SDK-owned, final) diff --git a/conformance/event-feed/schema.json b/conformance/event-feed/schema.json index b6513611b..dbbb3057c 100644 --- a/conformance/event-feed/schema.json +++ b/conformance/event-feed/schema.json @@ -1327,7 +1327,7 @@ "required": [ "ms" ], - "description": "Advance virtual now by ms. The normative virtual-advance algorithm (README) fires due timers in deadline order, re-evaluating after each fire, with timers scheduled during the advance whose deadlines land inside the window also firing and ties breaking by creation order. That reentrant clause is UNSCRIPTABLE wherever the connector runs concurrently with the driver, so no fixture may rely on it: every driver MUST REJECT an advance whose window would fire ANY timer, naming fireTimer as the deterministic alternative. The test is what would FIRE, decided from the clock's state before time moves — not what gets armed, which happens on the connector's schedule and can only be sampled. An advance that fires nothing never wakes anything, so it cannot cause an arm; that is what makes the check complete rather than probabilistic. A firing that replaces nothing is rejected too. And an advance is deterministic only from a scripted rendezvous point: an action's completion can precede the timer arms its transition causes, so every advance must be the scenario's first step or immediately follow expectTimers — whose exact-set match is the authored settle — and drivers enforce that adjacency at load, rejecting an empty rendezvous set too (it cannot contain an arm of the preceding transition, so its match orders nothing). The authored set must include at least one timer the preceding transition arms; a transition that only REARMS a timer of the same kind and count is invisible to set matching, and a script that would advance behind one uses fireTimer instead.", + "description": "Advance virtual now by ms. The normative virtual-advance algorithm (README) fires due timers in deadline order, re-evaluating after each fire, with timers scheduled during the advance whose deadlines land inside the window also firing and ties breaking by creation order. That reentrant clause is UNSCRIPTABLE wherever the connector runs concurrently with the driver, so no fixture may rely on it: every driver MUST REJECT an advance whose window would fire ANY timer, naming fireTimer as the deterministic alternative. The test is what would FIRE, decided from the clock's state before time moves — not what gets armed, which happens on the connector's schedule and can only be sampled. An advance that fires nothing never wakes anything, so it cannot cause an arm; that is what makes the check complete rather than probabilistic. A firing that replaces nothing is rejected too. And an advance is deterministic only from a scripted rendezvous point: an action's completion can precede the timer arms its transition causes, so every advance must be the scenario's first step or immediately follow a TWO-STEP rendezvous: expectState, then expectTimers, enforced at load (an empty expectTimers set is rejected with it — it orders nothing). Neither step alone settles: a set match can coincide with a transient mid-surgery set (the welcome transition stops handshake-deadline and arms confirmation-deadline in separate clock acquisitions), and an announcement can precede a tail arm. Together they do: the announcement bounds the surgery, and any timer still unarmed at the announcement is exactly what the exact-set match then waits for. Both block under the watchdog, so wrong authorship fails loudly rather than diverging. A transition that announces no state change — or only rearms a timer of the same kind and count — is invisible to this rendezvous, and a script that would advance behind one uses fireTimer instead.", "properties": { "ms": { "type": "integer", diff --git a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go index 6e0ddc870..ee74d7fe6 100644 --- a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go @@ -39,18 +39,22 @@ type scenarioStep struct { // scenarioConfig is the schema's `config` object — the connector construction // options a scenario selects. type scenarioConfig struct { - Types []string `json:"types"` - Buckets []int64 `json:"buckets"` - Creators []int64 `json:"creators"` - Performers []int64 `json:"performers"` - ExcludePerformers []int64 `json:"exclude_performers"` - ActorTypes []string `json:"actorTypes"` - Position string `json:"position"` - ConfirmationDeadlineMs int64 `json:"confirmationDeadlineMs"` - RepairPollBaseMs int64 `json:"repairPollBaseMs"` - BackoffBaseMs int64 `json:"backoffBaseMs"` - BackoffCapMs int64 `json:"backoffCapMs"` - StalenessMs int64 `json:"stalenessMs"` + Types []string `json:"types"` + Buckets []int64 `json:"buckets"` + Creators []int64 `json:"creators"` + Performers []int64 `json:"performers"` + ExcludePerformers []int64 `json:"exclude_performers"` + ActorTypes []string `json:"actorTypes"` + Position string `json:"position"` + // The five durations are POINTERS because presence is meaning: the + // schema says minimum 1, and a plain int64 read an explicit zero as + // "absent, use the default" — accepting a value the schema rejects and + // silently substituting another. + ConfirmationDeadlineMs *int64 `json:"confirmationDeadlineMs"` + RepairPollBaseMs *int64 `json:"repairPollBaseMs"` + BackoffBaseMs *int64 `json:"backoffBaseMs"` + BackoffCapMs *int64 `json:"backoffCapMs"` + StalenessMs *int64 `json:"stalenessMs"` LiveBufferCapacity int `json:"liveBufferCapacity"` DedupeCapacity int `json:"dedupeCapacity"` SignalDisposition map[string]string `json:"signalDisposition"` @@ -322,25 +326,29 @@ func parseScenario(raw []byte, file string) (*scenario, error) { sc.Steps = append(sc.Steps, step) } - // An advance is deterministic only from a scripted rendezvous point. An + // An advance is deterministic only from a scripted rendezvous point, and + // the rendezvous is TWO steps: expectState, then expectTimers. An // action's completion can precede the timer arms its transition causes — // expectConnect returns when the dial is recorded, while the handshake - // deadline arms on the connector's goroutine after — so an advance right - // behind an action races those arms: the same script is rejected on one - // schedule ("would fire") and accepted on another, with time moved past a - // deadline about to arm. The rendezvous is AUTHORED, not guessed: - // expectTimers polls until the outstanding set exactly matches a settled - // state's set (§23's per-state exact-set invariants are what make the - // match a settle), and this load rule makes its absence unscriptable. + // deadline arms on the connector's goroutine after — and a set match + // ALONE can coincide with a transient mid-surgery set (the welcome + // transition stops handshake-deadline and arms confirmation-deadline in + // separate clock acquisitions, so an authored set can exist in between). + // The state announcement bounds the surgery: expectState blocks until + // the transition announces, and in every announced state any timer still + // unarmed at the announcement is exactly what the following exact-set + // match waits for — so the pair settles where either alone races. Both + // blocks fail loudly on the watchdog when the authored state or set is + // wrong; nothing diverges silently. for i, step := range sc.Steps { if step.Kind != "advance" || i == 0 { continue } - if sc.Steps[i-1].Kind != "expectTimers" { + if i < 2 || sc.Steps[i-1].Kind != "expectTimers" || sc.Steps[i-2].Kind != "expectState" { return nil, fmt.Errorf("step %d: an advance must be the scenario's first step or immediately follow "+ - "expectTimers — an action's completion can precede the timer arms its transition causes, so an "+ - "unrendezvoused advance cannot mean the same thing on every schedule; expectTimers' exact-set "+ - "match is the settle", i+1) + "an expectState + expectTimers rendezvous — an action's completion can precede the timer arms "+ + "its transition causes, and a set match alone can coincide with a transient mid-surgery set; "+ + "the state announcement bounds the surgery and the exact-set match settles what follows it", i+1) } // The match must be able to MEAN settled: an empty set can never // contain an arm of the preceding transition, so it matches before @@ -560,22 +568,23 @@ type ( // --- validation ---------------------------------------------------------- func validateConfig(cfg scenarioConfig) error { - // Absent decodes as 0 and means "default", so only set values are ranged. + // A nil pointer is absence and means "default"; every SUPPLIED value is + // ranged, explicit zero included. for _, f := range []struct { name string - v int64 + v *int64 }{ {"confirmationDeadlineMs", cfg.ConfirmationDeadlineMs}, {"repairPollBaseMs", cfg.RepairPollBaseMs}, {"stalenessMs", cfg.StalenessMs}, } { - if f.v != 0 { - if err := checkScenarioMs(f.name, f.v, 1); err != nil { + if f.v != nil { + if err := checkScenarioMs(f.name, *f.v, 1); err != nil { return err } } } - if cfg.BackoffBaseMs != 0 || cfg.BackoffCapMs != 0 { + if cfg.BackoffBaseMs != nil || cfg.BackoffCapMs != nil { return fmt.Errorf("backoffBaseMs/backoffCapMs are not modeled: SPEC §23 pins the Go connector's full-jitter base and cap as constants, with no construction option to override") } for kind, disposition := range cfg.SignalDisposition { diff --git a/go/pkg/basecamp/eventfeed/scenario_harness_test.go b/go/pkg/basecamp/eventfeed/scenario_harness_test.go index d556561e6..1a912f1fe 100644 --- a/go/pkg/basecamp/eventfeed/scenario_harness_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_harness_test.go @@ -708,11 +708,11 @@ func (h *scenarioHarness) newConnector(cfg scenarioConfig) (*eventfeed.Connector }, }), } - if cfg.ConfirmationDeadlineMs > 0 { - opts = append(opts, eventfeed.WithConfirmationDeadline(millis(cfg.ConfirmationDeadlineMs))) + if cfg.ConfirmationDeadlineMs != nil { + opts = append(opts, eventfeed.WithConfirmationDeadline(millis(*cfg.ConfirmationDeadlineMs))) } - if cfg.RepairPollBaseMs > 0 { - opts = append(opts, eventfeed.WithRepairInterval(millis(cfg.RepairPollBaseMs))) + if cfg.RepairPollBaseMs != nil { + opts = append(opts, eventfeed.WithRepairInterval(millis(*cfg.RepairPollBaseMs))) } if cfg.LiveBufferCapacity > 0 { opts = append(opts, eventfeed.WithLiveBufferCapacity(cfg.LiveBufferCapacity)) @@ -728,8 +728,8 @@ func (h *scenarioHarness) newConnector(cfg scenarioConfig) (*eventfeed.Connector if err != nil { return nil, err } - if cfg.StalenessMs > 0 { - conn.SetStaleAfter(millis(cfg.StalenessMs)) + if cfg.StalenessMs != nil { + conn.SetStaleAfter(millis(*cfg.StalenessMs)) } conn.OnStateChanged(h.recordState) conn.OnBufferOccupancy(h.recordOccupancy) diff --git a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go index ae9296236..784d136d6 100644 --- a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go @@ -462,6 +462,32 @@ func TestScenarioDriverRejectsUnmodelledScripts(t *testing.T) { script: `{"name":"x","description":"d","config":{"stalenessMs":9223372036855},"steps":[{"advance":{"ms":1}}],"finally":{"state":"closed"}}`, wants: "10 virtual years", }, + { + // Explicit zero is not omission: the schema says minimum 1, and a + // plain int64 decode read {"stalenessMs":0} as "absent, use the + // default" — accepting a value the schema rejects and silently + // substituting another. Presence is preserved with pointers. + name: "config stalenessMs explicit zero", + script: `{"name":"x","description":"d","config":{"stalenessMs":0},"steps":[{"advance":{"ms":1}}],"finally":{"state":"closed"}}`, + wants: "must be in [1,", + }, + { + name: "config confirmationDeadlineMs explicit zero", + script: `{"name":"x","description":"d","config":{"confirmationDeadlineMs":0},"steps":[{"advance":{"ms":1}}],"finally":{"state":"closed"}}`, + wants: "must be in [1,", + }, + { + name: "config repairPollBaseMs explicit zero", + script: `{"name":"x","description":"d","config":{"repairPollBaseMs":0},"steps":[{"advance":{"ms":1}}],"finally":{"state":"closed"}}`, + wants: "must be in [1,", + }, + { + // The unmodeled pair is unmodeled at ANY supplied value: explicit + // zero used to slip past the != 0 check into silence. + name: "config backoffBaseMs explicit zero", + script: `{"name":"x","description":"d","config":{"backoffBaseMs":0},"steps":[{"advance":{"ms":1}}],"finally":{"state":"closed"}}`, + wants: "not modeled", + }, { // An advance is deterministic only from a scripted rendezvous: // an action's completion can precede the timer arms its @@ -473,7 +499,7 @@ func TestScenarioDriverRejectsUnmodelledScripts(t *testing.T) { // settle, and the load rule makes its absence unscriptable. name: "an advance not preceded by an expectTimers rendezvous", script: `{"name":"x","description":"d","steps":[{"expectMint":{"respond":{"status":200,"body":{"ticket":"{{TICKET:1}}","expires_in":120,"url":"{{CABLE_URL:1}}"}}}},{"expectConnect":{"url":"{{CABLE_URL:1}}"}},{"advance":{"ms":30000}}],"finally":{"state":"closed"}}`, - wants: "must be the scenario's first step or immediately follow expectTimers", + wants: "expectState + expectTimers rendezvous", }, { // An EMPTY rendezvous set can never contain an arm of the @@ -483,9 +509,21 @@ func TestScenarioDriverRejectsUnmodelledScripts(t *testing.T) { // rendezvous were absent. A scenario with nothing yet armed // advances as its first step instead. name: "an advance behind an empty expectTimers rendezvous", - script: `{"name":"x","description":"d","steps":[{"expectTimers":{"exact":{}}},{"advance":{"ms":1}}],"finally":{"state":"closed"}}`, + script: `{"name":"x","description":"d","steps":[{"expectState":{"is":"backoff"}},{"expectTimers":{"exact":{}}},{"advance":{"ms":1}}],"finally":{"state":"closed"}}`, wants: "an empty rendezvous orders nothing", }, + { + // The set match alone can coincide with a TRANSIENT mid-surgery + // set (the welcome transition stops handshake-deadline and arms + // confirmation-deadline in separate clock acquisitions, so + // {staleness:1} exists in between). The state announcement bounds + // the surgery: expectState blocks until the transition announces, + // and every timer still unarmed at an announcement is exactly + // what the set match then waits for. + name: "an advance whose rendezvous lacks the state barrier", + script: `{"name":"x","description":"d","steps":[{"expectMint":{"respond":{"status":200,"body":{"ticket":"{{TICKET:1}}","expires_in":120,"url":"{{CABLE_URL:1}}"}}}},{"expectConnect":{"url":"{{CABLE_URL:1}}"}},{"expectTimers":{"exact":{"handshake-deadline":1}}},{"advance":{"ms":30000}}],"finally":{"state":"closed"}}`, + wants: "expectState + expectTimers rendezvous", + }, { name: "droppedCount disagreeing with droppedIds", script: `{"name":"x","description":"d","steps":[{"expectSignal":{"kind":"bufferOverflow","droppedIds":[1],"droppedCount":2}}],"finally":{"state":"closed"}}`, @@ -580,7 +618,8 @@ func TestScenarioDriverRejectsSchedulingDependentAdvance(t *testing.T) { script := `{"name":"x","description":"d","steps":[ {"expectMint":{"respond":{"status":200,"body":{"ticket":"{{TICKET:1}}","expires_in":120,"url":"{{CABLE_URL:1}}"}}}}, {"expectConnect":{"url":"{{CABLE_URL:1}}"}}, - {"expectTimers":{"exact":{"handshake-deadline":1}}}, + {"expectState":{"is":"awaiting_welcome"}}, + {"expectTimers":{"exact":{"handshake-deadline":1,"staleness":1}}}, {"advance":{"ms":30000}}], "finally":{"state":"backoff"}}` err := underShortWatchdog(func() error { return runScenarioBytes([]byte(script), "x.json") }) @@ -624,6 +663,7 @@ func TestScenarioDriverRejectsSchedulingDependentAdvance(t *testing.T) { {"expectSubscribe":{"channel":"EventsChannel"}}, {"fireTimer":{"kind":"confirmation-deadline"}}, {"expectClientClose":{}}, + {"expectState":{"is":"backoff"}}, {"expectTimers":{"exact":{"backoff":1}}}, {"advance":{"ms":1000}}, {"expectMint":{"respond":{"status":200,"body":{"ticket":"{{TICKET:2}}","expires_in":120,"url":"{{CABLE_URL:2}}"}}}}, From 9598aeaed49aa5d56ccfed3411c65eeff898b934 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Mon, 24 Aug 2026 22:25:43 -0700 Subject: [PATCH 35/55] Null is the third state, and the decode chain grew a link when it was coded MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot's presence saga round two and Codex's ceiling round four, both taken. * *int64 preserved explicit zero but read explicit JSON null exactly as omission: {"stalenessMs":null} silently defaulted while the schema's "type": "integer" rejects it, and null slipped the unmodeled backoff rejection too. The five config durations are now a three-state optionalMs — absent, null, value — via a value-type UnmarshalJSON (encoding/json calls it for null where it short-circuits a pointer's, which is why the pointer could not see the difference). Everything SUPPLIED is judged: a value is ranged, null is refused with the schema's own reasoning, and the unmodeled pair rejects any supplied state including null. The driver keeps enforcing the schema's judgments portably; this is the same charter that put the range check in Go. Red-proven: five explicit-null self-tests (three ranged durations, both unmodeled) all passed the previous driver silently. * The ceiling's count moved because the chain moved. Re-derived at this head, decodeMessageEvent no longer unmarshals the payload straight into a typed struct: the exact-spelling decode (the null-vs-absent frame-fields work) goes through a per-field map[string]json.RawMessage first, whose field copies sum to frame scale when one field dominates. A near-limit correlated message therefore exists as FOUR coexisting frame-scale representations — wire bytes in the dispatching caller's hands, parseFrame's envelope RawMessage copy, the per-field RawMessage map, and the decoded Event's strings — until the decode returns. The ceiling becomes (pump depth + 6 + capacity) × MAX_FRAME_BYTES, with the four-link enumeration IN the SPEC text so the count is checkable by reading, and the closure statement now records how the count last moved: not by a fifth review noticing a party, but by a representation being CODED into the chain — exactly the growth mode the rule names. The earlier "three" was correct for the chain as it stood when it was derived; the chain changed underneath it in a base merge, which is the strongest argument yet for keeping the enumeration next to the number. All four mirrored sites carry the figure. Gates: both eventfeed packages green, plain and -race; tracking-issues live checker green (all 6 issues open); doc-constants-check green under LC_ALL=C; gofmt clean. --- SPEC.md | 35 +++++++----- go/pkg/basecamp/eventfeed/buffer_test.go | 2 +- go/pkg/basecamp/eventfeed/catchup.go | 13 +++-- go/pkg/basecamp/eventfeed/catchup_test.go | 2 +- .../eventfeed/scenario_fixture_test.go | 57 ++++++++++++++----- .../eventfeed/scenario_harness_test.go | 12 ++-- .../eventfeed/scenario_selftest_test.go | 28 +++++++++ 7 files changed, 105 insertions(+), 44 deletions(-) diff --git a/SPEC.md b/SPEC.md index d0a868a4d..02dad98ff 100644 --- a/SPEC.md +++ b/SPEC.md @@ -3637,17 +3637,20 @@ Two dispatch clarifications, pinned: closes the count: a frame lives in the hand-off queue (≤ pump depth), in the live buffer (≤ `EVENT_FEED_LIVE_BUFFER_CAPACITY`), in the single deferral slot (≤ 1), or in the hands of one of the exactly two goroutines that touch frames. The pump's hand - holds one frame. The state machine holds up to THREE frame-sized allocations at - once, and three is not a discovered constant but the decode chain's REPRESENTATION - COUNT: a message frame exists as the wire bytes, as `parseFrame`'s `json.RawMessage` - payload copy, and as the decoded `Event`'s strings — each step's output allocated - while its input is still in hand, and an adversarial near-limit payload can put - frame-scale text in any of the three. The ceiling is - (pump depth + 5 + `EVENT_FEED_LIVE_BUFFER_CAPACITY`) × `EVENT_FEED_MAX_FRAME_BYTES` + holds one frame. The state machine holds up to FOUR frame-sized allocations at + once, and four is not a discovered constant but the decode chain's REPRESENTATION + COUNT, checkable by reading the chain: a near-limit correlated message exists as + (1) the wire bytes, still in the dispatching caller's hands; (2) `parseFrame`'s + envelope `json.RawMessage` payload copy; (3) `decodeMessageEvent`'s per-field + `map[string]json.RawMessage`, whose field copies sum to frame scale when one field + dominates; and (4) the decoded `Event`'s strings — each step's output allocated + while its inputs are still in hand, all four coexisting until the decode returns. + The ceiling is + (pump depth + 6 + `EVENT_FEED_LIVE_BUFFER_CAPACITY`) × `EVENT_FEED_MAX_FRAME_BYTES` (≈ 10 GiB at the defaults' extreme, reached only if every slot holds a maximum-size - frame) — even under a slow consumer. The **+ 5** is five frame-sized allocations the + frame) — even under a slow consumer. The **+ 6** is six frame-sized allocations the queue's depth does not count — the deferral slot, the pump's in-hand frame, and the - state machine's in-hand frame at its decode-chain weight of three — retained by + state machine's in-hand frame at its decode-chain weight of four — retained by different parties at the same time: - the **pump's own in-flight frame** — the pump is a single reader, so it may hold exactly one frame it has already READ and not yet handed off. One rather than an unbounded @@ -3656,11 +3659,13 @@ Two dispatch clarifications, pinned: receive that lets a blocked pump refill the queue, so while the scan still holds that frame — examining, admitting, or parking it — the queue is full again and the pump may already hold its next read. A single consumer, so one frame in hand, for the pump's - own reason — weighted at the decode chain's length of three: wire bytes, - `RawMessage` copy, decoded `Event`, each transient past its step and every one in - this holder's hands, never a further party's. A fourth term here requires CODING a - fourth representation into the chain, not noticing one — which is what closes the - count. + own reason — weighted at the decode chain's length of four: wire bytes, envelope + `RawMessage` copy, per-field `RawMessage` map, decoded `Event`, each transient + past its step and every one in this holder's hands, never a further party's. A + further term here requires CODING a further representation into the chain, not + noticing one — which is what closes the count, and is also how it last moved: an + earlier revision counted three, and the count became four when the exact-spelling + per-field decode was coded in, exactly the growth mode this rule names. - the **deferred socket outcome** — the single slot the in-flight-poll servicing and the drain's scan park one receive in. It is retained while the queue behind it refills, so it is concurrent with a full queue and with both in-hand frames, not an alternative to @@ -3669,7 +3674,7 @@ Two dispatch clarifications, pinned: The enumeration cannot grow by a further party being noticed: every frame-sized allocation is in one of the three counted structures or in the hands of the pump or the state machine, each holder counted at its worst-case weight — the pump at one - (it never parses), the state machine at the decode chain's three. The live buffer's + (it never parses), the state machine at the decode chain's four. The live buffer's own weight stays one per slot: a buffered `Event` retains only the chain's LAST representation — its strings are copies, since Go's decoder never aliases its input buffer, and `Event` carries no raw-bytes field — so nothing of the first two diff --git a/go/pkg/basecamp/eventfeed/buffer_test.go b/go/pkg/basecamp/eventfeed/buffer_test.go index b549e7577..e2c8f2c9b 100644 --- a/go/pkg/basecamp/eventfeed/buffer_test.go +++ b/go/pkg/basecamp/eventfeed/buffer_test.go @@ -7,7 +7,7 @@ import ( // TestLiveBufferAddClearsEvictedSlots pins the eviction half of the live // buffer's memory ceiling. SPEC.md §23 publishes the cable lane's worst case -// as (pump depth + 5 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × EVENT_FEED_MAX_FRAME_BYTES; +// as (pump depth + 6 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × EVENT_FEED_MAX_FRAME_BYTES; // a reslice alone removes the evicted event LOGICALLY while the slice that // results still points into the same backing array, whose prefix keeps that // event's strings reachable until a later reallocation. Under sustained diff --git a/go/pkg/basecamp/eventfeed/catchup.go b/go/pkg/basecamp/eventfeed/catchup.go index 23a0f15e9..5a4f52a7b 100644 --- a/go/pkg/basecamp/eventfeed/catchup.go +++ b/go/pkg/basecamp/eventfeed/catchup.go @@ -1117,16 +1117,17 @@ func (l *loop) probeFatal(at *attempt) (cycleOutcome, bool) { // // This is why the scan needs no queue and no share of pumpDepth: it retains // exactly what the single slot always retained. The published cable-lane -// retention bound — (pump depth + 5 + liveBufferCapacity) × MAX_FRAME_BYTES — is +// retention bound — (pump depth + 6 + liveBufferCapacity) × MAX_FRAME_BYTES — is // untouched, and so is the depth at which the pump blocks. The slot IS one of // that formula's frame-sized terms beyond the queue; the others are the frame // the pump has read and not yet handed off, and the frame the scan itself has // dequeued and not yet disposed of — the very receive that lets a blocked -// pump refill the queue behind it, counted at THREE, the decode chain's -// representation count: wire bytes, parseFrame's json.RawMessage payload -// copy, and the decoded Event's strings, each step's output allocated while -// its input is still in hand. All are retained WHILE the queue is full, which -// is why they are addends and not alternatives. +// pump refill the queue behind it, counted at FOUR, the decode chain's +// representation count: wire bytes, parseFrame's envelope json.RawMessage +// copy, decodeMessageEvent's per-field RawMessage map, and the decoded +// Event's strings, each step's output allocated while its inputs are still +// in hand. All are retained WHILE the queue is full, which is why they are +// addends and not alternatives. func (l *loop) deferForDrain(d *deferredFrame) { if l.deferred == nil { l.deferred = d diff --git a/go/pkg/basecamp/eventfeed/catchup_test.go b/go/pkg/basecamp/eventfeed/catchup_test.go index 997179986..9158868c7 100644 --- a/go/pkg/basecamp/eventfeed/catchup_test.go +++ b/go/pkg/basecamp/eventfeed/catchup_test.go @@ -1396,7 +1396,7 @@ func TestDrainScanAdmissionIsNotStranded(t *testing.T) { // TestDrainHoldsNoMoreThanTheLiveBufferCapacity: the live buffer's capacity // is a bound on events HELD AT ONCE — SPEC §23 sizes the cable lane's whole -// memory ceiling off it, "(pump depth + 5 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × +// memory ceiling off it, "(pump depth + 6 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × // EVENT_FEED_MAX_FRAME_BYTES" — so a drain must not be able to hold a batch // outside the buffer while the buffer refills to capacity behind it. // diff --git a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go index ee74d7fe6..369aa33ea 100644 --- a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go @@ -46,21 +46,44 @@ type scenarioConfig struct { ExcludePerformers []int64 `json:"exclude_performers"` ActorTypes []string `json:"actorTypes"` Position string `json:"position"` - // The five durations are POINTERS because presence is meaning: the - // schema says minimum 1, and a plain int64 read an explicit zero as - // "absent, use the default" — accepting a value the schema rejects and - // silently substituting another. - ConfirmationDeadlineMs *int64 `json:"confirmationDeadlineMs"` - RepairPollBaseMs *int64 `json:"repairPollBaseMs"` - BackoffBaseMs *int64 `json:"backoffBaseMs"` - BackoffCapMs *int64 `json:"backoffCapMs"` - StalenessMs *int64 `json:"stalenessMs"` + // The five durations are THREE-STATE because presence is meaning twice + // over: a plain int64 read an explicit zero as "absent, use the + // default", and a pointer read an explicit JSON null the same way — + // each accepting a value the schema rejects ("minimum": 1 for zero, + // "type": "integer" for null) and silently substituting another. The + // driver enforces the schema's judgments portably, so all three states + // the wire distinguishes are preserved: absent, null, and value. + ConfirmationDeadlineMs optionalMs `json:"confirmationDeadlineMs"` + RepairPollBaseMs optionalMs `json:"repairPollBaseMs"` + BackoffBaseMs optionalMs `json:"backoffBaseMs"` + BackoffCapMs optionalMs `json:"backoffCapMs"` + StalenessMs optionalMs `json:"stalenessMs"` LiveBufferCapacity int `json:"liveBufferCapacity"` DedupeCapacity int `json:"dedupeCapacity"` SignalDisposition map[string]string `json:"signalDisposition"` CheckpointStore *storeScript `json:"checkpointStore"` } +// optionalMs is one config duration in the three JSON states the schema +// distinguishes: absent (the zero optionalMs — use the default), JSON null +// (set, null — rejected, "type": "integer" refuses it), and a value (set, +// ranged). encoding/json calls a value type's UnmarshalJSON for null where it +// short-circuits a pointer's, which is exactly why this is not a *int64. +type optionalMs struct { + set bool + null bool + v int64 +} + +func (o *optionalMs) UnmarshalJSON(data []byte) error { + o.set = true + if string(data) == "null" { + o.null = true + return nil + } + return json.Unmarshal(data, &o.v) +} + // storeScript is the schema's scripted CheckpointStore. type storeScript struct { Load string `json:"load"` @@ -568,23 +591,27 @@ type ( // --- validation ---------------------------------------------------------- func validateConfig(cfg scenarioConfig) error { - // A nil pointer is absence and means "default"; every SUPPLIED value is - // ranged, explicit zero included. + // Absence means "default"; everything SUPPLIED is judged — a value is + // ranged (explicit zero included) and null is refused outright. for _, f := range []struct { name string - v *int64 + o optionalMs }{ {"confirmationDeadlineMs", cfg.ConfirmationDeadlineMs}, {"repairPollBaseMs", cfg.RepairPollBaseMs}, {"stalenessMs", cfg.StalenessMs}, } { - if f.v != nil { - if err := checkScenarioMs(f.name, *f.v, 1); err != nil { + switch { + case !f.o.set: + case f.o.null: + return fmt.Errorf("%s supplied as JSON null: the schema's type is integer and null is not one — omit the key for the default", f.name) + default: + if err := checkScenarioMs(f.name, f.o.v, 1); err != nil { return err } } } - if cfg.BackoffBaseMs != nil || cfg.BackoffCapMs != nil { + if cfg.BackoffBaseMs.set || cfg.BackoffCapMs.set { return fmt.Errorf("backoffBaseMs/backoffCapMs are not modeled: SPEC §23 pins the Go connector's full-jitter base and cap as constants, with no construction option to override") } for kind, disposition := range cfg.SignalDisposition { diff --git a/go/pkg/basecamp/eventfeed/scenario_harness_test.go b/go/pkg/basecamp/eventfeed/scenario_harness_test.go index 1a912f1fe..d7c4809f3 100644 --- a/go/pkg/basecamp/eventfeed/scenario_harness_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_harness_test.go @@ -708,11 +708,11 @@ func (h *scenarioHarness) newConnector(cfg scenarioConfig) (*eventfeed.Connector }, }), } - if cfg.ConfirmationDeadlineMs != nil { - opts = append(opts, eventfeed.WithConfirmationDeadline(millis(*cfg.ConfirmationDeadlineMs))) + if cfg.ConfirmationDeadlineMs.set { + opts = append(opts, eventfeed.WithConfirmationDeadline(millis(cfg.ConfirmationDeadlineMs.v))) } - if cfg.RepairPollBaseMs != nil { - opts = append(opts, eventfeed.WithRepairInterval(millis(*cfg.RepairPollBaseMs))) + if cfg.RepairPollBaseMs.set { + opts = append(opts, eventfeed.WithRepairInterval(millis(cfg.RepairPollBaseMs.v))) } if cfg.LiveBufferCapacity > 0 { opts = append(opts, eventfeed.WithLiveBufferCapacity(cfg.LiveBufferCapacity)) @@ -728,8 +728,8 @@ func (h *scenarioHarness) newConnector(cfg scenarioConfig) (*eventfeed.Connector if err != nil { return nil, err } - if cfg.StalenessMs != nil { - conn.SetStaleAfter(millis(*cfg.StalenessMs)) + if cfg.StalenessMs.set { + conn.SetStaleAfter(millis(cfg.StalenessMs.v)) } conn.OnStateChanged(h.recordState) conn.OnBufferOccupancy(h.recordOccupancy) diff --git a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go index 784d136d6..3bfd9246c 100644 --- a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go @@ -488,6 +488,34 @@ func TestScenarioDriverRejectsUnmodelledScripts(t *testing.T) { script: `{"name":"x","description":"d","config":{"backoffBaseMs":0},"steps":[{"advance":{"ms":1}}],"finally":{"state":"closed"}}`, wants: "not modeled", }, + { + // Explicit null is the presence saga's second act: *int64 read + // {"stalenessMs":null} and omission both as nil, silently + // defaulting a value the schema's "type": "integer" rejects. + name: "config stalenessMs explicit null", + script: `{"name":"x","description":"d","config":{"stalenessMs":null},"steps":[{"advance":{"ms":1}}],"finally":{"state":"closed"}}`, + wants: "supplied as JSON null", + }, + { + name: "config confirmationDeadlineMs explicit null", + script: `{"name":"x","description":"d","config":{"confirmationDeadlineMs":null},"steps":[{"advance":{"ms":1}}],"finally":{"state":"closed"}}`, + wants: "supplied as JSON null", + }, + { + name: "config repairPollBaseMs explicit null", + script: `{"name":"x","description":"d","config":{"repairPollBaseMs":null},"steps":[{"advance":{"ms":1}}],"finally":{"state":"closed"}}`, + wants: "supplied as JSON null", + }, + { + name: "config backoffBaseMs explicit null", + script: `{"name":"x","description":"d","config":{"backoffBaseMs":null},"steps":[{"advance":{"ms":1}}],"finally":{"state":"closed"}}`, + wants: "not modeled", + }, + { + name: "config backoffCapMs explicit null", + script: `{"name":"x","description":"d","config":{"backoffCapMs":null},"steps":[{"advance":{"ms":1}}],"finally":{"state":"closed"}}`, + wants: "not modeled", + }, { // An advance is deterministic only from a scripted rendezvous: // an action's completion can precede the timer arms its From 0fa21f3d028bc88ab02a2634ab34d4ab5bb2ae49 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Mon, 24 Aug 2026 22:45:11 -0700 Subject: [PATCH 36/55] A registered file must be sweepable, and the rearm residue gets its name MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Codex's settle round four is declined with the mechanism examined and the contract strengthened; Copilot's ghost-file finding is taken. * Ghost rows. A mentions: 0 row guards a non-canonical promise by NAME alone, so a rename, delete, or typo made swept.fetch empty, 0 == 0 passed, and the row silently guarded nothing — green until the issue closed and CI blocked on a phantom promise. Every registered path must now be in the sweep's view (present, text, not excluded), failing closed with the row named. Red-proven: the ghost-file row passed the previous checker. The check also flushed out four self-test cases registering rows for files absent from their corpora — including the deleted-mention case, which now stages its file present with the sentence gone, truer to what it tests. All 42 cases pass. * The rearm rendezvous, examined rather than re-patched. Codex asks the barrier to require a NEWLY observed transition (a generation tied to the preceding action). The mechanism was worked through and it reintroduces the disease one level up: whether a state announcement lands before or after the PRECEDING expectation completes is itself a scheduling race — fixture 05's streaming announcement races its expectCheckpoint's completion — so generation-newness would make the suite's one legitimate advance flaky. Reachability was checked honestly: no committed fixture advances behind an announcement-free rearm. Fixture 05's serves are protocol frames whose receipt-side staleness resets happen-before the announcements its rendezvous observes, and its config pins staleness and repair-poll at ~11.5 virtual days against a 121-second window — the schema's own stalenessMs guidance — so old and new deadlines alike sit far outside the advance. What changes is the contract's specificity: the residue paragraph in README and SPEC now names the concrete case (a served live frame's pump-side staleness rearm), the sanctioned mitigation (override stalenessMs large, as fixture 05 does), and fireTimer for a script that wants the firing itself — a stated limit with a worked example instead of an abstract sentence. Gates: tracking-issues self-test 42/42; live checker green (all 26 registered files visible to the real sweep, all 6 issues open); doc-constants-check green under LC_ALL=C. --- SPEC.md | 7 +++++-- conformance/event-feed/README.md | 10 ++++++++-- scripts/check-known-defect-issues-open | 14 +++++++++++++ scripts/test-check-known-defect-issues-open | 22 +++++++++++++++++++++ 4 files changed, 49 insertions(+), 4 deletions(-) diff --git a/SPEC.md b/SPEC.md index 02dad98ff..49a236eec 100644 --- a/SPEC.md +++ b/SPEC.md @@ -4168,8 +4168,11 @@ acquisitions), and an announcement can precede a tail arm. Together they do — announcement bounds the surgery, and any timer still unarmed at the announcement is exactly what the exact-set match then waits for, both blocking under the watchdog so wrong authorship fails loudly. A transition that announces no state change, or only -rearms a timer of the same kind and count, is invisible to this rendezvous — a script -that would advance behind one uses `fireTimer`. `conformance/event-feed/schema.json`'s +rearms a timer of the same kind and count, is invisible to this rendezvous — a served +live frame's pump-side `staleness` rearm is the concrete case. Such a script overrides +`stalenessMs` large so no deadline, old or new, sits inside a window it advances (the +schema's own guidance, and what the suite's one advance does), or uses `fireTimer` for +the firing it actually wants. `conformance/event-feed/schema.json`'s `$defs.advance` states both, and the driver obligation is enforced there. Teardown discipline: disposing a connection attempt — deadline lapse, staleness, socket diff --git a/conformance/event-feed/README.md b/conformance/event-feed/README.md index 7fedbf04d..7b172b25b 100644 --- a/conformance/event-feed/README.md +++ b/conformance/event-feed/README.md @@ -201,8 +201,14 @@ the announcement is exactly what the following exact-set match waits for. Both steps block under the scenario watchdog, so a wrongly authored state or set fails loudly instead of diverging silently. What stays outside this rendezvous is a transition that announces no state change, or only rearms a -timer of the same kind and count — invisible to both barriers; a script that -would advance behind one writes `fireTimer` instead. +timer of the same kind and count — invisible to both barriers. The concrete +case is a live frame served in a socket-open state: its receipt rearms +`staleness` pump-side with no announcement, so an advance behind it would race +the rearm's deadline shift. A script that must advance across served frames +takes the schema's own `stalenessMs` guidance — override it large, so both the +old and the new deadline sit outside any window the script advances (fixture +05 does exactly this, at ~11.5 virtual days against a 121-second window) — and +a script that wants the staleness firing itself writes `fireTimer`. ## Contract notes the fixtures encode (SDK-owned, final) diff --git a/scripts/check-known-defect-issues-open b/scripts/check-known-defect-issues-open index 55d56021c..60f2a7730 100755 --- a/scripts/check-known-defect-issues-open +++ b/scripts/check-known-defect-issues-open @@ -56,6 +56,7 @@ # Exercised offline by scripts/test-check-known-defect-issues-open. require 'json' +require 'set' require 'yaml' ROOT = File.expand_path('..', __dir__) @@ -279,6 +280,7 @@ end sweep_root = ENV['PROSE_SWEEP_ROOT'] || ROOT registered_mentions = rows.to_h { |e| [ [ e['file'], e['issue'] ], e['mentions'] ] } swept = Hash.new { |h, k| h[k] = [] } +scanned = Set.new sweep_files(sweep_root).each do |rel| next if rel == 'spec/tracking-issues.yml' || rel == 'scripts/test-check-known-defect-issues-open' || @@ -290,6 +292,7 @@ sweep_files(sweep_root).each do |rel| content = File.binread(path) next if content.index("\0".b) + scanned << rel content = content.force_encoding(Encoding::UTF_8).scrub content.to_enum(:scan, SWEEP_MENTION).each do m = Regexp.last_match @@ -303,6 +306,17 @@ swept.each do |(rel, issue), lines| lines.each { |line| violations << "#{rel}:#{line} — a canonical \"tracked in ##{issue}\" claim, and [#{rel}, ##{issue}] is not registered" } end end +# A mentions: 0 row guards a non-canonical promise by NAME alone, so a rename, +# delete, or typo makes swept.fetch empty, 0 == 0 passes, and the row silently +# guards nothing — green until the issue closes and CI blocks on a phantom. +# Every registered path must be in the sweep's view (present, text, and not +# excluded from the scan). +registered_mentions.each_key do |(rel, _issue)| + next if scanned.include?(rel) + + violations << "#{rel} is registered but the sweep cannot see it (missing, renamed, binary, or excluded) — " \ + 'a row for a file the sweep cannot read guards nothing; fix the path or remove the row' +end # Both directions, so a new mention cannot ride an existing row and a deleted # sentence cannot leave its count claiming more than the file holds. registered_mentions.each do |(rel, issue), expected| diff --git a/scripts/test-check-known-defect-issues-open b/scripts/test-check-known-defect-issues-open index 0d783e751..f1694c7da 100755 --- a/scripts/test-check-known-defect-issues-open +++ b/scripts/test-check-known-defect-issues-open @@ -336,6 +336,7 @@ expect_fail(failures, "9. missing gh fails closed", out, status, out, status, calls = run_checker( allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, tracking: { "prose_tracking_issues" => [ { "issue" => 303, "file" => "README.md", "site" => "README.md — a gap", "mentions" => 0 } ] }, + sweep: { "README.md" => "a gap, promised outside the canonical grammar\n" }, states: "303=open" ) expect_pass(failures, "N. prose registry entries are collected", out, status, @@ -353,6 +354,7 @@ end out, status, _ = run_checker( allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, tracking: { "prose_tracking_issues" => [ { "issue" => 404, "file" => "SPEC.md", "site" => "SPEC.md — a bound", "mentions" => 0 } ] }, + sweep: { "SPEC.md" => "a bound, promised outside the canonical grammar\n" }, states: "404=closed" ) expect_fail(failures, "N+1. a CLOSED prose tracking issue fails", out, status, @@ -366,6 +368,7 @@ expect_fail(failures, "N+1. a CLOSED prose tracking issue fails", out, status, out, status, _ = run_checker( allowlist: { "sdk_routes_known_defective" => [ defective(101) ], "bc3_routes_not_modeled" => [] }, tracking: { "prose_tracking_issues" => [ { "issue" => 303, "file" => "README.md", "site" => "README.md — a gap", "mentions" => 0 } ] }, + sweep: { "README.md" => "a gap, promised outside the canonical grammar\n" }, states: "101=open,303=closed" ) expect_fail(failures, "N+2. a closed registry entry fails even with a green allowlist", out, status, @@ -515,6 +518,24 @@ out, status, _ = run_checker( expect_fail(failures, "N+6l. a multi-document allowlist fails instead of dropping entries", out, status, "YAML documents") +# --- N+18. A row whose file the sweep cannot see fails --------------------------- +# +# A mentions: 0 row guards a non-canonical promise by NAME alone, so when its +# file is renamed, deleted, or mistyped, swept.fetch returns empty, 0 == 0, +# and the row silently stops guarding anything — green until the issue closes +# and CI blocks on a phantom promise. Every registered path must be in the +# sweep's view. + +out, status, _ = run_checker( + allowlist: { "sdk_routes_known_defective" => [], "bc3_routes_not_modeled" => [] }, + tracking: { "prose_tracking_issues" => [ + { "issue" => 303, "file" => "ghost.md", "site" => "ghost.md — a promise whose file went away", "mentions" => 0 } + ] }, + states: "303=open" +) +expect_fail(failures, "N+18. a row for a file the sweep cannot see fails", out, status, + "cannot see") + # --- N+7. A registry file without the key fails ----------------------------------- # # The missing-FILE case (N+3) fails closed; a file whose key is misspelled or @@ -653,6 +674,7 @@ out, status, _ = run_checker( tracking: { "prose_tracking_issues" => [ { "issue" => 303, "file" => "docs/note.md", "site" => "docs/note.md — a gap", "mentions" => 1 } ] }, + sweep: { "docs/note.md" => "the sentence went away in an edit\n" }, states: "303=open" ) expect_fail(failures, "N+15. a deleted mention fails until the count moves", out, status, From d58d38bde9077f96ad9b2e11a342176645c82d55 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Mon, 24 Aug 2026 23:00:53 -0700 Subject: [PATCH 37/55] Give the envelope its three states, and split the ceiling into what it was MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot's envelope presence gap and ceiling round five, both taken — the second by reframing the instrument rather than writing a fifth number. * assertDelayMs had every presence gap the config durations just closed: the pointer read "assertDelayMs": null as omission, silently dropping the assertion the script wrote, and inside a non-null envelope an absent or null min/max decoded to int64(0) — in range, silently a different envelope than the one authored, though the schema requires both members. The envelope is now three-state (optionalEnvelope) with three-state members (optionalMs): null refused at both levels with the schema's reasoning, an absent member a fixture error, values ranged, min ≤ max preserved — and the inner decode re-establishes DisallowUnknownFields, which the outer strict decoder cannot reach through a custom unmarshaler. Red-proven: three self-tests (null envelope, missing min, null member) all passed the previous driver silently. * The ceiling treadmill's fifth round named map bucket storage and copied keys — overhead proportional to MEMBER COUNT, which no per-representation count can bound: a frame of many tiny members inflates the maps past any chain-length figure. That is the signal the number was conflating two different questions, so the accounting now splits. RETAINED storage — what a consumer sizes against: frames and events held across blocking points, by holder — is (pump depth + 3 + capacity) × MAX_FRAME_BYTES, the enumeration that was stable before decode weights entered it. TRANSIENT decode-time allocation rides on top, per frame, in the state machine's hands alone: the representation chain plus decoder overhead proportional to member count, bounded by a small implementation-topology multiple that the SPEC deliberately does NOT publish — it moved when the exact-spelling decode was coded in, and pinning it would turn every decoder refactor into a spec change. What is contract: transients exist one frame at a time, between dequeue and decode return, so peak memory is the retained formula plus ONE frame's transient — never a per-slot or per-queue multiplier. All four mirrored sites carry the split. Gates: both eventfeed packages green, plain and -race; tracking-issues self-test 42/42; live checker green (all 6 issues open); doc-constants-check green under LC_ALL=C; gofmt clean. --- SPEC.md | 69 +++++++++---------- go/pkg/basecamp/eventfeed/buffer_test.go | 3 +- go/pkg/basecamp/eventfeed/catchup.go | 21 +++--- go/pkg/basecamp/eventfeed/catchup_test.go | 2 +- .../eventfeed/scenario_conformance_test.go | 4 +- .../eventfeed/scenario_fixture_test.go | 60 +++++++++++++--- .../eventfeed/scenario_selftest_test.go | 21 ++++++ 7 files changed, 119 insertions(+), 61 deletions(-) diff --git a/SPEC.md b/SPEC.md index 49a236eec..7cfbef595 100644 --- a/SPEC.md +++ b/SPEC.md @@ -3631,27 +3631,19 @@ Two dispatch clarifications, pinned: transport-authored, unbounded by the seam contract. The built-in transport's errors are bounded by construction (fixed shapes whose renderings are configured text or placeholders, never server bytes); a custom transport's error is its author's to - bound. In the Go reference, worst-case cable-lane retention is bounded - multiplicatively — every retained item is itself bounded by - `EVENT_FEED_MAX_FRAME_BYTES`, and retention is an enumeration by HOLDER, which is what - closes the count: a frame lives in the hand-off queue (≤ pump depth), in the live - buffer (≤ `EVENT_FEED_LIVE_BUFFER_CAPACITY`), in the single deferral slot (≤ 1), or in - the hands of one of the exactly two goroutines that touch frames. The pump's hand - holds one frame. The state machine holds up to FOUR frame-sized allocations at - once, and four is not a discovered constant but the decode chain's REPRESENTATION - COUNT, checkable by reading the chain: a near-limit correlated message exists as - (1) the wire bytes, still in the dispatching caller's hands; (2) `parseFrame`'s - envelope `json.RawMessage` payload copy; (3) `decodeMessageEvent`'s per-field - `map[string]json.RawMessage`, whose field copies sum to frame scale when one field - dominates; and (4) the decoded `Event`'s strings — each step's output allocated - while its inputs are still in hand, all four coexisting until the decode returns. - The ceiling is - (pump depth + 6 + `EVENT_FEED_LIVE_BUFFER_CAPACITY`) × `EVENT_FEED_MAX_FRAME_BYTES` + bound. In the Go reference the accounting SPLITS, because two kinds of memory answer + different questions. + + **RETAINED storage is what a consumer sizes against** — raw frames and buffered + events held across blocking points, an enumeration by HOLDER, which is what closes + the count: the hand-off queue (≤ pump depth), the live buffer + (≤ `EVENT_FEED_LIVE_BUFFER_CAPACITY`), the single deferral slot (≤ 1), and one + in-hand frame for each of the exactly two goroutines that touch frames. The retained + worst case is + (pump depth + 3 + `EVENT_FEED_LIVE_BUFFER_CAPACITY`) × `EVENT_FEED_MAX_FRAME_BYTES` (≈ 10 GiB at the defaults' extreme, reached only if every slot holds a maximum-size - frame) — even under a slow consumer. The **+ 6** is six frame-sized allocations the - queue's depth does not count — the deferral slot, the pump's in-hand frame, and the - state machine's in-hand frame at its decode-chain weight of four — retained by - different parties at the same time: + frame) — even under a slow consumer. The **+ 3** is three frame-sized retentions the + queue's depth does not count, held by different parties at the same time: - the **pump's own in-flight frame** — the pump is a single reader, so it may hold exactly one frame it has already READ and not yet handed off. One rather than an unbounded number for that reason: one reader holds at most one frame outside the queue. @@ -3659,26 +3651,33 @@ Two dispatch clarifications, pinned: receive that lets a blocked pump refill the queue, so while the scan still holds that frame — examining, admitting, or parking it — the queue is full again and the pump may already hold its next read. A single consumer, so one frame in hand, for the pump's - own reason — weighted at the decode chain's length of four: wire bytes, envelope - `RawMessage` copy, per-field `RawMessage` map, decoded `Event`, each transient - past its step and every one in this holder's hands, never a further party's. A - further term here requires CODING a further representation into the chain, not - noticing one — which is what closes the count, and is also how it last moved: an - earlier revision counted three, and the count became four when the exact-spelling - per-field decode was coded in, exactly the growth mode this rule names. + own reason. - the **deferred socket outcome** — the single slot the in-flight-poll servicing and the drain's scan park one receive in. It is retained while the queue behind it refills, so it is concurrent with a full queue and with both in-hand frames, not an alternative to any of them. - The enumeration cannot grow by a further party being noticed: every frame-sized - allocation is in one of the three counted structures or in the hands of the pump or - the state machine, each holder counted at its worst-case weight — the pump at one - (it never parses), the state machine at the decode chain's four. The live buffer's - own weight stays one per slot: a buffered `Event` retains only the chain's LAST - representation — its strings are copies, since Go's decoder never aliases its input - buffer, and `Event` carries no raw-bytes field — so nothing of the first two - representations survives admission. + **TRANSIENT decode-time allocation rides on top, per frame, in the state machine's + hands alone** (the pump never parses), bounded by a small implementation-topology + multiple of the frame being decoded rather than by a published constant. The multiple + covers the decode chain's representations — the wire bytes, `parseFrame`'s envelope + `json.RawMessage` copy, `decodeMessageEvent`'s per-field + `map[string]json.RawMessage`, the decoded `Event`'s strings — plus decoder overhead + proportional to member count: map bucket storage and copied keys, which a frame of + many tiny members inflates past any per-representation count. That is exactly why + the multiplier is NOT published: it is implementation topology, not contract — it + moved when the exact-spelling per-field decode was coded in, and pinning a number + would turn every decoder refactor into a spec change. What IS contract: transients + exist only between a frame's dequeue and its decode returning, one frame at a time, + so **peak memory is the retained formula plus ONE frame's transient allocation** — + never a per-slot or per-queue multiplier. + + The retained enumeration cannot grow by a further party being noticed: every + retained frame is in one of the three counted structures or in the hands of the pump + or the state machine. The live buffer's weight is one per slot: a buffered `Event` + retains only the chain's LAST representation — its strings are copies, since Go's + decoder never aliases its input buffer, and `Event` carries no raw-bytes field — so + no transient survives admission. The formula is the cable lane's retention, and only that — every counted item is a raw socket frame or a buffered live event. The poll lane sits outside it on purpose: diff --git a/go/pkg/basecamp/eventfeed/buffer_test.go b/go/pkg/basecamp/eventfeed/buffer_test.go index e2c8f2c9b..25124bea8 100644 --- a/go/pkg/basecamp/eventfeed/buffer_test.go +++ b/go/pkg/basecamp/eventfeed/buffer_test.go @@ -7,7 +7,8 @@ import ( // TestLiveBufferAddClearsEvictedSlots pins the eviction half of the live // buffer's memory ceiling. SPEC.md §23 publishes the cable lane's worst case -// as (pump depth + 6 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × EVENT_FEED_MAX_FRAME_BYTES; +// as (pump depth + 3 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × EVENT_FEED_MAX_FRAME_BYTES +// retained, plus one frame's transient decode allocation; // a reslice alone removes the evicted event LOGICALLY while the slice that // results still points into the same backing array, whose prefix keeps that // event's strings reachable until a later reallocation. Under sustained diff --git a/go/pkg/basecamp/eventfeed/catchup.go b/go/pkg/basecamp/eventfeed/catchup.go index 5a4f52a7b..2fe341648 100644 --- a/go/pkg/basecamp/eventfeed/catchup.go +++ b/go/pkg/basecamp/eventfeed/catchup.go @@ -1116,18 +1116,17 @@ func (l *loop) probeFatal(at *attempt) (cycleOutcome, bool) { // is actually reported. // // This is why the scan needs no queue and no share of pumpDepth: it retains -// exactly what the single slot always retained. The published cable-lane -// retention bound — (pump depth + 6 + liveBufferCapacity) × MAX_FRAME_BYTES — is -// untouched, and so is the depth at which the pump blocks. The slot IS one of -// that formula's frame-sized terms beyond the queue; the others are the frame -// the pump has read and not yet handed off, and the frame the scan itself has +// exactly what the single slot always retained. The published RETAINED bound +// — (pump depth + 3 + liveBufferCapacity) × MAX_FRAME_BYTES — is untouched, +// and so is the depth at which the pump blocks. The slot IS one of that +// formula's frame-sized terms beyond the queue; the others are the frame the +// pump has read and not yet handed off, and the frame the scan itself has // dequeued and not yet disposed of — the very receive that lets a blocked -// pump refill the queue behind it, counted at FOUR, the decode chain's -// representation count: wire bytes, parseFrame's envelope json.RawMessage -// copy, decodeMessageEvent's per-field RawMessage map, and the decoded -// Event's strings, each step's output allocated while its inputs are still -// in hand. All are retained WHILE the queue is full, which is why they are -// addends and not alternatives. +// pump refill the queue behind it. All are retained WHILE the queue is full, +// which is why they are addends and not alternatives. Decode-time TRANSIENTS +// (the representation chain plus map/key overhead) ride per frame on top and +// are deliberately not in the formula — SPEC §23 splits the accounting: +// peak = retained + one frame's transient. func (l *loop) deferForDrain(d *deferredFrame) { if l.deferred == nil { l.deferred = d diff --git a/go/pkg/basecamp/eventfeed/catchup_test.go b/go/pkg/basecamp/eventfeed/catchup_test.go index 9158868c7..f46766687 100644 --- a/go/pkg/basecamp/eventfeed/catchup_test.go +++ b/go/pkg/basecamp/eventfeed/catchup_test.go @@ -1396,7 +1396,7 @@ func TestDrainScanAdmissionIsNotStranded(t *testing.T) { // TestDrainHoldsNoMoreThanTheLiveBufferCapacity: the live buffer's capacity // is a bound on events HELD AT ONCE — SPEC §23 sizes the cable lane's whole -// memory ceiling off it, "(pump depth + 6 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × +// RETAINED ceiling off it, "(pump depth + 3 + EVENT_FEED_LIVE_BUFFER_CAPACITY) × // EVENT_FEED_MAX_FRAME_BYTES" — so a drain must not be able to hold a batch // outside the buffer while the buffer refills to capacity behind it. // diff --git a/go/pkg/basecamp/eventfeed/scenario_conformance_test.go b/go/pkg/basecamp/eventfeed/scenario_conformance_test.go index 0968caefe..af3308c7f 100644 --- a/go/pkg/basecamp/eventfeed/scenario_conformance_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_conformance_test.go @@ -654,10 +654,10 @@ func (d *driver) fireTimer(step *fireTimerStep) error { if !ok { return fmt.Errorf("no %s timer is outstanding: outstanding %v", step.Kind, d.h.clock.Outstanding()) } - if step.AssertDelayMs == nil { + if !step.AssertDelayMs.set { return nil } - low, high := millis(step.AssertDelayMs.Min), millis(step.AssertDelayMs.Max) + low, high := millis(step.AssertDelayMs.env.Min.v), millis(step.AssertDelayMs.env.Max.v) if delay < low || delay > high { return fmt.Errorf("the %s timer was armed for %s, want [%s, %s]", step.Kind, delay, low, high) } diff --git a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go index 369aa33ea..1bfa034f6 100644 --- a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go @@ -226,13 +226,39 @@ type advanceStep struct { } type fireTimerStep struct { - Kind string `json:"kind"` - AssertDelayMs *delayEnvelope `json:"assertDelayMs"` + Kind string `json:"kind"` + AssertDelayMs optionalEnvelope `json:"assertDelayMs"` } +// delayEnvelope's members are three-state for the same reason the config +// durations are: min and max are schema-REQUIRED integers, and a plain int64 +// read an absent or null member as 0 — inside the allowed range, silently +// converting the authored envelope into a different one. type delayEnvelope struct { - Min int64 `json:"min"` - Max int64 `json:"max"` + Min optionalMs `json:"min"` + Max optionalMs `json:"max"` +} + +// optionalEnvelope is assertDelayMs in the three JSON states: absent (no +// assertion — the zero value), null (set, null — refused, the schema's type +// is object), and a value (set, decoded strictly: the outer decoder's +// DisallowUnknownFields does not reach inside a custom unmarshaler, so the +// strictness is re-established here). +type optionalEnvelope struct { + set bool + null bool + env delayEnvelope +} + +func (o *optionalEnvelope) UnmarshalJSON(data []byte) error { + o.set = true + if string(data) == "null" { + o.null = true + return nil + } + dec := json.NewDecoder(bytes.NewReader(data)) + dec.DisallowUnknownFields() + return dec.Decode(&o.env) } type expectCheckpointStep struct { @@ -487,15 +513,27 @@ func decodeDirective(kind string, body json.RawMessage) (any, error) { if err := decodeStrict(body, step); err != nil { return nil, err } - if step.AssertDelayMs != nil { - if step.AssertDelayMs.Min > step.AssertDelayMs.Max { - return nil, fmt.Errorf("assertDelayMs min %d exceeds max %d", step.AssertDelayMs.Min, step.AssertDelayMs.Max) + if step.AssertDelayMs.set { + if step.AssertDelayMs.null { + return nil, fmt.Errorf("assertDelayMs supplied as JSON null: the schema's type is object and null is not one — omit the key to fire without a delay assertion") } - if err := checkScenarioMs("assertDelayMs min", step.AssertDelayMs.Min, 0); err != nil { - return nil, err + env := step.AssertDelayMs.env + for _, m := range []struct { + name string + o optionalMs + }{{"min", env.Min}, {"max", env.Max}} { + if !m.o.set { + return nil, fmt.Errorf("assertDelayMs needs both min and max — the schema requires them, and an absent %s is a different envelope than the one authored", m.name) + } + if m.o.null { + return nil, fmt.Errorf("assertDelayMs %s supplied as JSON null: the schema's type is integer and null is not one", m.name) + } + if err := checkScenarioMs("assertDelayMs "+m.name, m.o.v, 0); err != nil { + return nil, err + } } - if err := checkScenarioMs("assertDelayMs max", step.AssertDelayMs.Max, 0); err != nil { - return nil, err + if env.Min.v > env.Max.v { + return nil, fmt.Errorf("assertDelayMs min %d exceeds max %d", env.Min.v, env.Max.v) } } return step, validateTimerKind(step.Kind) diff --git a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go index 3bfd9246c..e0a99289b 100644 --- a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go @@ -516,6 +516,27 @@ func TestScenarioDriverRejectsUnmodelledScripts(t *testing.T) { script: `{"name":"x","description":"d","config":{"backoffCapMs":null},"steps":[{"advance":{"ms":1}}],"finally":{"state":"closed"}}`, wants: "not modeled", }, + { + // The envelope has the same three states the config durations do: + // a pointer read "assertDelayMs": null as omission, skipping the + // assertion the script wrote. + name: "fireTimer assertDelayMs explicit null", + script: `{"name":"x","description":"d","steps":[{"fireTimer":{"kind":"backoff","assertDelayMs":null}}],"finally":{"state":"closed"}}`, + wants: "supplied as JSON null", + }, + { + // min and max are schema-required: an absent member decoded to + // int64(0), inside the allowed range, silently converting the + // authored envelope into a different one. + name: "fireTimer assertDelayMs missing min", + script: `{"name":"x","description":"d","steps":[{"fireTimer":{"kind":"backoff","assertDelayMs":{"max":10}}}],"finally":{"state":"closed"}}`, + wants: "needs both min and max", + }, + { + name: "fireTimer assertDelayMs null member", + script: `{"name":"x","description":"d","steps":[{"fireTimer":{"kind":"backoff","assertDelayMs":{"min":null,"max":10}}}],"finally":{"state":"closed"}}`, + wants: "supplied as JSON null", + }, { // An advance is deterministic only from a scripted rendezvous: // an action's completion can precede the timer arms its From 001ce96111e5f3fcf32e793ffe045c6d0f95be99 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Mon, 24 Aug 2026 23:14:29 -0700 Subject: [PATCH 38/55] Judge ms numbers by mathematical value, as the schema does MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot's number-model finding, taken red-first. draft 2020-12's "integer" is any number whose MATHEMATICAL value is integral — 1000.0 and 1e3 are integer instances a schema-valid fixture may carry — while json.Unmarshal into int64 judges the spelling, so those fixtures failed only in the Go driver, against the loader's portable-enforcement charter. This is the repo's float-spelled-int class again, the one FlexInt absorbs on the rich-text lane. Every ms field now parses through one helper: json.Number first as int64, else as float64 with an integrality check — decidable without precision loss, since every in-range value sits far below float64's 2^53 exact-integer ceiling (maxScenarioMs ≈ 3.16e11), which the helper states. A non-integral number is refused in the schema's terms ("is not an integer: the schema's type is integer — a number whose mathematical value is integral"), never in encoding/json's; a number beyond 2^53 is refused as beyond any modeled value. optionalMs (the five config durations and both envelope members) and advance.ms (a new scenarioMs type, so the field-level unmarshaler leaves decodeStrict's unknown-key strictness intact) share the helper. Red-proven: 1000.0 and 1e3 failed to load against the previous driver, and 1000.5 was refused with encoding/json's message rather than the schema's — five cases across the focused spelling test and the rejection table, all red first, all green after, with the accepted spellings asserted to decode to exactly 1000. Gates: both eventfeed packages green, plain and -race; doc-constants green under LC_ALL=C; gofmt clean. --- .../eventfeed/scenario_conformance_test.go | 2 +- .../eventfeed/scenario_fixture_test.go | 47 +++++++++++++++++-- .../eventfeed/scenario_selftest_test.go | 45 ++++++++++++++++++ 3 files changed, 90 insertions(+), 4 deletions(-) diff --git a/go/pkg/basecamp/eventfeed/scenario_conformance_test.go b/go/pkg/basecamp/eventfeed/scenario_conformance_test.go index af3308c7f..f650189e7 100644 --- a/go/pkg/basecamp/eventfeed/scenario_conformance_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_conformance_test.go @@ -636,7 +636,7 @@ func (d *driver) nextClientFrame(what string) (clientFrame, error) { // AdvanceSettling remains for a caller that genuinely wants a chained firing // with an explicit rendezvous. It is deliberately not reachable from a fixture. func (d *driver) advance(step *advanceStep) error { - if due, ok := d.h.clock.AdvanceIfQuiet(millis(step.Ms)); !ok { + if due, ok := d.h.clock.AdvanceIfQuiet(millis(int64(step.Ms))); !ok { return fmt.Errorf( "advance of %dms would fire %v: whether a timer armed by one of those firings lands inside the "+ "same window depends on goroutine scheduling, so this script cannot mean the same thing in "+ diff --git a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go index 1bfa034f6..901384500 100644 --- a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go @@ -16,6 +16,7 @@ import ( "errors" "fmt" "io" + "math" "regexp" "strconv" "strings" @@ -81,7 +82,47 @@ func (o *optionalMs) UnmarshalJSON(data []byte) error { o.null = true return nil } - return json.Unmarshal(data, &o.v) + v, err := parseIntegralMs(data) + o.v = v + return err +} + +// parseIntegralMs parses one JSON number the way draft 2020-12's "integer" +// judges it: by MATHEMATICAL value, not spelling — 1000.0 and 1e3 are integer +// instances a schema-valid fixture may carry, and only this driver was +// refusing them (the float-spelled-int class FlexInt absorbs on the +// rich-text lane). Integrality is decidable without precision loss here: +// every in-range value sits far below float64's 2^53 exact-integer ceiling +// (maxScenarioMs ≈ 3.16e11), so a spelling that only parses as a float still +// carries its value exactly. +func parseIntegralMs(data []byte) (int64, error) { + var n json.Number + if err := json.Unmarshal(data, &n); err != nil { + return 0, err + } + if v, err := n.Int64(); err == nil { + return v, nil + } + f, err := n.Float64() + if err != nil { + return 0, err + } + if f != math.Trunc(f) { + return 0, fmt.Errorf("%s is not an integer: the schema's type is integer — a number whose mathematical value is integral", n) + } + if math.Abs(f) > 1<<53 { + return 0, fmt.Errorf("%s is beyond float64's exact-integer range — and beyond any modeled ms value", n) + } + return int64(f), nil +} + +// scenarioMs is a required ms value under the same number model. +type scenarioMs int64 + +func (m *scenarioMs) UnmarshalJSON(data []byte) error { + v, err := parseIntegralMs(data) + *m = scenarioMs(v) + return err } // storeScript is the schema's scripted CheckpointStore. @@ -222,7 +263,7 @@ type goneBody struct { } type advanceStep struct { - Ms int64 `json:"ms"` + Ms scenarioMs `json:"ms"` } type fireTimerStep struct { @@ -507,7 +548,7 @@ func decodeDirective(kind string, body json.RawMessage) (any, error) { if err := decodeStrict(body, step); err != nil { return nil, err } - return step, checkScenarioMs("advance ms", step.Ms, 1) + return step, checkScenarioMs("advance ms", int64(step.Ms), 1) case "fireTimer": step := &fireTimerStep{} if err := decodeStrict(body, step); err != nil { diff --git a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go index e0a99289b..7b8c5794a 100644 --- a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go @@ -537,6 +537,14 @@ func TestScenarioDriverRejectsUnmodelledScripts(t *testing.T) { script: `{"name":"x","description":"d","steps":[{"fireTimer":{"kind":"backoff","assertDelayMs":{"min":null,"max":10}}}],"finally":{"state":"closed"}}`, wants: "supplied as JSON null", }, + { + // draft 2020-12 judges the VALUE, not the spelling: 1000.5 is not + // an integer instance, and the refusal should say so in the + // schema's terms rather than in encoding/json's. + name: "advance ms non-integral number", + script: `{"name":"x","description":"d","steps":[{"advance":{"ms":1000.5}}],"finally":{"state":"closed"}}`, + wants: "is not an integer", + }, { // An advance is deterministic only from a scripted rendezvous: // an action's completion can precede the timer arms its @@ -653,6 +661,43 @@ func TestScenarioDriverRejectsUnmatchedActions(t *testing.T) { // deliberately so — the arming rule could only be enforced by sampling, and a // sampled MUST is not one. // +// TestScenarioMsAcceptsIntegralNumberSpellings pins the schema's number +// model onto the loader: draft 2020-12's "integer" is any number whose +// MATHEMATICAL value is integral, so 1000.0 and 1e3 are integer instances a +// schema-valid fixture may carry, and only the Go driver was refusing them — +// the same float-spelled-int class FlexInt absorbs on the rich-text lane. +// Integrality is decidable without precision loss: everything in range sits +// far below float64's 2^53 exact-integer ceiling. +func TestScenarioMsAcceptsIntegralNumberSpellings(t *testing.T) { + base := `{"name":"x","description":"d","config":{"stalenessMs":%s},"steps":[{"advance":{"ms":%s}}],"finally":{"state":"closed"}}` + cases := []struct { + name, staleness, ms, wantErr string + }{ + {"float spelling", "1000.0", "1000.0", ""}, + {"exponent spelling", "1e3", "1e3", ""}, + {"non-integral", "1000.5", "1000", "is not an integer"}, + {"non-integral ms", "1000", "1000.5", "is not an integer"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + raw := fmt.Sprintf(base, tc.staleness, tc.ms) + sc, err := parseScenario([]byte(raw), "x.json") + if tc.wantErr == "" { + if err != nil { + t.Fatalf("an integral spelling must load: %v", err) + } + if got := int64(sc.Config.StalenessMs.v); got != 1000 { + t.Errorf("stalenessMs decoded to %d, want 1000", got) + } + return + } + if err == nil || !strings.Contains(err.Error(), tc.wantErr) { + t.Fatalf("err = %v, want one naming %q", err, tc.wantErr) + } + }) + } +} + // The control matters as much as the mutants: an advance over a window with // nothing due is ordinary and must still pass, or the guard would be rejecting // every advance and the suite's one real advance (fixture 05) would be failing From 27b50e8f5fb3a12537e20b38efb54f15b9e42ad7 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Mon, 24 Aug 2026 23:30:52 -0700 Subject: [PATCH 39/55] Judge integrality on the literal, refuse quoted numbers, appease unconvert MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Lint failure and Copilot's + Codex's three threads (two of them the same defect), folded into one commit. * Lint: golangci-lint's unconvert flagged the int64() around a field that is already int64 in the new spelling test. Dropped; the local run reports 0 issues. * Integrality is a fact about the TEXT, and the float64 detour erased it: Float64 rounds 1000.00000000000001 to exactly 1000 — and 315575999999.99999 to exactly the maximum, which then PASSED the range check too — before any Trunc could look. The 2^53 argument guaranteed exact representation for integers, not for arbitrary decimals; both threads are right. The literal json.Number preserves is now judged exactly with big.Rat: integral iff the rational's denominator is one, ranged via the numerator's IsInt64, no float anywhere. An exponent is read as text first and refused for its magnitude outside ±40, so an exponent bomb (1e999999999) is never materialized into a billion-digit rational. * json.Number's own Unmarshal accepts quoted numeric strings, so {"ms":"1000"} decoded and passed every later check while the schema's type refuses string instances. The first non-space byte decides: a leading quote is refused as a string in the schema's terms before any number parsing. Red-proven: five new cases against the previous parser — the rounding-boundary fraction and the near-maximum fraction both ACCEPTED (the latter as the maximum itself), both quoted spellings ACCEPTED, and the exponent bomb refused with the wrong words — alongside the exact-maximum float spelling accepted both sides as the control. All green after, with accepted spellings asserted to decode to the literal's integral value. Gates: both eventfeed packages green, plain and -race; golangci-lint 0 issues locally; doc-constants-check green under LC_ALL=C; gofmt clean. --- .../eventfeed/scenario_fixture_test.go | 43 ++++++++++++------- .../eventfeed/scenario_selftest_test.go | 16 ++++++- 2 files changed, 42 insertions(+), 17 deletions(-) diff --git a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go index 901384500..ef99cc26f 100644 --- a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go @@ -16,7 +16,7 @@ import ( "errors" "fmt" "io" - "math" + "math/big" "regexp" "strconv" "strings" @@ -91,29 +91,42 @@ func (o *optionalMs) UnmarshalJSON(data []byte) error { // judges it: by MATHEMATICAL value, not spelling — 1000.0 and 1e3 are integer // instances a schema-valid fixture may carry, and only this driver was // refusing them (the float-spelled-int class FlexInt absorbs on the -// rich-text lane). Integrality is decidable without precision loss here: -// every in-range value sits far below float64's 2^53 exact-integer ceiling -// (maxScenarioMs ≈ 3.16e11), so a spelling that only parses as a float still -// carries its value exactly. +// rich-text lane). Integrality is a fact about the TEXT, which json.Number +// preserves: a float64 detour rounds 1000.00000000000001 to exactly 1000 — +// and 315575999999.99999 to exactly the maximum — before any check can look, +// so the literal is judged exactly, with big.Rat. Two gates come first: a +// quoted "1000" is a STRING instance the schema refuses, though json.Number's +// own Unmarshal would take it; and an exponent is read as a NUMBER'S text +// before anything is materialized, so an exponent bomb (1e999999999) is +// refused for its magnitude, never expanded. func parseIntegralMs(data []byte) (int64, error) { + trimmed := strings.TrimLeft(string(data), " \t\r\n") + if strings.HasPrefix(trimmed, `"`) { + return 0, fmt.Errorf("%s is a string: the schema's type is integer — quote-wrapping a number makes it a different instance", trimmed) + } var n json.Number if err := json.Unmarshal(data, &n); err != nil { return 0, err } - if v, err := n.Int64(); err == nil { - return v, nil + lit := n.String() + if i := strings.IndexAny(lit, "eE"); i >= 0 { + exp, err := strconv.Atoi(strings.TrimPrefix(lit[i+1:], "+")) + if err != nil || exp > 40 || exp < -40 { + return 0, fmt.Errorf("%s is beyond any modeled ms value: its exponent alone puts it outside [0, %d]", lit, maxScenarioMs) + } } - f, err := n.Float64() - if err != nil { - return 0, err + r, ok := new(big.Rat).SetString(lit) + if !ok { + return 0, fmt.Errorf("%s is not a number this driver can read", lit) } - if f != math.Trunc(f) { - return 0, fmt.Errorf("%s is not an integer: the schema's type is integer — a number whose mathematical value is integral", n) + if !r.IsInt() { + return 0, fmt.Errorf("%s is not an integer: the schema's type is integer — a number whose mathematical value is integral", lit) } - if math.Abs(f) > 1<<53 { - return 0, fmt.Errorf("%s is beyond float64's exact-integer range — and beyond any modeled ms value", n) + num := r.Num() + if !num.IsInt64() { + return 0, fmt.Errorf("%s is beyond any modeled ms value", lit) } - return int64(f), nil + return num.Int64(), nil } // scenarioMs is a required ms value under the same number model. diff --git a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go index 7b8c5794a..8e70b47f8 100644 --- a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go @@ -677,6 +677,18 @@ func TestScenarioMsAcceptsIntegralNumberSpellings(t *testing.T) { {"exponent spelling", "1e3", "1e3", ""}, {"non-integral", "1000.5", "1000", "is not an integer"}, {"non-integral ms", "1000", "1000.5", "is not an integer"}, + // Integrality is a fact about the TEXT: float64 rounds these to + // exactly 1000 and exactly the maximum before any Trunc can look. + {"rounding-boundary fraction", "1000.00000000000001", "1000", "is not an integer"}, + {"near-maximum fraction", "315575999999.99999", "1000", "is not an integer"}, + {"exact maximum float spelling", "315576000000.0", "1000", ""}, + // A quoted "1000" is a STRING instance — schema type integer refuses + // it, and json.Number's Unmarshal would happily have taken it. + {"quoted number", `"1000"`, "1000", "is a string"}, + {"quoted ms", "1000", `"1000"`, "is a string"}, + // An exponent bomb must be refused by reading the exponent, never by + // materializing the number. + {"exponent bomb", "1e999999999", "1000", "beyond any modeled ms value"}, } for _, tc := range cases { t.Run(tc.name, func(t *testing.T) { @@ -686,8 +698,8 @@ func TestScenarioMsAcceptsIntegralNumberSpellings(t *testing.T) { if err != nil { t.Fatalf("an integral spelling must load: %v", err) } - if got := int64(sc.Config.StalenessMs.v); got != 1000 { - t.Errorf("stalenessMs decoded to %d, want 1000", got) + if got := sc.Config.StalenessMs.v; got != 1000 && got != 315576000000 { + t.Errorf("stalenessMs decoded to %d, want the literal's integral value", got) } return } From 09878333e90f193e3d57c765e6762d15cddd46ce Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Tue, 25 Aug 2026 01:36:12 -0700 Subject: [PATCH 40/55] Judge a number's magnitude by its value, not its exponent's spelling Both bots caught the same edge in the exponent guard: a significand can offset any exponent, so 1e44-digits x e-41 is exactly 1000 and the lexical +/-40 cap rejected schema-valid integers. Two exact string judgments replace it: the most significant nonzero digit's decimal place caps the value (above place 13 nothing fits the range), and a least significant nonzero digit below the units place is non-integral outright (decimal digits do not carry) -- which refuses 1e-999999999 without a billion-digit denominator. A 100000-character literal cap comes first so no multi-megabyte number is walked into a rational. The test table (authored before the session limit took the subagent that started this round) covers both bots' offset spellings, the fraction-led spelling of one, both bombs, and the parse-bomb literal; run against the pre-fix parser the offset cases fail to load, which is the red proof. --- .../eventfeed/scenario_fixture_test.go | 49 +++++++++++++++++-- .../eventfeed/scenario_selftest_test.go | 35 +++++++------ 2 files changed, 67 insertions(+), 17 deletions(-) diff --git a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go index ef99cc26f..6b8d590c3 100644 --- a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go @@ -109,10 +109,53 @@ func parseIntegralMs(data []byte) (int64, error) { return 0, err } lit := n.String() + // A bomb is refused before anything is materialized, and by the value's + // EFFECTIVE magnitude, never the exponent's spelling alone — draft + // 2020-12 constrains the mathematical value, and a significand can + // offset any exponent (1e44-digits × e-41 is exactly 1000). Two string + // judgments suffice, both exact: + // - the most significant nonzero digit's decimal place caps the + // value: above place 13 nothing fits [0, maxScenarioMs] (12 digits); + // - a least significant nonzero digit below the units place makes the + // value non-integral outright (decimal digits do not carry), which + // refuses 1e-999999999 without a 10^999999999 denominator. + // A length cap comes first so no multi-megabyte literal is ever walked + // into a rational. + if len(lit) > 100000 { + return 0, fmt.Errorf("a %d-character number is beyond any modeled ms value (literals are capped at 100000 characters)", len(lit)) + } + mant, exp := lit, 0 if i := strings.IndexAny(lit, "eE"); i >= 0 { - exp, err := strconv.Atoi(strings.TrimPrefix(lit[i+1:], "+")) - if err != nil || exp > 40 || exp < -40 { - return 0, fmt.Errorf("%s is beyond any modeled ms value: its exponent alone puts it outside [0, %d]", lit, maxScenarioMs) + mant = lit[:i] + e, err := strconv.Atoi(strings.TrimPrefix(lit[i+1:], "+")) + if err != nil { + return 0, fmt.Errorf("%s is not a number this driver can read", lit) + } + exp = e + } + digits := strings.TrimPrefix(mant, "-") + point := strings.IndexByte(digits, '.') + intLen := len(digits) + if point >= 0 { + intLen = point + digits = digits[:point] + digits[point+1:] + } + firstNZ, lastNZ := -1, -1 + for i := 0; i < len(digits); i++ { + if digits[i] >= '1' && digits[i] <= '9' { + if firstNZ < 0 { + firstNZ = i + } + lastNZ = i + } + } + if firstNZ >= 0 { // zero needs no magnitude judgment + // Digit i occupies decimal place intLen - i + exp (units = 1). + if msd := intLen - firstNZ + exp; msd > 13 { + return 0, fmt.Errorf("%s is beyond any modeled ms value", lit) + } + if lsd := intLen - lastNZ + exp; lsd < 1 { + return 0, fmt.Errorf("%s is not an integer: the schema's type is integer — a number whose mathematical value is integral", lit) } } r, ok := new(big.Rat).SetString(lit) diff --git a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go index 8e70b47f8..7d10d8f3f 100644 --- a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go @@ -672,23 +672,30 @@ func TestScenarioMsAcceptsIntegralNumberSpellings(t *testing.T) { base := `{"name":"x","description":"d","config":{"stalenessMs":%s},"steps":[{"advance":{"ms":%s}}],"finally":{"state":"closed"}}` cases := []struct { name, staleness, ms, wantErr string + want int64 }{ - {"float spelling", "1000.0", "1000.0", ""}, - {"exponent spelling", "1e3", "1e3", ""}, - {"non-integral", "1000.5", "1000", "is not an integer"}, - {"non-integral ms", "1000", "1000.5", "is not an integer"}, + {"float spelling", "1000.0", "1000.0", "", 1000}, + {"exponent spelling", "1e3", "1e3", "", 1000}, + {"non-integral", "1000.5", "1000", "is not an integer", 0}, + {"non-integral ms", "1000", "1000.5", "is not an integer", 0}, // Integrality is a fact about the TEXT: float64 rounds these to // exactly 1000 and exactly the maximum before any Trunc can look. - {"rounding-boundary fraction", "1000.00000000000001", "1000", "is not an integer"}, - {"near-maximum fraction", "315575999999.99999", "1000", "is not an integer"}, - {"exact maximum float spelling", "315576000000.0", "1000", ""}, + {"rounding-boundary fraction", "1000.00000000000001", "1000", "is not an integer", 0}, + {"near-maximum fraction", "315575999999.99999", "1000", "is not an integer", 0}, + {"exact maximum float spelling", "315576000000.0", "1000", "", 315576000000}, // A quoted "1000" is a STRING instance — schema type integer refuses // it, and json.Number's Unmarshal would happily have taken it. - {"quoted number", `"1000"`, "1000", "is a string"}, - {"quoted ms", "1000", `"1000"`, "is a string"}, - // An exponent bomb must be refused by reading the exponent, never by - // materializing the number. - {"exponent bomb", "1e999999999", "1000", "beyond any modeled ms value"}, + {"quoted number", `"1000"`, "1000", "is a string", 0}, + {"quoted ms", "1000", `"1000"`, "is a string", 0}, + // A bomb is refused by its EFFECTIVE magnitude — significand length + // plus exponent — never by materializing the number, and never by + // the exponent's spelling alone: a significand can offset it. + {"exponent bomb", "1e999999999", "1000", "beyond any modeled ms value", 0}, + {"significand offsets a negative exponent", "100000000000000000000000000000000000000000000e-41", "1000", "", 1000}, + {"significand offsets to one hundred", "1000000000000000000000000000000000000000000e-40", "1000", "", 100}, + {"fraction-led spelling of one", "0.00000000000000000000000000000000000000001e41", "1000", "", 1}, + {"negative exponent bomb", "1e-999999999", "1000", "is not an integer", 0}, + {"parse-bomb literal", strings.Repeat("9", 100001), "1000", "characters", 0}, } for _, tc := range cases { t.Run(tc.name, func(t *testing.T) { @@ -698,8 +705,8 @@ func TestScenarioMsAcceptsIntegralNumberSpellings(t *testing.T) { if err != nil { t.Fatalf("an integral spelling must load: %v", err) } - if got := sc.Config.StalenessMs.v; got != 1000 && got != 315576000000 { - t.Errorf("stalenessMs decoded to %d, want the literal's integral value", got) + if got := sc.Config.StalenessMs.v; got != tc.want { + t.Errorf("stalenessMs decoded to %d, want %d", got, tc.want) } return } From 4377b371639467f10c625a9411b73039baa047cf Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Tue, 25 Aug 2026 01:47:47 -0700 Subject: [PATCH 41/55] Scope the peak to payload, and sanction the literal cap in the schema Copilot round: the peak equality read as a process-memory bound while the section itself allows an unbounded transport-authored error item and counts no runtime metadata -- it now says peak FRAME-PAYLOAD retention and names both exclusions. And the 100000-character literal cap was a Go-only constraint the schema never declared (1 followed by 100k zeros times e-100000 is mathematically 1): the schema's top-level description now sanctions the bound for every driver as a resource limit on spellings, not a value constraint, and the driver's comment points at the sanction. --- SPEC.md | 9 +++++++-- conformance/event-feed/schema.json | 2 +- go/pkg/basecamp/eventfeed/scenario_fixture_test.go | 4 +++- 3 files changed, 11 insertions(+), 4 deletions(-) diff --git a/SPEC.md b/SPEC.md index 7cfbef595..eefa2a0b6 100644 --- a/SPEC.md +++ b/SPEC.md @@ -3669,8 +3669,13 @@ Two dispatch clarifications, pinned: moved when the exact-spelling per-field decode was coded in, and pinning a number would turn every decoder refactor into a spec change. What IS contract: transients exist only between a frame's dequeue and its decode returning, one frame at a time, - so **peak memory is the retained formula plus ONE frame's transient allocation** — - never a per-slot or per-queue multiplier. + so **peak frame-payload retention is the retained formula plus ONE frame's + transient allocation** — never a per-slot or per-queue multiplier. Payload, not + process memory: the equality deliberately excludes the two things it cannot + bound — the single transport-authored error item, whose size is its author's + (stated above), and runtime metadata (channel and `pumpItem` storage, slice and + map headers), which scales with the configured capacities, not with frame + bytes. The retained enumeration cannot grow by a further party being noticed: every retained frame is in one of the three counted structures or in the hands of the pump diff --git a/conformance/event-feed/schema.json b/conformance/event-feed/schema.json index dbbb3057c..d5e4ab732 100644 --- a/conformance/event-feed/schema.json +++ b/conformance/event-feed/schema.json @@ -2,7 +2,7 @@ "$schema": "https://json-schema.org/draft/2020-12/schema", "$id": "https://basecamp.com/schemas/event-feed-scenario.json", "title": "Event Feed Connector Tier-2 Scenario", - "description": "One strictly-ordered interleaved script driving the SPEC §23 connector over its five seams: HTTP exchanges (mint/poll), cable frames, time directives, and observations, in one `steps` array. Strict-matched actions (mint, connect, poll, outbound frame, client close, checkpoint save) must each match their expect step under the per-action-class rules in this family's README ('Strictness semantics, per action class': saves and outbound frames are arrival-strict; mint/poll seam calls are parked and matched in order); observation directives (delivered, buffered, timers, state, signals, handler invocations) are rendezvous assertions evaluated cumulatively under a small wall-clock watchdog while virtual time is frozen. All counts (mintCount, connectCount) count SEAM CALLS — one fully-governed generated call each, with SPEC §7 retries inside — never wire attempts. Time flows only through the injected Clock seam; the virtual-advance algorithm and the ownership-cut/dequeued-frames semantics are normative prose in this family's README and SPEC §23. Validated by `make event-feed-fixtures-check`.", + "description": "One strictly-ordered interleaved script driving the SPEC §23 connector over its five seams. NUMERIC LITERALS: schema keywords judge mathematical value (1e3 and 1000.0 are integer instances) and drivers judge integrality and range on the literal's exact value — but a driver MAY refuse any numeric literal longer than 100000 characters regardless of value: a sanctioned resource bound on pathological spellings, not a value constraint. Continuing: HTTP exchanges (mint/poll), cable frames, time directives, and observations, in one `steps` array. Strict-matched actions (mint, connect, poll, outbound frame, client close, checkpoint save) must each match their expect step under the per-action-class rules in this family's README ('Strictness semantics, per action class': saves and outbound frames are arrival-strict; mint/poll seam calls are parked and matched in order); observation directives (delivered, buffered, timers, state, signals, handler invocations) are rendezvous assertions evaluated cumulatively under a small wall-clock watchdog while virtual time is frozen. All counts (mintCount, connectCount) count SEAM CALLS — one fully-governed generated call each, with SPEC §7 retries inside — never wire attempts. Time flows only through the injected Clock seam; the virtual-advance algorithm and the ownership-cut/dequeued-frames semantics are normative prose in this family's README and SPEC §23. Validated by `make event-feed-fixtures-check`.", "type": "object", "additionalProperties": false, "required": [ diff --git a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go index 6b8d590c3..af71871f8 100644 --- a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go @@ -120,7 +120,9 @@ func parseIntegralMs(data []byte) (int64, error) { // value non-integral outright (decimal digits do not carry), which // refuses 1e-999999999 without a 10^999999999 denominator. // A length cap comes first so no multi-megabyte literal is ever walked - // into a rational. + // into a rational — the schema's top-level description sanctions exactly + // this bound (a resource limit on spellings, not a value constraint), so + // refusing "1" + 100k zeros + e-100000 (mathematically 1) is conformant. if len(lit) > 100000 { return 0, fmt.Errorf("a %d-character number is beyond any modeled ms value (literals are capped at 100000 characters)", len(lit)) } From 66e7d7a5dbefb265e79829e21172e50eac6674ce Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Tue, 25 Aug 2026 02:04:30 -0700 Subject: [PATCH 42/55] Bound the exponent before the place arithmetic can wrap Copilot: the platform's max integer as an exponent wrapped intLen - firstNZ + exp negative and bypassed the magnitude judgments. The exponent is bounded at +/-200000 right after parsing -- the literal cap already bounds the significand at 100000 digits, so no in-range value needs more to spell -- and both integer-extreme spellings are pinned. The negative-bomb case's expected message moves to the new guard's, which fires first. --- go/pkg/basecamp/eventfeed/scenario_fixture_test.go | 8 ++++++++ go/pkg/basecamp/eventfeed/scenario_selftest_test.go | 6 +++++- 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go index af71871f8..1a948ee2a 100644 --- a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go @@ -133,6 +133,14 @@ func parseIntegralMs(data []byte) (int64, error) { if err != nil { return 0, fmt.Errorf("%s is not a number this driver can read", lit) } + // Bound the exponent before any place arithmetic: at the platform's + // integer extremes, intLen - firstNZ + exp wraps and the magnitude + // judgments below judge garbage. The literal cap above bounds the + // significand at 100000 digits, so no in-range value needs an + // exponent beyond ±200000 to spell. + if e > 200000 || e < -200000 { + return 0, fmt.Errorf("%s is beyond any modeled ms value", lit) + } exp = e } digits := strings.TrimPrefix(mant, "-") diff --git a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go index 7d10d8f3f..80127ddc9 100644 --- a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go @@ -694,8 +694,12 @@ func TestScenarioMsAcceptsIntegralNumberSpellings(t *testing.T) { {"significand offsets a negative exponent", "100000000000000000000000000000000000000000000e-41", "1000", "", 1000}, {"significand offsets to one hundred", "1000000000000000000000000000000000000000000e-40", "1000", "", 100}, {"fraction-led spelling of one", "0.00000000000000000000000000000000000000001e41", "1000", "", 1}, - {"negative exponent bomb", "1e-999999999", "1000", "is not an integer", 0}, + {"negative exponent bomb", "1e-999999999", "1000", "beyond any modeled ms value", 0}, {"parse-bomb literal", strings.Repeat("9", 100001), "1000", "characters", 0}, + // The platform's max integer as an exponent must not wrap the place + // arithmetic into acceptance. + {"max-int exponent", "1e9223372036854775807", "1000", "beyond any modeled ms value", 0}, + {"min-int exponent", "1e-9223372036854775808", "1000", "beyond any modeled ms value", 0}, } for _, tc := range cases { t.Run(tc.name, func(t *testing.T) { From 3a8fae8f75a5017f2f0ae49454d8d38d52088ac7 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Tue, 25 Aug 2026 02:16:16 -0700 Subject: [PATCH 43/55] Short-circuit zero before its exponent reaches a rational Codex: a zero significand skipped the magnitude guard (firstNZ stays -1), handing the raw exponent onward. Zero now returns immediately -- integral however spelled, judged by the range check like any other zero -- pinned by the 0e199999 case landing in the range refusal, never a parse error. --- go/pkg/basecamp/eventfeed/scenario_fixture_test.go | 8 +++++++- go/pkg/basecamp/eventfeed/scenario_selftest_test.go | 3 +++ 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go index 1a948ee2a..b3188a326 100644 --- a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go @@ -159,7 +159,13 @@ func parseIntegralMs(data []byte) (int64, error) { lastNZ = i } } - if firstNZ >= 0 { // zero needs no magnitude judgment + if firstNZ < 0 { + // Zero, however spelled (0, 0.000, 0e200000): integral, in range + // judgment's hands, and returned without ever expanding an exponent + // into a rational. + return 0, nil + } + { // Digit i occupies decimal place intLen - i + exp (units = 1). if msd := intLen - firstNZ + exp; msd > 13 { return 0, fmt.Errorf("%s is beyond any modeled ms value", lit) diff --git a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go index 80127ddc9..9d2824de6 100644 --- a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go @@ -700,6 +700,9 @@ func TestScenarioMsAcceptsIntegralNumberSpellings(t *testing.T) { // arithmetic into acceptance. {"max-int exponent", "1e9223372036854775807", "1000", "beyond any modeled ms value", 0}, {"min-int exponent", "1e-9223372036854775808", "1000", "beyond any modeled ms value", 0}, + // Zero short-circuits before any exponent expansion; a zero + // stalenessMs is then the RANGE check's refusal, not a parse error. + {"zero with a large exponent", "0e199999", "1000", "must be in [1,", 0}, } for _, tc := range cases { t.Run(tc.name, func(t *testing.T) { From b7d28399698eef54a683fad045102c6c4ec73f6b Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Tue, 25 Aug 2026 02:27:11 -0700 Subject: [PATCH 44/55] Decide zero before the exponent is even parsed Codex: 0e200001 is nine characters with value zero, and the exponent cap ran first. The significand scan now precedes exponent parsing entirely -- an exponent multiplies a significand, and a zero one needs no arithmetic -- pinned by the beyond-cap zero landing in the range refusal. --- .../eventfeed/scenario_fixture_test.go | 39 +++++++++++-------- .../eventfeed/scenario_selftest_test.go | 1 + 2 files changed, 23 insertions(+), 17 deletions(-) diff --git a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go index b3188a326..28b993499 100644 --- a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go @@ -126,22 +126,10 @@ func parseIntegralMs(data []byte) (int64, error) { if len(lit) > 100000 { return 0, fmt.Errorf("a %d-character number is beyond any modeled ms value (literals are capped at 100000 characters)", len(lit)) } - mant, exp := lit, 0 + mant, expText := lit, "" if i := strings.IndexAny(lit, "eE"); i >= 0 { mant = lit[:i] - e, err := strconv.Atoi(strings.TrimPrefix(lit[i+1:], "+")) - if err != nil { - return 0, fmt.Errorf("%s is not a number this driver can read", lit) - } - // Bound the exponent before any place arithmetic: at the platform's - // integer extremes, intLen - firstNZ + exp wraps and the magnitude - // judgments below judge garbage. The literal cap above bounds the - // significand at 100000 digits, so no in-range value needs an - // exponent beyond ±200000 to spell. - if e > 200000 || e < -200000 { - return 0, fmt.Errorf("%s is beyond any modeled ms value", lit) - } - exp = e + expText = strings.TrimPrefix(lit[i+1:], "+") } digits := strings.TrimPrefix(mant, "-") point := strings.IndexByte(digits, '.') @@ -160,11 +148,28 @@ func parseIntegralMs(data []byte) (int64, error) { } } if firstNZ < 0 { - // Zero, however spelled (0, 0.000, 0e200000): integral, in range - // judgment's hands, and returned without ever expanding an exponent - // into a rational. + // Zero, however spelled (0, 0.000, 0e200001): integral, the range + // judgment's to refuse, and decided before the exponent is even + // parsed — an exponent multiplies a significand, and this one is + // zero. return 0, nil } + exp := 0 + if expText != "" { + e, err := strconv.Atoi(expText) + if err != nil { + return 0, fmt.Errorf("%s is not a number this driver can read", lit) + } + // Bound the exponent before any place arithmetic: at the platform's + // integer extremes, intLen - firstNZ + exp wraps and the magnitude + // judgments below judge garbage. The literal cap above bounds the + // significand at 100000 digits, so no in-range value needs an + // exponent beyond ±200000 to spell. + if e > 200000 || e < -200000 { + return 0, fmt.Errorf("%s is beyond any modeled ms value", lit) + } + exp = e + } { // Digit i occupies decimal place intLen - i + exp (units = 1). if msd := intLen - firstNZ + exp; msd > 13 { diff --git a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go index 9d2824de6..e6c0949f7 100644 --- a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go @@ -703,6 +703,7 @@ func TestScenarioMsAcceptsIntegralNumberSpellings(t *testing.T) { // Zero short-circuits before any exponent expansion; a zero // stalenessMs is then the RANGE check's refusal, not a parse error. {"zero with a large exponent", "0e199999", "1000", "must be in [1,", 0}, + {"zero beyond the exponent cap", "0e200001", "1000", "must be in [1,", 0}, } for _, tc := range cases { t.Run(tc.name, func(t *testing.T) { From 6aab6e78fd0071703d118ac1cd2d9a4ad7a25658 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Tue, 25 Aug 2026 03:16:36 -0700 Subject: [PATCH 45/55] State the rendezvous guarantee at its true strength: flaky, never green MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Settle rounds five and six, both declined as mechanism — with one sentence of the contract corrected, because Codex's new observation is true about the WORDING even though it changes nothing about the design. Round five re-litigates the stated residue verbatim: a serve in streaming whose pump-side staleness rearm races a following advance is the contract paragraph's own named example, with its two sanctioned outcomes (override stalenessMs large, as the suite's one advance does at ~11.5 virtual days against a 121-second window; or fireTimer for the firing the script actually wants). Reachability: exactly one advance exists across all committed fixtures and it follows the mitigation. Prior threads 3849496601, 3849540097, and 3849863896 carry the instrument history, including why the timer-generation barrier was examined and declined — whether an announcement lands before or after the preceding expectation completes is itself a scheduling race, so generation-newness makes the one legitimate advance flaky. Round six shows the "fails loudly" sentence overclaimed by a schedule: a pair naming the PRE-action state can pass on the schedule where the action has not yet been processed, so wrong authorship is at worst FLAKY — red whenever the transition lands first — never stably green. That is a fact about the wording, not the design: the settled guarantee always belonged to correctly authored pairs, and both documents now say exactly that instead of implying every wrong script fails every run. A loader-side transition table could reject pre-action pairs statically, but it would be a second model of the connector for the loader to drift against — a new instrument, not a cheap strengthening — and no committed script authors a pre-action pair. Gates: doc-constants-check green under LC_ALL=C; tracking-issues live checker green; no code changed, so the Go suites stand at this head's green. --- SPEC.md | 5 ++++- conformance/event-feed/README.md | 7 ++++++- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/SPEC.md b/SPEC.md index eefa2a0b6..12ab7f387 100644 --- a/SPEC.md +++ b/SPEC.md @@ -4171,7 +4171,10 @@ set match can coincide with a transient mid-surgery set (timer surgery spans clo acquisitions), and an announcement can precede a tail arm. Together they do — the announcement bounds the surgery, and any timer still unarmed at the announcement is exactly what the exact-set match then waits for, both blocking under the watchdog so -wrong authorship fails loudly. A transition that announces no state change, or only +wrong authorship fails loudly on every schedule where the stale pair no longer holds — +a pre-action pair can pass on the schedule where the action is not yet processed, so a +wrong script is at worst flaky, never stably green; the settled guarantee is for +correctly authored pairs. A transition that announces no state change, or only rearms a timer of the same kind and count, is invisible to this rendezvous — a served live frame's pump-side `staleness` rearm is the concrete case. Such a script overrides `stalenessMs` large so no deadline, old or new, sits inside a window it advances (the diff --git a/conformance/event-feed/README.md b/conformance/event-feed/README.md index 7b172b25b..cbea41ebe 100644 --- a/conformance/event-feed/README.md +++ b/conformance/event-feed/README.md @@ -199,7 +199,12 @@ precede a tail arm: Backoff announces before its timer is armed. Together they settle — the announcement bounds the surgery, and any timer still unarmed at the announcement is exactly what the following exact-set match waits for. Both steps block under the scenario watchdog, so a wrongly authored state or set -fails loudly instead of diverging silently. What stays outside this +fails loudly on every schedule where the pair no longer holds. The precise +guarantee: a pair naming the PRE-action state can pass on the schedule where +the action has not yet been processed, so wrong authorship is at worst FLAKY — +red whenever the transition lands first — never stably green; the settled +guarantee belongs to correctly authored pairs, the ones the per-state tables +define. What stays outside this rendezvous is a transition that announces no state change, or only rearms a timer of the same kind and count — invisible to both barriers. The concrete case is a live frame served in a socket-open state: its receipt rearms From 72ef91fe430bcf2df4e7996065257b953e3dd396 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Tue, 25 Aug 2026 04:36:41 -0700 Subject: [PATCH 46/55] Point the row-15 redaction prose at the test that exists MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fixture 30's description and the README's row-15 note both named TestRedirectRefusalExposesOnlyTheLocationOrigin as the redaction proof's owner; the test that actually feeds the secret-bearing cause and walks the whole rendering and unwrap chain is TestRedirectRefusalRendersNoServerValue. The described obligation was met all along — only the name pointed at a ghost. --- conformance/event-feed/README.md | 2 +- .../fixtures/30-continuation-redirect-cross-origin.json | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/conformance/event-feed/README.md b/conformance/event-feed/README.md index cbea41ebe..dec5834d3 100644 --- a/conformance/event-feed/README.md +++ b/conformance/event-feed/README.md @@ -456,7 +456,7 @@ was. The driver performs the redaction itself, before the connector runs: no path or query text from the `Location` ever reaches the connector, so a connector that echoed its entire input verbatim would pass fixture 30 unchanged. Claiming it here would have been a kill that cannot fail. That proof -belongs to `TestRedirectRefusalExposesOnlyTheLocationOrigin`, which feeds a +belongs to `TestRedirectRefusalRendersNoServerValue`, which feeds a secret-bearing cause and asserts the terminal's whole rendering and cause chain never carry it — a test that exists today — and, for the real-adapter path, to Layer 1 once its adapters land. diff --git a/conformance/event-feed/fixtures/30-continuation-redirect-cross-origin.json b/conformance/event-feed/fixtures/30-continuation-redirect-cross-origin.json index 5d3b517f7..317744fe0 100644 --- a/conformance/event-feed/fixtures/30-continuation-redirect-cross-origin.json +++ b/conformance/event-feed/fixtures/30-continuation-redirect-cross-origin.json @@ -1,6 +1,6 @@ { "name": "30-continuation-redirect-cross-origin", -"description": "A VALIDATED same-origin `next` answers 302 with a cross-origin Location: the poll seam suppresses automatic redirect-following, and the foreign Location is Terminal(invalid_continuation). {{NEXT:1}} substitutes same-origin, so the pre-poll validation PASSES and the second poll seam call is made (contrast fixture 26, where no request reaches the URL at all) \u2014 the redirect answer is where the per-hop rule bites. What tier 2 verifies is the per-hop refusal DECISION \u2014 the driver classifies the scripted Location through the SHIPPED predicate (checkContinuation, via a test export), never an ad-hoc re-implementation \u2014 and the loop's response: the continuation terminal, the socket explicitly closed, and no further seam calls. That the adapter's HTTP client makes zero requests to the refused URL is below the poll seam and owned by Layer-1 adapter conformance \u2014 a recorded obligation, not an existing proof: those adapters are still pending (tracked in #819), so follow-cross-origin-redirect itself remains NOT killed at tier 2 (row 15). The Location host is literal, never substituted, and never served. Redaction is deliberately not claimed here: the driver reduces the Location to its origin best-effort (an unreducible Location carries none) before the connector runs, so no path or query text exists for a connector to over-echo; TestRedirectRefusalExposesOnlyTheLocationOrigin owns that proof. See the row-15 note in README.md.", "config": { +"description": "A VALIDATED same-origin `next` answers 302 with a cross-origin Location: the poll seam suppresses automatic redirect-following, and the foreign Location is Terminal(invalid_continuation). {{NEXT:1}} substitutes same-origin, so the pre-poll validation PASSES and the second poll seam call is made (contrast fixture 26, where no request reaches the URL at all) \u2014 the redirect answer is where the per-hop rule bites. What tier 2 verifies is the per-hop refusal DECISION \u2014 the driver classifies the scripted Location through the SHIPPED predicate (checkContinuation, via a test export), never an ad-hoc re-implementation \u2014 and the loop's response: the continuation terminal, the socket explicitly closed, and no further seam calls. That the adapter's HTTP client makes zero requests to the refused URL is below the poll seam and owned by Layer-1 adapter conformance \u2014 a recorded obligation, not an existing proof: those adapters are still pending (tracked in #819), so follow-cross-origin-redirect itself remains NOT killed at tier 2 (row 15). The Location host is literal, never substituted, and never served. Redaction is deliberately not claimed here: the driver reduces the Location to its origin best-effort (an unreducible Location carries none) before the connector runs, so no path or query text exists for a connector to over-echo; TestRedirectRefusalRendersNoServerValue owns that proof. See the row-15 note in README.md.", "config": { "position": "{{POS:0}}" }, "steps": [ From 4442bd10bb42ec0be8608f66709884fa4aff5bb5 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Tue, 25 Aug 2026 04:50:29 -0700 Subject: [PATCH 47/55] Absorb the exact-head round's three suppressed notes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The exponent parse moves from Atoi to a fixed 64-bit ParseInt: int is 32 bits on some targets, where 1e9223372036854775807 failed as unreadable before reaching the magnitude judgment and the diagnostic changed by platform. A 64-bit overflow can only mean the exponent is past the ±200000 bound, so range errors take the bound's own verdict — pinned by beyond-int64 cases that fail against the Atoi version on every width. The schema's advance description and the rendezvous loader's comment both still claimed wrong authorship "fails loudly rather than diverging"; both now carry the README/SPEC qualification — a pre-action pair can pass on the schedule where the action is not yet processed, so wrong authorship is at worst flaky, never stably green. --- conformance/event-feed/schema.json | 2 +- .../eventfeed/scenario_fixture_test.go | 20 +++++++++++++++---- .../eventfeed/scenario_selftest_test.go | 5 +++++ 3 files changed, 22 insertions(+), 5 deletions(-) diff --git a/conformance/event-feed/schema.json b/conformance/event-feed/schema.json index d5e4ab732..ef982d496 100644 --- a/conformance/event-feed/schema.json +++ b/conformance/event-feed/schema.json @@ -1327,7 +1327,7 @@ "required": [ "ms" ], - "description": "Advance virtual now by ms. The normative virtual-advance algorithm (README) fires due timers in deadline order, re-evaluating after each fire, with timers scheduled during the advance whose deadlines land inside the window also firing and ties breaking by creation order. That reentrant clause is UNSCRIPTABLE wherever the connector runs concurrently with the driver, so no fixture may rely on it: every driver MUST REJECT an advance whose window would fire ANY timer, naming fireTimer as the deterministic alternative. The test is what would FIRE, decided from the clock's state before time moves — not what gets armed, which happens on the connector's schedule and can only be sampled. An advance that fires nothing never wakes anything, so it cannot cause an arm; that is what makes the check complete rather than probabilistic. A firing that replaces nothing is rejected too. And an advance is deterministic only from a scripted rendezvous point: an action's completion can precede the timer arms its transition causes, so every advance must be the scenario's first step or immediately follow a TWO-STEP rendezvous: expectState, then expectTimers, enforced at load (an empty expectTimers set is rejected with it — it orders nothing). Neither step alone settles: a set match can coincide with a transient mid-surgery set (the welcome transition stops handshake-deadline and arms confirmation-deadline in separate clock acquisitions), and an announcement can precede a tail arm. Together they do: the announcement bounds the surgery, and any timer still unarmed at the announcement is exactly what the exact-set match then waits for. Both block under the watchdog, so wrong authorship fails loudly rather than diverging. A transition that announces no state change — or only rearms a timer of the same kind and count — is invisible to this rendezvous, and a script that would advance behind one uses fireTimer instead.", + "description": "Advance virtual now by ms. The normative virtual-advance algorithm (README) fires due timers in deadline order, re-evaluating after each fire, with timers scheduled during the advance whose deadlines land inside the window also firing and ties breaking by creation order. That reentrant clause is UNSCRIPTABLE wherever the connector runs concurrently with the driver, so no fixture may rely on it: every driver MUST REJECT an advance whose window would fire ANY timer, naming fireTimer as the deterministic alternative. The test is what would FIRE, decided from the clock's state before time moves — not what gets armed, which happens on the connector's schedule and can only be sampled. An advance that fires nothing never wakes anything, so it cannot cause an arm; that is what makes the check complete rather than probabilistic. A firing that replaces nothing is rejected too. And an advance is deterministic only from a scripted rendezvous point: an action's completion can precede the timer arms its transition causes, so every advance must be the scenario's first step or immediately follow a TWO-STEP rendezvous: expectState, then expectTimers, enforced at load (an empty expectTimers set is rejected with it — it orders nothing). Neither step alone settles: a set match can coincide with a transient mid-surgery set (the welcome transition stops handshake-deadline and arms confirmation-deadline in separate clock acquisitions), and an announcement can precede a tail arm. Together they do: the announcement bounds the surgery, and any timer still unarmed at the announcement is exactly what the exact-set match then waits for. Both block under the watchdog, so wrong authorship fails on every schedule where the stale pair no longer holds \u2014 a pair naming the PRE-action state can pass on the schedule where the action is not yet processed, so a wrong script is at worst flaky, never stably green; the settled guarantee belongs to correctly authored pairs. A transition that announces no state change — or only rearms a timer of the same kind and count — is invisible to this rendezvous, and a script that would advance behind one uses fireTimer instead.", "properties": { "ms": { "type": "integer", diff --git a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go index 28b993499..a949d9f74 100644 --- a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go @@ -156,8 +156,17 @@ func parseIntegralMs(data []byte) (int64, error) { } exp := 0 if expText != "" { - e, err := strconv.Atoi(expText) + // ParseInt at a FIXED width, not Atoi: int is 32 bits on some + // targets, where an exponent like 1e9223372036854775807 would fail + // as unreadable before reaching the magnitude judgment and change + // the diagnostic by platform. A 64-bit overflow can only mean the + // exponent is beyond the ±200000 bound below, so range errors take + // the bound's own verdict. + e, err := strconv.ParseInt(expText, 10, 64) if err != nil { + if errors.Is(err, strconv.ErrRange) { + return 0, fmt.Errorf("%s is beyond any modeled ms value", lit) + } return 0, fmt.Errorf("%s is not a number this driver can read", lit) } // Bound the exponent before any place arithmetic: at the platform's @@ -168,7 +177,7 @@ func parseIntegralMs(data []byte) (int64, error) { if e > 200000 || e < -200000 { return 0, fmt.Errorf("%s is beyond any modeled ms value", lit) } - exp = e + exp = int(e) } { // Digit i occupies decimal place intLen - i + exp (units = 1). @@ -505,8 +514,11 @@ func parseScenario(raw []byte, file string) (*scenario, error) { // the transition announces, and in every announced state any timer still // unarmed at the announcement is exactly what the following exact-set // match waits for — so the pair settles where either alone races. Both - // blocks fail loudly on the watchdog when the authored state or set is - // wrong; nothing diverges silently. + // blocks fail on the watchdog on every schedule where the stale pair no + // longer holds; a pair naming the PRE-action state can still pass on the + // schedule where the action is not yet processed, so wrong authorship is + // at worst flaky — never stably green (README "settle semantics"; the + // settled guarantee belongs to correctly authored pairs). for i, step := range sc.Steps { if step.Kind != "advance" || i == 0 { continue diff --git a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go index e6c0949f7..e4e4e15c2 100644 --- a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go @@ -700,6 +700,11 @@ func TestScenarioMsAcceptsIntegralNumberSpellings(t *testing.T) { // arithmetic into acceptance. {"max-int exponent", "1e9223372036854775807", "1000", "beyond any modeled ms value", 0}, {"min-int exponent", "1e-9223372036854775808", "1000", "beyond any modeled ms value", 0}, + // Exponents past int64 overflow the fixed-width parse itself; the + // range error takes the bound's own verdict, so the diagnostic is + // the same on every platform width. + {"beyond-int64 exponent", "1e9223372036854775808", "1000", "beyond any modeled ms value", 0}, + {"beyond-int64 negative exponent", "1e-9223372036854775809", "1000", "beyond any modeled ms value", 0}, // Zero short-circuits before any exponent expansion; a zero // stalenessMs is then the RANGE check's refusal, not a parse error. {"zero with a large exponent", "0e199999", "1000", "must be in [1,", 0}, From a220b04564541236c99a830d9d77b59a294d5ab7 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Wed, 9 Sep 2026 20:58:13 -0700 Subject: [PATCH 48/55] Register main's two #818 tracking claims with the registry the rebase brought them under --- spec/tracking-issues.yml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/spec/tracking-issues.yml b/spec/tracking-issues.yml index ffde60dfa..daea0a245 100644 --- a/spec/tracking-issues.yml +++ b/spec/tracking-issues.yml @@ -123,6 +123,10 @@ prose_tracking_issues: file: "MIGRATING.md" site: "MIGRATING.md — the generated copy of the parse is untouched pending #798" mentions: 0 + - issue: 818 + file: "SPEC.md" + site: "SPEC.md Appendix F — OAuth endpoint address enforcement beyond Go (§16 req 5–6) is pending #818's umbrella" + mentions: 2 - issue: 819 file: "SPEC.md" site: "SPEC.md §23 zero-egress paragraph — the Layer-1 302 test's adapters are pending #819" From 90a528de247cd7c7c90a4f1f6530a1bb8b9df08d Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Thu, 10 Sep 2026 16:27:19 -0700 Subject: [PATCH 49/55] Judge every fixture number literal once, at load, so the schema's all-numeric claim holds for every integer field MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The schema's top-level description promises that drivers judge each numeric literal by its exact mathematical value, so 1000.0 and 1e3 are integer instances a schema-valid fixture may carry. The Go loader honored that only for the ms-converted fields, which went through parseIntegralMs; every other schema integer — a status, an id, a capacity, a close code — decoded straight into int/int64, where encoding/json refuses 200.0 or 1e2 outright. So a fixture the schema accepts was non-portable by this driver alone, and each new integer field would have needed its own number type to opt in. Replace the per-field parser with one walk over the whole document before any typed decode: decode with UseNumber, judge every literal as text (the 100000-character cap, the exponent bound, the digit-place integrality and magnitude judgments, then big.Rat for the exact value), rewrite each integral literal to its plain integer spelling, and hand the typed script the result. The walk's magnitude cap is int64, the widest integer any schema field decodes into; the ms fields' 10-year range stays a value check after decode, since its floor differs by field. optionalMs and scenarioMs keep only what the walk cannot decide for them: the three JSON states and the refusal of a quoted number as the string instance it is. A refusal names the member path. The existing ms-spelling table keeps every case; its out-of-range spellings are now named by the walk's bound rather than by "ms value". A new table loads a status, an id and a capacity spelled 200.0 / 1e2 / 1.0 (and 2000e-1, 0.1e3) equal to their integer-spelled twin, and refuses one non-integral literal in each position naming the member. --- .../eventfeed/scenario_fixture_test.go | 310 +++++++++++------- .../eventfeed/scenario_selftest_test.go | 68 +++- 2 files changed, 251 insertions(+), 127 deletions(-) diff --git a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go index a949d9f74..36e3e6dc3 100644 --- a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go @@ -69,7 +69,11 @@ type scenarioConfig struct { // distinguishes: absent (the zero optionalMs — use the default), JSON null // (set, null — rejected, "type": "integer" refuses it), and a value (set, // ranged). encoding/json calls a value type's UnmarshalJSON for null where it -// short-circuits a pointer's, which is exactly why this is not a *int64. +// short-circuits a pointer's, which is exactly why this is not a *int64. The +// number itself arrives already judged and rewritten to its integer spelling +// by normalizeNumbers, so only the STRING gate remains here: a quoted "1000" +// is a string instance the schema refuses, though json.Number's own +// Unmarshal would take it. type optionalMs struct { set bool null bool @@ -82,133 +86,24 @@ func (o *optionalMs) UnmarshalJSON(data []byte) error { o.null = true return nil } - v, err := parseIntegralMs(data) - o.v = v - return err -} - -// parseIntegralMs parses one JSON number the way draft 2020-12's "integer" -// judges it: by MATHEMATICAL value, not spelling — 1000.0 and 1e3 are integer -// instances a schema-valid fixture may carry, and only this driver was -// refusing them (the float-spelled-int class FlexInt absorbs on the -// rich-text lane). Integrality is a fact about the TEXT, which json.Number -// preserves: a float64 detour rounds 1000.00000000000001 to exactly 1000 — -// and 315575999999.99999 to exactly the maximum — before any check can look, -// so the literal is judged exactly, with big.Rat. Two gates come first: a -// quoted "1000" is a STRING instance the schema refuses, though json.Number's -// own Unmarshal would take it; and an exponent is read as a NUMBER'S text -// before anything is materialized, so an exponent bomb (1e999999999) is -// refused for its magnitude, never expanded. -func parseIntegralMs(data []byte) (int64, error) { + return unmarshalScenarioInt(data, &o.v) +} + +// unmarshalScenarioInt decodes one already-normalized integer literal, +// refusing a string instance in the schema's own terms. +func unmarshalScenarioInt(data []byte, dst *int64) error { trimmed := strings.TrimLeft(string(data), " \t\r\n") if strings.HasPrefix(trimmed, `"`) { - return 0, fmt.Errorf("%s is a string: the schema's type is integer — quote-wrapping a number makes it a different instance", trimmed) - } - var n json.Number - if err := json.Unmarshal(data, &n); err != nil { - return 0, err - } - lit := n.String() - // A bomb is refused before anything is materialized, and by the value's - // EFFECTIVE magnitude, never the exponent's spelling alone — draft - // 2020-12 constrains the mathematical value, and a significand can - // offset any exponent (1e44-digits × e-41 is exactly 1000). Two string - // judgments suffice, both exact: - // - the most significant nonzero digit's decimal place caps the - // value: above place 13 nothing fits [0, maxScenarioMs] (12 digits); - // - a least significant nonzero digit below the units place makes the - // value non-integral outright (decimal digits do not carry), which - // refuses 1e-999999999 without a 10^999999999 denominator. - // A length cap comes first so no multi-megabyte literal is ever walked - // into a rational — the schema's top-level description sanctions exactly - // this bound (a resource limit on spellings, not a value constraint), so - // refusing "1" + 100k zeros + e-100000 (mathematically 1) is conformant. - if len(lit) > 100000 { - return 0, fmt.Errorf("a %d-character number is beyond any modeled ms value (literals are capped at 100000 characters)", len(lit)) - } - mant, expText := lit, "" - if i := strings.IndexAny(lit, "eE"); i >= 0 { - mant = lit[:i] - expText = strings.TrimPrefix(lit[i+1:], "+") - } - digits := strings.TrimPrefix(mant, "-") - point := strings.IndexByte(digits, '.') - intLen := len(digits) - if point >= 0 { - intLen = point - digits = digits[:point] + digits[point+1:] - } - firstNZ, lastNZ := -1, -1 - for i := 0; i < len(digits); i++ { - if digits[i] >= '1' && digits[i] <= '9' { - if firstNZ < 0 { - firstNZ = i - } - lastNZ = i - } + return fmt.Errorf("%s is a string: the schema's type is integer — quote-wrapping a number makes it a different instance", trimmed) } - if firstNZ < 0 { - // Zero, however spelled (0, 0.000, 0e200001): integral, the range - // judgment's to refuse, and decided before the exponent is even - // parsed — an exponent multiplies a significand, and this one is - // zero. - return 0, nil - } - exp := 0 - if expText != "" { - // ParseInt at a FIXED width, not Atoi: int is 32 bits on some - // targets, where an exponent like 1e9223372036854775807 would fail - // as unreadable before reaching the magnitude judgment and change - // the diagnostic by platform. A 64-bit overflow can only mean the - // exponent is beyond the ±200000 bound below, so range errors take - // the bound's own verdict. - e, err := strconv.ParseInt(expText, 10, 64) - if err != nil { - if errors.Is(err, strconv.ErrRange) { - return 0, fmt.Errorf("%s is beyond any modeled ms value", lit) - } - return 0, fmt.Errorf("%s is not a number this driver can read", lit) - } - // Bound the exponent before any place arithmetic: at the platform's - // integer extremes, intLen - firstNZ + exp wraps and the magnitude - // judgments below judge garbage. The literal cap above bounds the - // significand at 100000 digits, so no in-range value needs an - // exponent beyond ±200000 to spell. - if e > 200000 || e < -200000 { - return 0, fmt.Errorf("%s is beyond any modeled ms value", lit) - } - exp = int(e) - } - { - // Digit i occupies decimal place intLen - i + exp (units = 1). - if msd := intLen - firstNZ + exp; msd > 13 { - return 0, fmt.Errorf("%s is beyond any modeled ms value", lit) - } - if lsd := intLen - lastNZ + exp; lsd < 1 { - return 0, fmt.Errorf("%s is not an integer: the schema's type is integer — a number whose mathematical value is integral", lit) - } - } - r, ok := new(big.Rat).SetString(lit) - if !ok { - return 0, fmt.Errorf("%s is not a number this driver can read", lit) - } - if !r.IsInt() { - return 0, fmt.Errorf("%s is not an integer: the schema's type is integer — a number whose mathematical value is integral", lit) - } - num := r.Num() - if !num.IsInt64() { - return 0, fmt.Errorf("%s is beyond any modeled ms value", lit) - } - return num.Int64(), nil + return json.Unmarshal(data, dst) } // scenarioMs is a required ms value under the same number model. type scenarioMs int64 func (m *scenarioMs) UnmarshalJSON(data []byte) error { - v, err := parseIntegralMs(data) - *m = scenarioMs(v) - return err + return unmarshalScenarioInt(data, (*int64)(m)) } // storeScript is the schema's scripted CheckpointStore. @@ -465,6 +360,10 @@ func parseScenario(raw []byte, file string) (*scenario, error) { if err := rejectDuplicateKeys(raw); err != nil { return nil, err } + raw, err := normalizeNumbers(raw) + if err != nil { + return nil, err + } top, err := objectKeys(raw) if err != nil { return nil, err @@ -1116,6 +1015,179 @@ func rejectDuplicateKeys(raw []byte) error { } } +// normalizeNumbers rewrites every number literal in raw to its integer +// spelling, judging each the way draft 2020-12's "integer" does: by +// MATHEMATICAL value, not spelling — 1000.0, 1e3 and 200.0 are integer +// instances a schema-valid fixture may carry, and the schema's every number +// is an integer. One walk over the whole document, before any typed decode, +// is what makes the schema's all-numeric cross-driver claim true for every +// field at once: a status, an id, a capacity and a duration are all judged +// here, and the typed script then reads plain integer spellings — so no +// field needs its own number type, and adding a field cannot reopen the +// class. Integrality is a fact about the TEXT, which json.Number preserves: +// a float64 detour rounds 1000.00000000000001 to exactly 1000 before any +// check can look, so the literal is judged exactly, with big.Rat. An +// exponent is read as a NUMBER'S text before anything is materialized, so an +// exponent bomb (1e999999999) is refused for its magnitude, never expanded. +// Strings pass through untouched: a quoted "1000" stays the STRING instance +// the schema refuses, for the typed decode to reject. +func normalizeNumbers(raw []byte) ([]byte, error) { + dec := json.NewDecoder(bytes.NewReader(raw)) + dec.UseNumber() + var doc any + if err := dec.Decode(&doc); err != nil { + return nil, err + } + if dec.More() { + return nil, fmt.Errorf("trailing JSON content") + } + if err := integralizeNumbers(doc, "fixture"); err != nil { + return nil, err + } + var buf bytes.Buffer + enc := json.NewEncoder(&buf) + enc.SetEscapeHTML(false) + if err := enc.Encode(doc); err != nil { + return nil, err + } + return buf.Bytes(), nil +} + +// integralizeNumbers rewrites, in place, every json.Number under doc to its +// integer spelling, naming the member path in any refusal. +func integralizeNumbers(doc any, path string) error { + switch v := doc.(type) { + case map[string]any: + for key, member := range v { + at := path + "." + key + if n, ok := member.(json.Number); ok { + lit, err := integerSpelling(n.String()) + if err != nil { + return fmt.Errorf("%s: %w", at, err) + } + v[key] = json.Number(lit) + continue + } + if err := integralizeNumbers(member, at); err != nil { + return err + } + } + case []any: + for i, member := range v { + at := fmt.Sprintf("%s[%d]", path, i) + if n, ok := member.(json.Number); ok { + lit, err := integerSpelling(n.String()) + if err != nil { + return fmt.Errorf("%s: %w", at, err) + } + v[i] = json.Number(lit) + continue + } + if err := integralizeNumbers(member, at); err != nil { + return err + } + } + case json.Number: + // A bare top-level number is not a fixture; the typed decode says so. + } + return nil +} + +// integerSpelling answers the plain decimal integer spelling of one JSON +// number literal, or why it has none. A bomb is refused before anything is +// materialized, and by the value's EFFECTIVE magnitude, never the exponent's +// spelling alone — draft 2020-12 constrains the mathematical value, and a +// significand can offset any exponent (1e44-digits × e-41 is exactly 1000). +// Two string judgments suffice, both exact: +// - the most significant nonzero digit's decimal place caps the value: +// above place 19 nothing fits int64, the widest integer any schema +// field decodes into (the ms fields' 10-year range is judged after, on +// the value, since its floor differs by field); +// - a least significant nonzero digit below the units place makes the +// value non-integral outright (decimal digits do not carry), which +// refuses 1e-999999999 without a 10^999999999 denominator. +// +// A length cap comes first so no multi-megabyte literal is ever walked into +// a rational — the schema's top-level description sanctions exactly this +// bound (a resource limit on spellings, not a value constraint), so refusing +// "1" + 100k zeros + e-100000 (mathematically 1) is conformant. +func integerSpelling(lit string) (string, error) { + if len(lit) > 100000 { + return "", fmt.Errorf("a %d-character number is beyond any modeled value (literals are capped at 100000 characters)", len(lit)) + } + mant, expText := lit, "" + if i := strings.IndexAny(lit, "eE"); i >= 0 { + mant = lit[:i] + expText = strings.TrimPrefix(lit[i+1:], "+") + } + digits := strings.TrimPrefix(mant, "-") + point := strings.IndexByte(digits, '.') + intLen := len(digits) + if point >= 0 { + intLen = point + digits = digits[:point] + digits[point+1:] + } + firstNZ, lastNZ := -1, -1 + for i := 0; i < len(digits); i++ { + if digits[i] >= '1' && digits[i] <= '9' { + if firstNZ < 0 { + firstNZ = i + } + lastNZ = i + } + } + if firstNZ < 0 { + // Zero, however spelled (0, 0.000, 0e200001): integral, and decided + // before the exponent is even parsed — an exponent multiplies a + // significand, and this one is zero. + return "0", nil + } + exp := 0 + if expText != "" { + // ParseInt at a FIXED width, not Atoi: int is 32 bits on some + // targets, where an exponent like 1e9223372036854775807 would fail + // as unreadable before reaching the magnitude judgment and change + // the diagnostic by platform. A 64-bit overflow can only mean the + // exponent is beyond the ±200000 bound below, so range errors take + // the bound's own verdict. + e, err := strconv.ParseInt(expText, 10, 64) + if err != nil { + if errors.Is(err, strconv.ErrRange) { + return "", fmt.Errorf("%s is beyond any modeled value", lit) + } + return "", fmt.Errorf("%s is not a number this driver can read", lit) + } + // Bound the exponent before any place arithmetic: at the platform's + // integer extremes, intLen - firstNZ + exp wraps and the magnitude + // judgments below judge garbage. The literal cap above bounds the + // significand at 100000 digits, so no in-range value needs an + // exponent beyond ±200000 to spell. + if e > 200000 || e < -200000 { + return "", fmt.Errorf("%s is beyond any modeled value", lit) + } + exp = int(e) + } + // Digit i occupies decimal place intLen - i + exp (units = 1). + if msd := intLen - firstNZ + exp; msd > 19 { + return "", fmt.Errorf("%s is beyond any modeled value", lit) + } + if lsd := intLen - lastNZ + exp; lsd < 1 { + return "", fmt.Errorf("%s is not an integer: the schema's type is integer — a number whose mathematical value is integral", lit) + } + r, ok := new(big.Rat).SetString(lit) + if !ok { + return "", fmt.Errorf("%s is not a number this driver can read", lit) + } + if !r.IsInt() { + return "", fmt.Errorf("%s is not an integer: the schema's type is integer — a number whose mathematical value is integral", lit) + } + num := r.Num() + if !num.IsInt64() { + return "", fmt.Errorf("%s is beyond any modeled value", lit) + } + return num.String(), nil +} + // decodeStrict decodes exactly one JSON value into dst, rejecting unknown // object keys and trailing content. func decodeStrict(raw json.RawMessage, dst any) error { diff --git a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go index e4e4e15c2..796668d72 100644 --- a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go @@ -17,6 +17,7 @@ import ( "net/http" "os" "path/filepath" + "reflect" "strings" "testing" "time" @@ -666,8 +667,10 @@ func TestScenarioDriverRejectsUnmatchedActions(t *testing.T) { // MATHEMATICAL value is integral, so 1000.0 and 1e3 are integer instances a // schema-valid fixture may carry, and only the Go driver was refusing them — // the same float-spelled-int class FlexInt absorbs on the rich-text lane. -// Integrality is decidable without precision loss: everything in range sits -// far below float64's 2^53 exact-integer ceiling. +// The literal is judged exactly, as text, by the one normalization walk the +// loader runs before any typed decode; the ms fields' 10-year range is then +// the value check's verdict, which is why an out-of-range spelling here is +// named by the walk's int64 bound and an in-range-but-zero one by the range. func TestScenarioMsAcceptsIntegralNumberSpellings(t *testing.T) { base := `{"name":"x","description":"d","config":{"stalenessMs":%s},"steps":[{"advance":{"ms":%s}}],"finally":{"state":"closed"}}` cases := []struct { @@ -690,21 +693,21 @@ func TestScenarioMsAcceptsIntegralNumberSpellings(t *testing.T) { // A bomb is refused by its EFFECTIVE magnitude — significand length // plus exponent — never by materializing the number, and never by // the exponent's spelling alone: a significand can offset it. - {"exponent bomb", "1e999999999", "1000", "beyond any modeled ms value", 0}, + {"exponent bomb", "1e999999999", "1000", "beyond any modeled value", 0}, {"significand offsets a negative exponent", "100000000000000000000000000000000000000000000e-41", "1000", "", 1000}, {"significand offsets to one hundred", "1000000000000000000000000000000000000000000e-40", "1000", "", 100}, {"fraction-led spelling of one", "0.00000000000000000000000000000000000000001e41", "1000", "", 1}, - {"negative exponent bomb", "1e-999999999", "1000", "beyond any modeled ms value", 0}, + {"negative exponent bomb", "1e-999999999", "1000", "beyond any modeled value", 0}, {"parse-bomb literal", strings.Repeat("9", 100001), "1000", "characters", 0}, // The platform's max integer as an exponent must not wrap the place // arithmetic into acceptance. - {"max-int exponent", "1e9223372036854775807", "1000", "beyond any modeled ms value", 0}, - {"min-int exponent", "1e-9223372036854775808", "1000", "beyond any modeled ms value", 0}, + {"max-int exponent", "1e9223372036854775807", "1000", "beyond any modeled value", 0}, + {"min-int exponent", "1e-9223372036854775808", "1000", "beyond any modeled value", 0}, // Exponents past int64 overflow the fixed-width parse itself; the // range error takes the bound's own verdict, so the diagnostic is // the same on every platform width. - {"beyond-int64 exponent", "1e9223372036854775808", "1000", "beyond any modeled ms value", 0}, - {"beyond-int64 negative exponent", "1e-9223372036854775809", "1000", "beyond any modeled ms value", 0}, + {"beyond-int64 exponent", "1e9223372036854775808", "1000", "beyond any modeled value", 0}, + {"beyond-int64 negative exponent", "1e-9223372036854775809", "1000", "beyond any modeled value", 0}, // Zero short-circuits before any exponent expansion; a zero // stalenessMs is then the RANGE check's refusal, not a parse error. {"zero with a large exponent", "0e199999", "1000", "must be in [1,", 0}, @@ -730,6 +733,55 @@ func TestScenarioMsAcceptsIntegralNumberSpellings(t *testing.T) { } } +// TestScenarioIntegerFieldsAcceptIntegralNumberSpellings pins the same +// number model onto EVERY schema integer, not only the ms fields: the +// schema's top-level description promises drivers judge each numeric +// literal by its exact value, and that promise is only true cross-driver if +// a status, an id and a capacity spelled 200.0 / 1e2 / 1.0 load as their +// integer spellings do. One normalization walk over the document is what +// makes it true at once, so no field type has to opt in — and a non-integral +// literal in any of them is refused in the schema's terms, naming the member. +func TestScenarioIntegerFieldsAcceptIntegralNumberSpellings(t *testing.T) { + base := `{"name":"x","description":"d","config":{"liveBufferCapacity":%s},"steps":[{"expectMint":{"respond":{"status":%s,"body":{"ticket":"{{TICKET:1}}","expires_in":120,"url":"{{CABLE_URL:1}}"}}}}],"finally":{"state":"closed","delivered":{"exact":[%s]}}}` + load := func(t *testing.T, capacity, status, id string) (*scenario, error) { + t.Helper() + return parseScenario([]byte(fmt.Sprintf(base, capacity, status, id)), "x.json") + } + want, err := load(t, "1", "200", "100") + if err != nil { + t.Fatalf("the integer spellings must load: %v", err) + } + cases := []struct{ name, capacity, status, id string }{ + {"float spellings", "1.0", "200.0", "100.0"}, + {"exponent spellings", "1e0", "2e2", "1e2"}, + {"mixed spellings", "1.0", "2000e-1", "0.1e3"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + got, err := load(t, tc.capacity, tc.status, tc.id) + if err != nil { + t.Fatalf("an integral spelling must load: %v", err) + } + if !reflect.DeepEqual(got, want) { + t.Errorf("loaded scenario differs from its integer-spelled twin:\n got %+v\nwant %+v", got, want) + } + }) + } + refused := []struct{ name, capacity, status, id, wants string }{ + {"non-integral capacity", "1.5", "200", "100", "config.liveBufferCapacity: 1.5 is not an integer"}, + {"non-integral status", "1", "200.5", "100", "respond.status: 200.5 is not an integer"}, + {"non-integral id", "1", "200", "100.25", "delivered.exact[0]: 100.25 is not an integer"}, + } + for _, tc := range refused { + t.Run(tc.name, func(t *testing.T) { + _, err := load(t, tc.capacity, tc.status, tc.id) + if err == nil || !strings.Contains(err.Error(), tc.wants) { + t.Fatalf("err = %v, want one naming %q", err, tc.wants) + } + }) + } +} + // The control matters as much as the mutants: an advance over a window with // nothing due is ordinary and must still pass, or the guard would be rejecting // every advance and the suite's one real advance (fixture 05) would be failing From e553d74c633cd56aed9f90b2929d0c230565087a Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Tue, 15 Sep 2026 22:10:22 -0700 Subject: [PATCH 50/55] Conformance: carry the shipped event shape into fixture 31, and bound every integer the schema names MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fixture 31's rows take the shipped nine-key poll row and eleven-key push payload the foundations now require. Every integer field in the tier-2 schema carries an explicit, portable ceiling: 9223372036854775807 for ids and epoch positions (§10's 64-bit contract), 2147483647 for counts, capacities and seconds — the narrowest native integer any driver language carries — and 65535 for a close code. The driver refused literals past int64 at load, so a schema-valid fixture could be accepted by an arbitrary-precision driver and refused by Go's; the schema is now the single judge of range, and the drivers enforce what it says rather than their own width. --- ...st-snapshot-straggler-below-served-id.json | 7 ++ conformance/event-feed/schema.json | 94 ++++++++++++------- 2 files changed, 69 insertions(+), 32 deletions(-) diff --git a/conformance/event-feed/fixtures/31-post-snapshot-straggler-below-served-id.json b/conformance/event-feed/fixtures/31-post-snapshot-straggler-below-served-id.json index 12c1ce693..3a98bc7b1 100644 --- a/conformance/event-feed/fixtures/31-post-snapshot-straggler-below-served-id.json +++ b/conformance/event-feed/fixtures/31-post-snapshot-straggler-below-served-id.json @@ -51,6 +51,7 @@ "created_at": "2026-08-01T12:00:00Z", "bucket_id": 2, "creator_id": 3, + "performed_by_id": null, "recording_id": 900 } ], @@ -87,6 +88,8 @@ "created_at": "2026-08-01T12:00:00Z", "bucket_id": 2, "creator_id": 3, + "performed_by_id": null, + "actor_type": "person", "recording_id": 900, "visible_to_clients": false } @@ -111,6 +114,8 @@ "created_at": "2026-08-01T12:00:00Z", "bucket_id": 2, "creator_id": 3, + "performed_by_id": null, + "actor_type": "person", "recording_id": 900, "visible_to_clients": false } @@ -127,6 +132,8 @@ "created_at": "2026-08-01T12:00:00Z", "bucket_id": 2, "creator_id": 3, + "performed_by_id": null, + "actor_type": "person", "recording_id": 900, "visible_to_clients": false } diff --git a/conformance/event-feed/schema.json b/conformance/event-feed/schema.json index ef982d496..34096c841 100644 --- a/conformance/event-feed/schema.json +++ b/conformance/event-feed/schema.json @@ -2,7 +2,7 @@ "$schema": "https://json-schema.org/draft/2020-12/schema", "$id": "https://basecamp.com/schemas/event-feed-scenario.json", "title": "Event Feed Connector Tier-2 Scenario", - "description": "One strictly-ordered interleaved script driving the SPEC §23 connector over its five seams. NUMERIC LITERALS: schema keywords judge mathematical value (1e3 and 1000.0 are integer instances) and drivers judge integrality and range on the literal's exact value — but a driver MAY refuse any numeric literal longer than 100000 characters regardless of value: a sanctioned resource bound on pathological spellings, not a value constraint. Continuing: HTTP exchanges (mint/poll), cable frames, time directives, and observations, in one `steps` array. Strict-matched actions (mint, connect, poll, outbound frame, client close, checkpoint save) must each match their expect step under the per-action-class rules in this family's README ('Strictness semantics, per action class': saves and outbound frames are arrival-strict; mint/poll seam calls are parked and matched in order); observation directives (delivered, buffered, timers, state, signals, handler invocations) are rendezvous assertions evaluated cumulatively under a small wall-clock watchdog while virtual time is frozen. All counts (mintCount, connectCount) count SEAM CALLS — one fully-governed generated call each, with SPEC §7 retries inside — never wire attempts. Time flows only through the injected Clock seam; the virtual-advance algorithm and the ownership-cut/dequeued-frames semantics are normative prose in this family's README and SPEC §23. Validated by `make event-feed-fixtures-check`.", + "description": "One strictly-ordered interleaved script driving the SPEC §23 connector over its five seams. NUMERIC LITERALS: schema keywords judge mathematical value (1e3 and 1000.0 are integer instances) and drivers judge integrality and range on the literal's exact value — but a driver MAY refuse any numeric literal longer than 100000 characters regardless of value: a sanctioned resource bound on pathological spellings, not a value constraint. Every integer field carries an explicit, portable ceiling — 9223372036854775807 for ids and epoch positions (§10's 64-bit contract), 2147483647 for counts, capacities and seconds (the narrowest native integer any driver language carries), 65535 for a close code — so a schema-valid fixture is never one a driver must refuse at its own integer width. Continuing: HTTP exchanges (mint/poll), cable frames, time directives, and observations, in one `steps` array. Strict-matched actions (mint, connect, poll, outbound frame, client close, checkpoint save) must each match their expect step under the per-action-class rules in this family's README ('Strictness semantics, per action class': saves and outbound frames are arrival-strict; mint/poll seam calls are parked and matched in order); observation directives (delivered, buffered, timers, state, signals, handler invocations) are rendezvous assertions evaluated cumulatively under a small wall-clock watchdog while virtual time is frozen. All counts (mintCount, connectCount) count SEAM CALLS — one fully-governed generated call each, with SPEC §7 retries inside — never wire attempts. Time flows only through the injected Clock seam; the virtual-advance algorithm and the ownership-cut/dequeued-frames semantics are normative prose in this family's README and SPEC §23. Validated by `make event-feed-fixtures-check`.", "type": "object", "additionalProperties": false, "required": [ @@ -261,7 +261,8 @@ "maxItems": 100, "items": { "type": "integer", - "minimum": 1 + "minimum": 1, + "maximum": 9223372036854775807 }, "description": "Filter: bucket ids, ≤ 100, positive." }, @@ -271,7 +272,8 @@ "maxItems": 100, "items": { "type": "integer", - "minimum": 1 + "minimum": 1, + "maximum": 9223372036854775807 }, "description": "Filter: creator ids, ≤ 100, positive." }, @@ -281,7 +283,8 @@ "maxItems": 100, "items": { "type": "integer", - "minimum": 1 + "minimum": 1, + "maximum": 9223372036854775807 }, "description": "Filter: effective-performer ids, ≤ 100, positive (the server's `self` literal is caller-resolved to an id — SPEC §23 Consumer Surface)." }, @@ -291,7 +294,8 @@ "maxItems": 100, "items": { "type": "integer", - "minimum": 1 + "minimum": 1, + "maximum": 9223372036854775807 }, "description": "Filter: excluded effective-performer ids, ≤ 100, positive — the loop guard for an acting agent." }, @@ -344,12 +348,14 @@ "liveBufferCapacity": { "type": "integer", "minimum": 1, - "description": "Default 10000 events. Overflow fixtures set a small value so overflow is reachable in a handful of frames." + "description": "Default 10000 events. Overflow fixtures set a small value so overflow is reachable in a handful of frames.", + "maximum": 2147483647 }, "dedupeCapacity": { "type": "integer", "minimum": 1, - "description": "Default 10000 delivered ids. Deliberately decoupled from liveBufferCapacity." + "description": "Default 10000 delivered ids. Deliberately decoupled from liveBufferCapacity.", + "maximum": 2147483647 }, "signalDisposition": { "type": "object", @@ -554,7 +560,8 @@ "expires_in": { "type": "integer", "minimum": 1, - "description": "Server-owned (~120); NEVER used for client scheduling." + "description": "Server-owned (~120); NEVER used for client scheduling.", + "maximum": 2147483647 }, "url": { "type": "string", @@ -682,7 +689,8 @@ }, "message": { "type": "integer", - "description": "Optional epoch payload — both wire ping forms are legal." + "description": "Optional epoch payload — both wire ping forms are legal.", + "maximum": 9223372036854775807 } } }, @@ -787,7 +795,8 @@ "description": "Server-initiated WebSocket close (with a close frame; contrast `sever`).", "properties": { "code": { - "type": "integer" + "type": "integer", + "maximum": 65535 }, "reason": { "type": "string" @@ -1094,7 +1103,8 @@ }, "epoch_after_id": { "type": "integer", - "minimum": 0 + "minimum": 0, + "maximum": 9223372036854775807 }, "resume": { "type": "string", @@ -1195,7 +1205,8 @@ "properties": { "id": { "type": "integer", - "minimum": 1 + "minimum": 1, + "maximum": 9223372036854775807 }, "kind": { "type": "string", @@ -1215,22 +1226,26 @@ }, "bucket_id": { "type": "integer", - "minimum": 1 + "minimum": 1, + "maximum": 9223372036854775807 }, "creator_id": { "type": "integer", - "minimum": 1 + "minimum": 1, + "maximum": 9223372036854775807 }, "performed_by_id": { "type": [ "integer", "null" ], - "minimum": 1 + "minimum": 1, + "maximum": 9223372036854775807 }, "recording_id": { "type": "integer", - "minimum": 1 + "minimum": 1, + "maximum": 9223372036854775807 }, "details": { "type": "object", @@ -1258,7 +1273,8 @@ "properties": { "id": { "type": "integer", - "minimum": 1 + "minimum": 1, + "maximum": 9223372036854775807 }, "kind": { "type": "string", @@ -1278,18 +1294,21 @@ }, "bucket_id": { "type": "integer", - "minimum": 1 + "minimum": 1, + "maximum": 9223372036854775807 }, "creator_id": { "type": "integer", - "minimum": 1 + "minimum": 1, + "maximum": 9223372036854775807 }, "performed_by_id": { "type": [ "integer", "null" ], - "minimum": 1 + "minimum": 1, + "maximum": 9223372036854775807 }, "actor_type": { "enum": [ @@ -1299,7 +1318,8 @@ }, "recording_id": { "type": "integer", - "minimum": 1 + "minimum": 1, + "maximum": 9223372036854775807 }, "visible_to_clients": { "type": "boolean" @@ -1327,7 +1347,7 @@ "required": [ "ms" ], - "description": "Advance virtual now by ms. The normative virtual-advance algorithm (README) fires due timers in deadline order, re-evaluating after each fire, with timers scheduled during the advance whose deadlines land inside the window also firing and ties breaking by creation order. That reentrant clause is UNSCRIPTABLE wherever the connector runs concurrently with the driver, so no fixture may rely on it: every driver MUST REJECT an advance whose window would fire ANY timer, naming fireTimer as the deterministic alternative. The test is what would FIRE, decided from the clock's state before time moves — not what gets armed, which happens on the connector's schedule and can only be sampled. An advance that fires nothing never wakes anything, so it cannot cause an arm; that is what makes the check complete rather than probabilistic. A firing that replaces nothing is rejected too. And an advance is deterministic only from a scripted rendezvous point: an action's completion can precede the timer arms its transition causes, so every advance must be the scenario's first step or immediately follow a TWO-STEP rendezvous: expectState, then expectTimers, enforced at load (an empty expectTimers set is rejected with it — it orders nothing). Neither step alone settles: a set match can coincide with a transient mid-surgery set (the welcome transition stops handshake-deadline and arms confirmation-deadline in separate clock acquisitions), and an announcement can precede a tail arm. Together they do: the announcement bounds the surgery, and any timer still unarmed at the announcement is exactly what the exact-set match then waits for. Both block under the watchdog, so wrong authorship fails on every schedule where the stale pair no longer holds \u2014 a pair naming the PRE-action state can pass on the schedule where the action is not yet processed, so a wrong script is at worst flaky, never stably green; the settled guarantee belongs to correctly authored pairs. A transition that announces no state change — or only rearms a timer of the same kind and count — is invisible to this rendezvous, and a script that would advance behind one uses fireTimer instead.", + "description": "Advance virtual now by ms. The normative virtual-advance algorithm (README) fires due timers in deadline order, re-evaluating after each fire, with timers scheduled during the advance whose deadlines land inside the window also firing and ties breaking by creation order. That reentrant clause is UNSCRIPTABLE wherever the connector runs concurrently with the driver, so no fixture may rely on it: every driver MUST REJECT an advance whose window would fire ANY timer, naming fireTimer as the deterministic alternative. The test is what would FIRE, decided from the clock's state before time moves — not what gets armed, which happens on the connector's schedule and can only be sampled. An advance that fires nothing never wakes anything, so it cannot cause an arm; that is what makes the check complete rather than probabilistic. A firing that replaces nothing is rejected too. And an advance is deterministic only from a scripted rendezvous point: an action's completion can precede the timer arms its transition causes, so every advance must be the scenario's first step or immediately follow a TWO-STEP rendezvous: expectState, then expectTimers, enforced at load (an empty expectTimers set is rejected with it — it orders nothing). Neither step alone settles: a set match can coincide with a transient mid-surgery set (the welcome transition stops handshake-deadline and arms confirmation-deadline in separate clock acquisitions), and an announcement can precede a tail arm. Together they do: the announcement bounds the surgery, and any timer still unarmed at the announcement is exactly what the exact-set match then waits for. Both block under the watchdog, so wrong authorship fails on every schedule where the stale pair no longer holds — a pair naming the PRE-action state can pass on the schedule where the action is not yet processed, so a wrong script is at worst flaky, never stably green; the settled guarantee belongs to correctly authored pairs. A transition that announces no state change — or only rearms a timer of the same kind and count — is invisible to this rendezvous, and a script that would advance behind one uses fireTimer instead.", "properties": { "ms": { "type": "integer", @@ -1397,7 +1417,8 @@ }, "additionalProperties": { "type": "integer", - "minimum": 1 + "minimum": 1, + "maximum": 2147483647 } } } @@ -1493,7 +1514,8 @@ "properties": { "epochAfterId": { "type": "integer", - "minimum": 0 + "minimum": 0, + "maximum": 9223372036854775807 }, "resumeUrl": { "type": "string", @@ -1512,7 +1534,8 @@ "properties": { "count": { "type": "integer", - "minimum": 1 + "minimum": 1, + "maximum": 2147483647 } } }, @@ -1536,13 +1559,15 @@ "minItems": 1, "items": { "type": "integer", - "minimum": 1 + "minimum": 1, + "maximum": 9223372036854775807 } }, "droppedCount": { "type": "integer", "minimum": 1, - "description": "The harness asserts droppedCount == droppedIds.length at load — a disagreement fails the fixture, never passes silently." + "description": "The harness asserts droppedCount == droppedIds.length at load — a disagreement fails the fixture, never passes silently.", + "maximum": 2147483647 } } }, @@ -1559,7 +1584,8 @@ }, "epochAfterId": { "type": "integer", - "minimum": 0 + "minimum": 0, + "maximum": 9223372036854775807 }, "resumeUrl": { "type": "string", @@ -1582,7 +1608,8 @@ "type": "array", "items": { "type": "integer", - "minimum": 1 + "minimum": 1, + "maximum": 9223372036854775807 } } } @@ -1652,16 +1679,19 @@ }, "mintCount": { "type": "integer", - "minimum": 0 + "minimum": 0, + "maximum": 2147483647 }, "connectCount": { "type": "integer", - "minimum": 0 + "minimum": 0, + "maximum": 2147483647 }, "pollCount": { "type": "integer", "minimum": 0, - "description": "Poll SEAM calls — one fully-governed generated PollEvents call each, with SPEC §7 retries inside — never wire attempts." + "description": "Poll SEAM calls — one fully-governed generated PollEvents call each, with SPEC §7 retries inside — never wire attempts.", + "maximum": 2147483647 }, "timers": { "$ref": "#/$defs/timerSet" From 67f317f78581645c018ae732667197277830e056 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Tue, 15 Sep 2026 22:28:18 -0700 Subject: [PATCH 51/55] Conformance: retire the tracking entries that pointed at a closed issue Issue 798 closed on 2026-09-14 when #855 landed the generated retry loop's Retry-After clamping, so the registry's two entries for it tracked nothing and the gate this PR ships failed on its own rebase. The entries go, and MIGRATING.md's sentence that leaned on them is restated as the as-of fact it now is. Main's rewording of the two Retry-After sites the registry counted a canonical mention at is recorded as zero mentions. --- MIGRATING.md | 2 +- spec/tracking-issues.yml | 12 ++---------- 2 files changed, 3 insertions(+), 11 deletions(-) diff --git a/MIGRATING.md b/MIGRATING.md index 34d8aaa0d..0e932cfe1 100644 --- a/MIGRATING.md +++ b/MIGRATING.md @@ -834,7 +834,7 @@ the backoff curve; and a delta-seconds above the schedulable ceiling saturates instead of wrapping. The wire operations those paths perform are unchanged, and they already honoured the header on 429 — it is what the header parses to that moved. Typed service methods run the generated retry loop, which has its own -copy of the parse and is untouched (#798). +copy of the parse; its clamping landed with #855 (the defect was #798). **Wrong behaviour you get if you ignore it:** none, but the wait between attempts on a throttled account can now be seconds or minutes where it used to diff --git a/spec/tracking-issues.yml b/spec/tracking-issues.yml index daea0a245..94b15b08a 100644 --- a/spec/tracking-issues.yml +++ b/spec/tracking-issues.yml @@ -46,7 +46,7 @@ prose_tracking_issues: - issue: 775 file: "typescript/tests/retry-after.test.ts" site: "typescript/tests/retry-after.test.ts — pins TypeScript's side of the Retry-After status divergence" - mentions: 1 + mentions: 0 - issue: 775 file: "MIGRATING.md" site: "MIGRATING.md — the Retry-After migration note repeats the status-divergence promise" @@ -98,7 +98,7 @@ prose_tracking_issues: - issue: 799 file: "SPEC.md" site: "SPEC.md §6 — Python and Ruby still truncate the delay; #799 tracks the rounding convergence" - mentions: 1 + mentions: 0 - issue: 799 file: "MIGRATING.md" site: "MIGRATING.md — #799 tracks the cross-SDK convergence on Retry-After rounding" @@ -115,14 +115,6 @@ prose_tracking_issues: file: "go/pkg/basecamp/client_retry_after_test.go" site: "go/pkg/basecamp/client_retry_after_test.go — pins Go's side pending #799's convergence" mentions: 0 - - issue: 798 - file: "SPEC.md" - site: "SPEC.md §6 — the generated Go parser stays Atoi until #798 lands the template change" - mentions: 0 - - issue: 798 - file: "MIGRATING.md" - site: "MIGRATING.md — the generated copy of the parse is untouched pending #798" - mentions: 0 - issue: 818 file: "SPEC.md" site: "SPEC.md Appendix F — OAuth endpoint address enforcement beyond Go (§16 req 5–6) is pending #818's umbrella" From e85766a98cc956a6772d447ebce5bca07e12f511 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Tue, 15 Sep 2026 22:43:23 -0700 Subject: [PATCH 52/55] Conformance: leave an event's details untouched by the integer walk, and decode the ping epoch at 64 bits MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An event's details object is server-owned and forwarded verbatim — the schema types it as an arbitrary object — so its numbers are whatever the server publishes and are judged by no field of this schema. The integer normalization now leaves details subtrees exactly as spelled instead of refusing a fraction or a wide integer the schema permits. The ping frame's optional epoch gains the lower bound the other integer fields already had and is decoded at a fixed 64-bit width, so a schema-valid value loads the same on every target. --- conformance/event-feed/schema.json | 5 +++-- .../eventfeed/scenario_fixture_test.go | 9 ++++++++- .../eventfeed/scenario_selftest_test.go | 19 +++++++++++++++++++ 3 files changed, 30 insertions(+), 3 deletions(-) diff --git a/conformance/event-feed/schema.json b/conformance/event-feed/schema.json index 34096c841..18d24c7a9 100644 --- a/conformance/event-feed/schema.json +++ b/conformance/event-feed/schema.json @@ -689,8 +689,9 @@ }, "message": { "type": "integer", - "description": "Optional epoch payload — both wire ping forms are legal.", - "maximum": 9223372036854775807 + "description": "Optional epoch payload — both wire ping forms are legal. A non-negative 64-bit integer, decoded at that width by every driver.", + "maximum": 9223372036854775807, + "minimum": 0 } } }, diff --git a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go index 36e3e6dc3..2e57bf4f0 100644 --- a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go @@ -183,7 +183,7 @@ type connectOutcome struct { type serveStep struct { Frame string `json:"frame"` - Message *int `json:"message"` + Message *int64 `json:"message"` Identifier *string `json:"identifier"` Reason string `json:"reason"` Reconnect *bool `json:"reconnect"` @@ -1059,6 +1059,13 @@ func integralizeNumbers(doc any, path string) error { switch v := doc.(type) { case map[string]any: for key, member := range v { + if key == "details" { + // An event's details object is server-owned and forwarded + // verbatim — the schema types it as an arbitrary object — so + // its numbers are whatever the server publishes, judged by + // no field of this schema. Left exactly as spelled. + continue + } at := path + "." + key if n, ok := member.(json.Number); ok { lit, err := integerSpelling(n.String()) diff --git a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go index 796668d72..c1052289e 100644 --- a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go @@ -148,6 +148,25 @@ func TestScenarioDriverRejectsMutatedFixtures(t *testing.T) { // witness is retained because the lookahead that computes the current step errs // deliberately permissive (see stepSatisfiedLocked), and it is the check that // still fires if a step's arm is ever loosened. +// TestNormalizeNumbersLeavesDetailsAlone: an event's details object is +// forwarded verbatim and typed by the schema as an arbitrary object, so a +// fraction or an integer past int64 inside it is schema-valid fixture data +// the integer normalization must not judge — every other number in the +// document is an integer field of the schema and is. +func TestNormalizeNumbersLeavesDetailsAlone(t *testing.T) { + raw := []byte(`{"id":1e2,"details":{"ratio":0.5,"big":92233720368547758070,"nested":{"n":1.25}},"count":2.0}`) + out, err := normalizeNumbers(raw) + if err != nil { + t.Fatalf("normalizeNumbers: %v", err) + } + got := string(out) + for _, want := range []string{`"ratio":0.5`, `"big":92233720368547758070`, `"n":1.25`, `"id":100`, `"count":2`} { + if !strings.Contains(got, want) { + t.Errorf("normalized document %s lacks %s", got, want) + } + } +} + func TestScenarioDriverEnforcesDeliveryBeforeCheckpoint(t *testing.T) { const fixture = "02-confirmation-gating.json" control := readFixture(t, fixture) From 6a0413748aaefdd1c61f0f0bfa3d381f5ffd8c39 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Tue, 15 Sep 2026 23:36:12 -0700 Subject: [PATCH 53/55] Conformance: one rule for what the integer walk leaves alone, a close-code range, and two stale sentences MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The integer walk now leaves every opaque subtree exactly as spelled under one stated rule — an event's details object, and the body of a response the driver answers by status alone (every status but the 200 page and the 409 and 410 bodies it decodes) — rather than an exemption per finding. A server-owned fraction in either is schema-valid fixture data the walk has no field to judge it against. A server close code is a WebSocket status in RFC 6455's 1000–4999 range, decoded at a fixed width. The migration note no longer describes the retired two-tier Retry-After rule, the README's rendezvous example no longer attributes an announce-before-arm to Backoff (the Go reference arms first), and §23's buffer proof counts a buffered event's details bytes. --- MIGRATING.md | 13 +++--- SPEC.md | 6 ++- conformance/event-feed/README.md | 4 +- conformance/event-feed/schema.json | 4 +- .../eventfeed/scenario_conformance_test.go | 2 +- .../eventfeed/scenario_fixture_test.go | 43 ++++++++++++++++--- .../eventfeed/scenario_selftest_test.go | 4 +- 7 files changed, 56 insertions(+), 20 deletions(-) diff --git a/MIGRATING.md b/MIGRATING.md index 0e932cfe1..a0ae565cb 100644 --- a/MIGRATING.md +++ b/MIGRATING.md @@ -818,13 +818,12 @@ constructed. They now sleep the server's `Retry-After` — both wire forms, delta-seconds and HTTP-date — in place of the backoff, with no jitter and no ceiling beyond what the host can represent: a value the parser holds but the host cannot schedule saturates at 2147483647 seconds (~68 years) rather than -wrapping negative, and that figure does not vary by architecture. A value too -large for the parser's own `int64` is treated as malformed instead and falls -through to the backoff curve, as it always did. That split is Go's: SPEC §6's -parsing algorithm says only to parse a positive integer, #793 states the -two-tier rule (unrepresentable → malformed, unschedulable → saturate) in §6 -"Retry-After Honouring", and #799 tracks the cross-SDK convergence on -over-range values, which the six SDKs still answer differently. +wrapping negative, and that figure does not vary by architecture. Every +over-ceiling digit string saturates there, including one too large for the +parser's own `int64` — the earlier two-tier rule (unrepresentable → malformed, +unschedulable → saturate) is gone from every Go parser — and #799 tracks the +cross-SDK convergence on over-range values, which the six SDKs still answer +differently. **Two behaviours changed for `DownloadURL` and the rate-limiter hook as well**, because all three paths share `parseRetryAfter`: an HTTP-date's sub-second diff --git a/SPEC.md b/SPEC.md index 12ab7f387..97e540100 100644 --- a/SPEC.md +++ b/SPEC.md @@ -3681,8 +3681,10 @@ Two dispatch clarifications, pinned: retained frame is in one of the three counted structures or in the hands of the pump or the state machine. The live buffer's weight is one per slot: a buffered `Event` retains only the chain's LAST representation — its strings are copies, since Go's - decoder never aliases its input buffer, and `Event` carries no raw-bytes field — so - no transient survives admission. + decoder never aliases its input buffer, and its `details` bytes are the decoder's own + clone of that member (a bounded slice of the frame, retained per slot alongside the + strings; never the frame itself) — so no transient survives admission, and a slot's + weight is the decoded event, details included. The formula is the cable lane's retention, and only that — every counted item is a raw socket frame or a buffered live event. The poll lane sits outside it on purpose: diff --git a/conformance/event-feed/README.md b/conformance/event-feed/README.md index dec5834d3..22f2760dd 100644 --- a/conformance/event-feed/README.md +++ b/conformance/event-feed/README.md @@ -195,7 +195,9 @@ rejected with it — it orders nothing). Neither step alone settles. A set match can coincide with a transient mid-surgery set: the welcome transition stops `handshake-deadline` and arms `confirmation-deadline` in separate clock acquisitions, so an authored set can exist in the gap. An announcement can -precede a tail arm: Backoff announces before its timer is armed. Together they +precede a tail arm in a driver that announces first (the Go reference arms Backoff's +timer before announcing, precisely so the set is already exact — a port need not, and +the rendezvous must not depend on it). Together they settle — the announcement bounds the surgery, and any timer still unarmed at the announcement is exactly what the following exact-set match waits for. Both steps block under the scenario watchdog, so a wrongly authored state or set diff --git a/conformance/event-feed/schema.json b/conformance/event-feed/schema.json index 18d24c7a9..9f27830a7 100644 --- a/conformance/event-feed/schema.json +++ b/conformance/event-feed/schema.json @@ -797,7 +797,9 @@ "properties": { "code": { "type": "integer", - "maximum": 65535 + "maximum": 4999, + "minimum": 1000, + "description": "A WebSocket close status in the RFC 6455 range 1000–4999, decoded at a fixed 64-bit width by every driver." }, "reason": { "type": "string" diff --git a/go/pkg/basecamp/eventfeed/scenario_conformance_test.go b/go/pkg/basecamp/eventfeed/scenario_conformance_test.go index f650189e7..25291f545 100644 --- a/go/pkg/basecamp/eventfeed/scenario_conformance_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_conformance_test.go @@ -402,7 +402,7 @@ func (d *driver) serverClose(step *serverCloseStep) error { if d.peer == nil { return errors.New("no cable connection is open to close") } - return d.h.closePeer(d.peer, step.Code, step.Reason) + return d.h.closePeer(d.peer, int(step.Code), step.Reason) } // sever drops the TCP connection abruptly: no close frame, no disconnect diff --git a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go index 2e57bf4f0..57254a94e 100644 --- a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go @@ -192,7 +192,7 @@ type serveStep struct { } type serverCloseStep struct { - Code int `json:"code"` + Code int64 `json:"code"` Reason string `json:"reason"` } @@ -1053,17 +1053,48 @@ func normalizeNumbers(raw []byte) ([]byte, error) { return buf.Bytes(), nil } +// opaqueSubtree reports whether member, found under key in its parent +// object, is a subtree the driver forwards or ignores without decoding — +// an event's `details`, or a `respond.body` whose status the poll and mint +// seams answer by status alone (every status but the 200 page and the 409 +// and 410 bodies, whose members the driver does decode). The integer walk +// leaves such a subtree as spelled: the schema types it as an arbitrary +// object, so its numbers are the server's to spell. +func opaqueSubtree(key string, member any, parent map[string]any) bool { + if key == "details" { + return true + } + if key != "body" { + return false + } + if _, isObject := member.(map[string]any); !isObject { + return false + } + status, ok := parent["status"].(json.Number) + if !ok { + // A bodied respond with no status is the 200 page, which the driver + // decodes. + return false + } + switch status.String() { + case "200", "409", "410": + return false + } + return true +} + // integralizeNumbers rewrites, in place, every json.Number under doc to its // integer spelling, naming the member path in any refusal. func integralizeNumbers(doc any, path string) error { switch v := doc.(type) { case map[string]any: for key, member := range v { - if key == "details" { - // An event's details object is server-owned and forwarded - // verbatim — the schema types it as an arbitrary object — so - // its numbers are whatever the server publishes, judged by - // no field of this schema. Left exactly as spelled. + if opaqueSubtree(key, member, v) { + // Server-owned or never-decoded: an event's details object, + // and the body of a response the driver forwards as a status + // without reading it. Their numbers are whatever the server + // publishes, judged by no field of this schema, and are left + // exactly as spelled. continue } at := path + "." + key diff --git a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go index c1052289e..ae9458ac0 100644 --- a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go @@ -154,13 +154,13 @@ func TestScenarioDriverRejectsMutatedFixtures(t *testing.T) { // the integer normalization must not judge — every other number in the // document is an integer field of the schema and is. func TestNormalizeNumbersLeavesDetailsAlone(t *testing.T) { - raw := []byte(`{"id":1e2,"details":{"ratio":0.5,"big":92233720368547758070,"nested":{"n":1.25}},"count":2.0}`) + raw := []byte(`{"id":1e2,"details":{"ratio":0.5,"big":92233720368547758070,"nested":{"n":1.25}},"count":2.0,"respond":{"status":500,"body":{"retry_factor":0.5}},"page":{"status":200,"body":{"position":"p","events":[{"id":1e1}]}}}`) out, err := normalizeNumbers(raw) if err != nil { t.Fatalf("normalizeNumbers: %v", err) } got := string(out) - for _, want := range []string{`"ratio":0.5`, `"big":92233720368547758070`, `"n":1.25`, `"id":100`, `"count":2`} { + for _, want := range []string{`"ratio":0.5`, `"big":92233720368547758070`, `"n":1.25`, `"id":100`, `"count":2`, `"retry_factor":0.5`, `"id":10`} { if !strings.Contains(got, want) { t.Errorf("normalized document %s lacks %s", got, want) } From 656dcab9b3d352e320d578bec3f6672793a509ee Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Wed, 16 Sep 2026 00:18:49 -0700 Subject: [PATCH 54/55] Conformance: the opaque rule judges the status by value, capacities get a resource ceiling, and close codes are the sendable ones MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The opaque-subtree rule compared a status literal's spelling, and map iteration could reach a body before the walk had rewritten the `200.0` beside it; it now judges the status by value. The two capacities are capped at a million — a driver may allocate them eagerly, so the largest portable integer was never a practical ceiling. A server close code is one a WebSocket stack will actually frame: 1000–1003, 1007–1011, or 3000–4999. --- conformance/event-feed/schema.json | 26 ++++++++++++++----- .../eventfeed/scenario_fixture_test.go | 9 ++++++- .../eventfeed/scenario_selftest_test.go | 2 +- 3 files changed, 29 insertions(+), 8 deletions(-) diff --git a/conformance/event-feed/schema.json b/conformance/event-feed/schema.json index 9f27830a7..7f33445ad 100644 --- a/conformance/event-feed/schema.json +++ b/conformance/event-feed/schema.json @@ -348,14 +348,14 @@ "liveBufferCapacity": { "type": "integer", "minimum": 1, - "description": "Default 10000 events. Overflow fixtures set a small value so overflow is reachable in a handful of frames.", - "maximum": 2147483647 + "description": "Default 10000 events. Overflow fixtures set a small value so overflow is reachable in a handful of frames. Capped at 1,000,000 — a resource ceiling, since a driver may allocate the capacity eagerly.", + "maximum": 1000000 }, "dedupeCapacity": { "type": "integer", "minimum": 1, - "description": "Default 10000 delivered ids. Deliberately decoupled from liveBufferCapacity.", - "maximum": 2147483647 + "description": "Default 10000 delivered ids. Deliberately decoupled from liveBufferCapacity. Capped at 1,000,000 — a resource ceiling, since a driver may allocate the capacity eagerly.", + "maximum": 1000000 }, "signalDisposition": { "type": "object", @@ -797,9 +797,23 @@ "properties": { "code": { "type": "integer", - "maximum": 4999, "minimum": 1000, - "description": "A WebSocket close status in the RFC 6455 range 1000–4999, decoded at a fixed 64-bit width by every driver." + "maximum": 4999, + "anyOf": [ + { + "minimum": 1000, + "maximum": 1003 + }, + { + "minimum": 1007, + "maximum": 1011 + }, + { + "minimum": 3000, + "maximum": 4999 + } + ], + "description": "A WebSocket close status a server can actually send in a close frame: the standard 1000–1003 and 1007–1011, or the registered/private 3000–4999 range (RFC 6455 §7.4). The reserved 1004–1006 and 1012–2999 are not sendable and a WebSocket stack refuses to frame them. Decoded at a fixed 64-bit width by every driver." }, "reason": { "type": "string" diff --git a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go index 57254a94e..65e21ba82 100644 --- a/go/pkg/basecamp/eventfeed/scenario_fixture_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_fixture_test.go @@ -1076,7 +1076,14 @@ func opaqueSubtree(key string, member any, parent map[string]any) bool { // decodes. return false } - switch status.String() { + // Judged on the status's VALUE, not its spelling: map iteration may + // reach the body before the walk has rewritten a `200.0` beside it, and + // a literal the walk will refuse anyway is not opaque either way. + lit, err := integerSpelling(status.String()) + if err != nil { + return false + } + switch lit { case "200", "409", "410": return false } diff --git a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go index ae9458ac0..4a7c2cda8 100644 --- a/go/pkg/basecamp/eventfeed/scenario_selftest_test.go +++ b/go/pkg/basecamp/eventfeed/scenario_selftest_test.go @@ -154,7 +154,7 @@ func TestScenarioDriverRejectsMutatedFixtures(t *testing.T) { // the integer normalization must not judge — every other number in the // document is an integer field of the schema and is. func TestNormalizeNumbersLeavesDetailsAlone(t *testing.T) { - raw := []byte(`{"id":1e2,"details":{"ratio":0.5,"big":92233720368547758070,"nested":{"n":1.25}},"count":2.0,"respond":{"status":500,"body":{"retry_factor":0.5}},"page":{"status":200,"body":{"position":"p","events":[{"id":1e1}]}}}`) + raw := []byte(`{"id":1e2,"details":{"ratio":0.5,"big":92233720368547758070,"nested":{"n":1.25}},"count":2.0,"respond":{"status":500,"body":{"retry_factor":0.5}},"page":{"status":200.0,"body":{"position":"p","events":[{"id":1e1}]}}}`) out, err := normalizeNumbers(raw) if err != nil { t.Fatalf("normalizeNumbers: %v", err) From 358b89e94a53d05cbe28876a52d47c63426c2c43 Mon Sep 17 00:00:00 2001 From: Jeremy Daer Date: Wed, 16 Sep 2026 00:56:46 -0700 Subject: [PATCH 55/55] =?UTF-8?q?Conformance:=20the=20sendable=20close=20c?= =?UTF-8?q?odes=20are=20the=20IANA=20registry's,=201012=E2=80=931014=20inc?= =?UTF-8?q?luded?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- conformance/event-feed/schema.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/conformance/event-feed/schema.json b/conformance/event-feed/schema.json index 7f33445ad..d2b473e06 100644 --- a/conformance/event-feed/schema.json +++ b/conformance/event-feed/schema.json @@ -806,14 +806,14 @@ }, { "minimum": 1007, - "maximum": 1011 + "maximum": 1014 }, { "minimum": 3000, "maximum": 4999 } ], - "description": "A WebSocket close status a server can actually send in a close frame: the standard 1000–1003 and 1007–1011, or the registered/private 3000–4999 range (RFC 6455 §7.4). The reserved 1004–1006 and 1012–2999 are not sendable and a WebSocket stack refuses to frame them. Decoded at a fixed 64-bit width by every driver." + "description": "A WebSocket close status a server can actually send in a close frame, per the IANA WebSocket Close Code Number registry: the standard 1000–1003 and 1007–1014 (RFC 6455 §7.4.1 plus the registered 1012 Service Restart, 1013 Try Again Later, 1014 Bad Gateway), or the registered/private 3000–4999 range. The reserved 1004–1006 and 1015 are never sent in a frame, and 1016–2999 are unassigned. Decoded at a fixed 64-bit width by every driver." }, "reason": { "type": "string"